Skip to content

ci: read custom runner labels from repo variables - #3573

Merged
kodiakhq[bot] merged 6 commits into
mainfrom
claude/runner-label-vars
Nov 5, 2026
Merged

kodiakhq[bot] merged 6 commits into
mainfrom
claude/runner-label-vars

Conversation

@teeohhem

@teeohhem teeohhem commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

E2E and the image-build jobs run on org-level larger runners (8-core-playwright, Large-Runner-x64-32, Large-Runner-ARM64-32, ubuntu-latest-arm64). Those don't move with the repo when it transfers to the ClickHouse org. If nothing in the new org has those labels, the jobs don't fail; they sit queued waiting for a runner.

This reads each label from a repo variable and falls back to today's label, so nothing changes until a variable is set. A value is either a plain label (ubuntu-24.04) or a JSON array of labels (["self-hosted","linux","x64"]), so the variables can also target self-hosted runners that are selected by a label set. On transfer day we can point the jobs at whatever runners ClickHouse has, or at standard GitHub runners, from Settings → Secrets and variables → Actions → Variables, without a PR.

Variable Default Used by
RUNNER_E2E 8-core-playwright e2e-tests.yml (every PR)
RUNNER_LARGE_X64 Large-Runner-x64-32 Image builds in release.yml, release-nightly.yml
RUNNER_LARGE_ARM64 Large-Runner-ARM64-32 Image builds in release.yml, release-nightly.yml
RUNNER_ARM64 ubuntu-latest-arm64 arm64 build in release.yml, release-nightly.yml
E2E_WORKERS unset (Playwright config default, 4 on CI) e2e-tests.yml

E2E_WORKERS is already read by playwright.config.ts. It's passed through because 4 workers timed out on the 4-vCPU runner before; set it to 2 if RUNNER_E2E points at a smaller machine.

Standard-runner fallback: RUNNER_E2E=ubuntu-24.04 + E2E_WORKERS=2, RUNNER_LARGE_X64=ubuntu-24.04, RUNNER_LARGE_ARM64=ubuntu-24.04-arm, RUNNER_ARM64=ubuntu-24.04-arm. Image builds will be slower.

No changeset: CI only.

How to reproduce

  1. On this branch with no variables set, the E2E jobs on this PR run on 8-core-playwright as before (check the "Set up job" log for the runner name).
  2. Optional: set RUNNER_E2E=ubuntu-24.04 and E2E_WORKERS=2, re-run E2E, and confirm it picks up the standard runner and 2 workers. Unset both afterwards.

Not run: actionlint. I checked that the YAML parses and passes Prettier. vars is a documented context for both runs-on and strategy; the first CI run on this PR confirms the E2E path, and the release paths only run on main/nightly.

🤖 Generated with Claude Code

The E2E and image-build jobs run on org-level larger runners that don't
move with the repo. Reading the labels from repo variables, defaulting to
today's labels, lets us point them at whatever runners the new org has
without a PR on transfer day. E2E_WORKERS is passed through so a smaller
E2E runner can drop the Playwright worker count the same way.
@teeohhem
teeohhem requested a review from a team as a code owner November 5, 2026 18:11
@changeset-bot

changeset-bot Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 071e024

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions Bot added the review/tier-1 Trivial — auto-merge candidate once CI passes label Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🔴 Tier 4 — Critical

Touches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI.

Why this tier:

  • Background tasks or delivery pipeline substantially modified — 32 lines (bar: 30):
    • .github/workflows/release.yml

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Production files changed: 0
  • Production lines changed: 0
  • Critical-path lines changed: 32
  • Branch: claude/runner-label-vars
  • Author: teeohhem

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

✅ No issues found.

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 110 in total (1.22%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This is a CI-only change making GitHub Actions runner labels configurable via repo variables with fallback to the current hardcoded labels. The ${{ startsWith(X, '[') && fromJSON(X) || X || 'default' }} idiom was verified correct on every normal path: unset/empty resolves to the hardcoded default, a plain label passes through, and a JSON array parses into a label set. &&/|| precedence, vars availability in strategy.matrix and runs-on, and the folded multi-line runs-on scalar are all valid. The E2E_WORKERS: ${{ vars.E2E_WORKERS }} passthrough sets an empty string when unset, which packages/app/playwright.config.ts:48 handles correctly (raw ? Number(raw) : NaN), so it falls back to the default workers.

🟡 P2 -- recommended

  • .github/workflows/release.yml:810 -- The new matrix.runner/fromJSON runner expressions in release.yml and release-nightly.yml evaluate only on main/nightly triggers, so the override and JSON-array branches are unexercised by any PR-triggered run and a bad RUNNER_* value (e.g. malformed JSON) would first surface at release time.
    • Fix: Validate a configured RUNNER_* value through a workflow_dispatch smoke run before the first release that relies on it.
    • reliability, correctness
🔵 P3 nitpicks (1)
  • .github/workflows/e2e-tests.yml:14 -- The new RUNNER_E2E, RUNNER_LARGE_X64, RUNNER_LARGE_ARM64, RUNNER_ARM64, and E2E_WORKERS variables are discoverable only by reading inline comments across three workflow files, with no central listing of the variables and their expected format.
    • Fix: List the runner variables and their plain-label-or-JSON-array format in a CI/contributing doc so a maintainer can find the knobs on transfer day.

Reviewers (3): correctness, reliability, project-standards.

Testing gaps:

  • Only the unset-fallback path runs in this PR's CI; the plain-label and JSON-array override branches are exercised only once the repo variables are set post-transfer.
  • No PR-triggered workflow references the release.yml/release-nightly.yml jobs, so expression regressions there are caught only at release/nightly runtime.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 462 passed • 1 skipped • 1440s

Status Count
✅ Passed 462
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@vercel

vercel Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 5, 2026 9:27pm UTC
hyperdx-storybook Ready Ready Preview Nov 5, 2026 9:27pm UTC

Request Review

Self-hosted runners in the ClickHouse org are targeted by label sets
like [dataplane, self-hosted, linux, x64, xlarge], which a single-label
variable can't express. The variables now hold JSON: a quoted label or
an array.
Comment thread .github/workflows/e2e-tests.yml Resolved
fromJSON on a plain label like 8-core-playwright errors, so the E2E
jobs were never created once the variables were set without quotes.
Parse the value as JSON only when it is an array.
@github-actions github-actions Bot added review/tier-4 Critical — deep review + domain expert sign-off and removed review/tier-1 Trivial — auto-merge candidate once CI passes labels Nov 5, 2026
Comment thread .github/workflows/e2e-tests.yml Resolved
Comment thread .github/workflows/e2e-tests.yml Resolved
@kodiakhq
kodiakhq Bot merged commit a044642 into main Nov 5, 2026
28 checks passed
@kodiakhq
kodiakhq Bot deleted the claude/runner-label-vars branch November 5, 2026 21:36

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 071e0242 Deployed Nov 5, 2026 by vercel[bot]
Preview – hyperdx-storybook — 071e0242 Deployed Nov 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants