Ci optimization - #6
Draft
spacebear21 wants to merge 10 commits into
Draft
Conversation
Every push to a same-repo pull request branch triggered both a push run and a pull_request run of the test, format, and flake check workflows, doubling CI load for no extra signal. The duplicated runs saturate the runner concurrency pool and queue other workflows behind them; job start delays of up to nine minutes were observed on the C# native build matrix. Restrict push events to master. Pull requests keep their pull_request runs, and master pushes still produce the Test, Lint, and Format check runs that the release workflow's wait-for-ci step polls for. Release tags are verified to be ancestors of origin/master before publishing, so tagged commits always carry those check runs.
Both format.yml and rust.yml were named "Continuous integration", so the run list showed two indistinguishable workflows and anything keyed on the workflow name, such as the concurrency groups introduced next, would conflate them. Branch protection and the release workflow match on the Format job name, which is unchanged.
Without concurrency groups, pushing a new commit to a pull request leaves the previous commit's runs executing to completion. Those obsolete runs occupy the shared runner pool and queue the new runs and other workflows behind them. Group runs per workflow and pull request and cancel the old run when a new one starts. Other events (master pushes, tags, schedules, workflow_run) use the unique run id as their group so they are never cancelled or serialized.
github.base_ref is only set for pull_request events. On push events the job diffed against an empty ref, producing an empty diff, so cargo-mutants had nothing to do and the job spent a runner slot doing setup work for no signal.
All six build-nuget-native matrix legs computed the same rust-cache key: the job id, toolchain, and lockfiles are identical and the cross target is passed on the command line, outside the key. The legs raced to save one cache entry, only the first finisher (typically the fast linux-x64 build) won, and every other leg, including the 16 to 19 minute Windows cross builds, ran cold on every workflow run. Add the RID to the cache key so each leg keeps its own dependency cache, and cache cargo-xwin's MSVC CRT/SDK download alongside it.
The C# workflow only ran on pull requests, and GitHub Actions caches saved on a pull request branch are not visible to other pull requests. Even with per-RID cache keys, every new pull request therefore started the 16 to 19 minute Windows cross builds from a cold cache. Run build-nuget-native on master pushes that touch payjoin-ffi so its caches land on the default branch, where every pull request can restore them. This also verifies the cross builds post-merge. The test, pack, and smoke jobs are skipped on push since their coverage is already provided pre-merge.
The repository sits at 12 GB of Actions cache against GitHub's 10 GB budget, so entries are constantly evicted. Most of the space is per-branch duplicates: every pull request branch saves its own copy of the ~1.5 GB per-toolchain Test caches that already exist for master, and those duplicates evict smaller caches such as the C# native build entries, forcing cold 16 to 19 minute cross builds. Save rust-cache entries only from master, which every pull request can restore. DiffMutants runs only on pull requests, so instead of saving anything it now restores the stable Test cache via a shared key.
Each flake-check run rebuilds the vendored dependencies and the whole workspace from scratch in four matrix legs of 13 to 18 minutes, because no nix binary cache is configured for this workflow. Three of the legs then run the same workspace test suite that rust.yml has already run on the pull request, and the maintenance leg repeats the Lint and Format jobs. The nix-specific signal is only exercised when the nix packaging inputs change. Limit pull request runs to changes in nix files, the flake lock, the cargo lockfiles, and the toolchain file. This also fixes the previous paths-ignore, which skipped the flake checks on exactly the pull requests that only touched flake.lock, where they matter most. Master pushes still run the full check on every change, so a Rust-only change that breaks the nix build is caught after merge rather than never.
cargo's --verbose flag prints every rustc and test-binary invocation: in a passing CI Test job it accounted for roughly 6,900 of 9,600 log lines while adding nothing to failure diagnosis. Failure detail is unaffected: the test harness still captures per-test output, including the RUST_LOG=debug tracing output, and prints it only when a test fails.
Bump actions/checkout v6 to v7, actions/upload-artifact v4 to v7, actions/download-artifact v4 to v8, actions/setup-python v5 to v7, actions/setup-dotnet v4 to v6, and peter-evans/create-pull-request v7 to v8. Per their release notes these majors only move the actions to the Node 24 runtime and ESM, which GitHub-hosted runners already support; artifacts are downloaded by name here, so the download v5 path change for by-id downloads does not apply, and v8's digest-mismatch enforcement is a hardening we want. Also align the two lewagon/wait-on-check-action pins on v1.9.0, which release-image.yml still used at v1.5.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
Please confirm the following before requesting review:
AI
in the body of this PR.