Repository navigation
test(railway): qualify the production source plan offline - #1211
Merged
0xPlayerOne merged 1 commit intoOct 21, 2026
Merged
Conversation
The Control Plane production source plan refuses an ambiguous target, a deployment whose metadata contradicts the source, a history whose newest deployment is not in service and a read that failed. Those refusals were written, but not all of them were proven offline, so a rule could have been weakened without the suite noticing. The qualification drives the plan with a fake command runner only, and it covers the states the plan exists to refuse: a linked project with no production environment, with more than one production environment, with no control-api service and with more than one control-api service; a source with no active deployment, with more than one active deployment, with a missing deployment identifier, with missing deployment metadata, and with an image or a digest that contradicts the source image; a stopped and an errored deployment history; a status read, a source query and a plan command that failed or timed out; a read that is not JSON; and a production image that belongs to another repository or carries a digest that is not a full sha256. The qualification also proves the refusal evidence: the same input refuses with the same message every time, every refusal names the fact it refused, and no refusal repeats the output of the command it refused. One rule had to change for that. A command runner that throws was not a refusal: its error escaped with whatever the runner said. It now earns the same refusal a failed command earns, so the command is named and the output is withheld.
0xPlayerOne
marked this pull request as ready for review
October 21, 2026 10:03
0xPlayerOne
deleted the
test/railway-production-source-plan-qualification
branch
October 21, 2026 10:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mechanical publication validation refresh — 2026-10-21
Draft source publication only; auto-merge OFF. No readiness, merge, deployment or manual release. Implementation remains W4-owned.
Exact published source is existing commit
60ee60b6fca57041efa71b83579cebe5fa5c8ef3. Publisher reran the isolated offline source-plan suite: 48 pass / 0 fail. No Railway command ran. Authenticatedbun run requirements:checkalso passed: 200 requirements and 103 issue audits. The initial history-free archive check could not validatecandidate.commit; the same exact source passed once its existing Git history was available read-only.The Railway production source plan: offline qualification
What this is
The Control Plane production source plan refuses an ambiguous target, a deployment whose
metadata contradicts the source, a history whose newest deployment is not in service and a
read that failed. Those refusals were written, but not all of them were proven offline, so
a rule could have been weakened without the suite noticing.
This pull request qualifies them. It drives the plan with a fake command runner only:
no Railway command, status, plan, apply, promotion, provision or deployment is run
anywhere in these tests, and nothing in the change touches an infrastructure, credential
or environment.
The files
tests/railway-production-plan.test.mjsscripts/railway-production-plan.mjs.railway/production-source.jsand.railway/production-source.d.tsare unchanged: thedeclaration already refuses a missing, a mutable and a foreign production image, so the
qualification needed no rule of its own.
The qualification matrix
Ambiguous or multiple targets — a linked project with no production environment, with
more than one production environment, with no control-api service and with more than one
control-api service; a source with no active deployment and with more than one active
deployment. A production service that is not the control-api service is still ignored,
so the rule stays narrow.
Contradictory or missing deployment metadata — a missing project identifier, a
missing production environment identifier, a missing control-api service identifier, a
missing deployment identifier, missing deployment metadata, an active deployment whose
image contradicts the source image, one whose image alone contradicts it, one whose digest
alone contradicts it, a production image that belongs to another repository, and a
production image whose digest is not a full sha256.
Stopped and errored histories — a stopped deployment, a stopped deployment history,
an errored deployment history, and a stopped source that refuses before any plan command
is issued.
Injected command failures and timeouts — a failed status command, a failed source
query, a failed plan command, a command that timed out, a command runner that throws, a
status read that is not JSON and a source read that is not JSON.
Deterministic refusal evidence — the same input refuses with the same message every
time, every refusal names the fact it refused, and no refusal repeats the output of the
command it refused.
The one rule that changed
A command runner that throws was not a refusal: its error escaped with whatever the runner
said, so the failure was not the plan's own and could carry a runner's output into the
error. It now earns the same refusal a failed command earns:
That keeps the refusal uniform across a non-zero exit, a timeout and a throwing runner,
and it keeps the command's output out of the error.
The checks
bun test tests/railway-production-plan.test.mjsbun run format:checkbun run lintThe mutation checks disable one refusal rule at a time - the single-production-environment
rule, the single-control-api-service rule, the sole-active-deployment rule, the
successful-status rule, the not-stopped rule, the deployment-metadata-image rule, the
deployment-metadata-digest rule, the missing-identifier rule, the GraphQL-error rule, the
no-connected-repository rule, the invalid-JSON rule, the throwing-runner refusal and the
immutable-production-image rule - and each one makes the suite fail.
Known issues outside this change
bun run test:group:unit) crashes in the pinned Bun 1.4.3runtime with
panic: range end index 1884 out of range for slice of length 1024. Theidentical panic reproduces with these two files reverted, so it is a runtime bug rather
than a failure of this change.
bun run requirements:checkneeds an authenticated GitHub CLI, which this environmentdoes not have; the pre-commit gate stops there.
paths are outside the audit's composition profiles and the branch is based on
main.What this does not do
It runs no Railway command and changes no infrastructure, credential or environment. It
does not decide whether a production cutover, a promotion or a recovery may happen; it
proves only that the plan refuses the states it exists to refuse.