Skip to content

test(railway): qualify the production source plan offline - #1211

Merged
0xPlayerOne merged 1 commit into
mainfrom
test/railway-production-source-plan-qualification
Oct 21, 2026
Merged

0xPlayerOne merged 1 commit into
mainfrom
test/railway-production-source-plan-qualification

Conversation

@0xPlayerOne

@0xPlayerOne 0xPlayerOne commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Mechanical publication validation refresh — 2026-10-21

Draft source publication only; auto-merge OFF. No readiness, merge, deployment or manual release. Implementation remains W4-owned.

Exact published source is existing commit 60ee60b6fca57041efa71b83579cebe5fa5c8ef3. Publisher reran the isolated offline source-plan suite: 48 pass / 0 fail. No Railway command ran. Authenticated bun run requirements:check also passed: 200 requirements and 103 issue audits. The initial history-free archive check could not validate candidate.commit; the same exact source passed once its existing Git history was available read-only.


The Railway production source plan: offline qualification

What this is

The Control Plane production source plan refuses an ambiguous target, a deployment whose
metadata contradicts the source, a history whose newest deployment is not in service and a
read that failed. Those refusals were written, but not all of them were proven offline, so
a rule could have been weakened without the suite noticing.

This pull request qualifies them. It drives the plan with a fake command runner only:
no Railway command, status, plan, apply, promotion, provision or deployment is run
anywhere in these tests, and nothing in the change touches an infrastructure, credential
or environment.

The files

File What changed
tests/railway-production-plan.test.mjs 16 -> 48 tests; the qualification matrix
scripts/railway-production-plan.mjs one rule: a command runner that throws earns the same refusal a failed command

.railway/production-source.js and .railway/production-source.d.ts are unchanged: the
declaration already refuses a missing, a mutable and a foreign production image, so the
qualification needed no rule of its own.

The qualification matrix

Ambiguous or multiple targets — a linked project with no production environment, with
more than one production environment, with no control-api service and with more than one
control-api service; a source with no active deployment and with more than one active
deployment. A production service that is not the control-api service is still ignored,
so the rule stays narrow.

Contradictory or missing deployment metadata — a missing project identifier, a
missing production environment identifier, a missing control-api service identifier, a
missing deployment identifier, missing deployment metadata, an active deployment whose
image contradicts the source image, one whose image alone contradicts it, one whose digest
alone contradicts it, a production image that belongs to another repository, and a
production image whose digest is not a full sha256.

Stopped and errored histories — a stopped deployment, a stopped deployment history,
an errored deployment history, and a stopped source that refuses before any plan command
is issued.

Injected command failures and timeouts — a failed status command, a failed source
query, a failed plan command, a command that timed out, a command runner that throws, a
status read that is not JSON and a source read that is not JSON.

Deterministic refusal evidence — the same input refuses with the same message every
time, every refusal names the fact it refused, and no refusal repeats the output of the
command it refused.

The one rule that changed

A command runner that throws was not a refusal: its error escaped with whatever the runner
said, so the failure was not the plan's own and could carry a runner's output into the
error. It now earns the same refusal a failed command earns:

Railway <command> command failed; output was withheld.

That keeps the refusal uniform across a non-zero exit, a timeout and a throwing runner,
and it keeps the command's output out of the error.

The checks

Check Result
bun test tests/railway-production-plan.test.mjs 48 pass, 0 fail
bun run format:check clean over 1,935 files
bun run lint exit 0
mutation checks 13 of 13 refusal rules independently covered

The mutation checks disable one refusal rule at a time - the single-production-environment
rule, the single-control-api-service rule, the sole-active-deployment rule, the
successful-status rule, the not-stopped rule, the deployment-metadata-image rule, the
deployment-metadata-digest rule, the missing-identifier rule, the GraphQL-error rule, the
no-connected-repository rule, the invalid-JSON rule, the throwing-runner refusal and the
immutable-production-image rule - and each one makes the suite fail.

Known issues outside this change

  • The repository's unit lane (bun run test:group:unit) crashes in the pinned Bun 1.4.3
    runtime with panic: range end index 1884 out of range for slice of length 1024. The
    identical panic reproduces with these two files reverted, so it is a runtime bug rather
    than a failure of this change.
  • bun run requirements:check needs an authenticated GitHub CLI, which this environment
    does not have; the pre-commit gate stops there.
  • This pull request does not need an architecture digest refresh: the four source-plan
    paths are outside the audit's composition profiles and the branch is based on main.

What this does not do

It runs no Railway command and changes no infrastructure, credential or environment. It
does not decide whether a production cutover, a promotion or a recovery may happen; it
proves only that the plan refuses the states it exists to refuse.

The Control Plane production source plan refuses an ambiguous target, a
deployment whose metadata contradicts the source, a history whose newest
deployment is not in service and a read that failed. Those refusals were
written, but not all of them were proven offline, so a rule could have been
weakened without the suite noticing.

The qualification drives the plan with a fake command runner only, and it
covers the states the plan exists to refuse: a linked project with no
production environment, with more than one production environment, with no
control-api service and with more than one control-api service; a source with
no active deployment, with more than one active deployment, with a missing
deployment identifier, with missing deployment metadata, and with an image or
a digest that contradicts the source image; a stopped and an errored
deployment history; a status read, a source query and a plan command that
failed or timed out; a read that is not JSON; and a production image that
belongs to another repository or carries a digest that is not a full sha256.

The qualification also proves the refusal evidence: the same input refuses
with the same message every time, every refusal names the fact it refused,
and no refusal repeats the output of the command it refused.

One rule had to change for that. A command runner that throws was not a
refusal: its error escaped with whatever the runner said. It now earns the
same refusal a failed command earns, so the command is named and the output
is withheld.
@0xPlayerOne 0xPlayerOne changed the title test: railway-production-source-plan-qualification test(railway): qualify the production source plan offline Oct 21, 2026
@0xPlayerOne
0xPlayerOne marked this pull request as ready for review October 21, 2026 10:03
@0xPlayerOne
0xPlayerOne merged commit 2e68492 into main Oct 21, 2026
44 checks passed
@0xPlayerOne
0xPlayerOne deleted the test/railway-production-source-plan-qualification branch October 21, 2026 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant