Skip to content

fix(daemon): make doctor's exit code mean something - #818

Open
cuttlefisch wants to merge 1 commit into
mainfrom
feat/daemon-doctor-exit-code
Open

fix(daemon): make doctor's exit code mean something#818
cuttlefisch wants to merge 1 commit into
mainfrom
feat/daemon-doctor-exit-code

Conversation

@cuttlefisch

Copy link
Copy Markdown
Owner

The bug

mae-daemon doctor printed this:

collab config: 1 issue(s)
  - collab.auth.mode = 'key' but authorized_keys is empty — no client can connect

and exited 0.

Every automated consumer therefore reads that instance as healthy. A container
HEALTHCHECK is an exit code, so a daemon no client can reach reports itself fine.

The convention already existed and was simply incomplete — run_doctor returned 1 for a
side-by-side resource conflict, and 0 for everything else, including the config issues it had
just printed and a collab store that failed to open.

The fix

Count problems while walking the report, print a verdict line, return 1 if any. Counting rather
than returning early is deliberate: doctor's value is the full report, so a failure must not
truncate it.

A collab port already in use is deliberately not counted, and that distinction is the whole
design. Running doctor against a healthy, running instance finds its port bound — if that
counted, doctor would fail precisely when the service works. The mae_daemon role already
encodes the opposite reading: it treats available as the failure signal, because a bound
port means the daemon is up.

The role can now use the exit code

verify.yml asserted on scraped stdout, which was a workaround for this bug: Ansible's
command module fails on a non-zero rc for free and could not use it. It now asserts on rc
and quotes doctor's report on failure. failed_when: false stays so the assertion can quote
the report rather than letting Ansible abort with a generic message.

The wider benefit: a new failure mode doctor learns to detect is caught without adding another
string match to that file. ansible-lint passes at the production profile.

Tests

test why
an invalid config exits 1 the bug. Confirmed to fail against the unfixed code with the exact message above
a valid config still exits 0 a gate that always fails gets disabled
a bound collab port stays 0 guards the distinction that makes this safe against a running instance

Each fixture asserts its own premise first — that check_collab() really is non-empty, or
really is empty — so the exit-code assertions cannot pass vacuously against a fixture that
quietly stopped being invalid.


Found while preparing a real containerised deployment, where the missing exit code is what
stops doctor being usable as a healthcheck at all.

`mae-daemon doctor` printed

    collab config: 1 issue(s)
      - collab.auth.mode = 'key' but authorized_keys is empty — no client can connect

and exited **0**. Every automated consumer therefore read that instance as
healthy. A `HEALTHCHECK` on a container is exactly an exit code, so a
misconfigured daemon that no client can reach reports itself fine.

The convention already existed and was simply incomplete: `run_doctor` returned
1 for a side-by-side resource conflict, and 0 for everything else — including
config issues it had just printed, and a collab store that failed to open.

Now it counts problems as it goes, prints a verdict line, and returns 1 if any.
Counting rather than returning early is deliberate: doctor's whole value is the
full report, so a failure must not truncate it.

**A collab port already in use is NOT counted**, and that distinction is the
whole design. Running doctor against a healthy, running instance finds its port
bound — if that counted, doctor would fail precisely when the service works.
mae_daemon's own verify play already encodes the opposite reading: it treats
`available` as the failure signal, because a bound port means the daemon is up.

The role now asserts on `rc` and quotes doctor's report on failure, instead of
only string-matching stdout. That was a workaround for this bug — Ansible's
`command` module fails on a non-zero rc for free, and could not use it. The
task keeps `failed_when: false` so the assertion can quote the report rather
than letting Ansible abort with a generic message. Asserting on rc also means a
new failure mode doctor learns to detect is caught without adding another
string match. ansible-lint passes at the production profile.

Three tests, and the middle one is the point:

  - an invalid config must exit 1 — confirmed to FAIL against the unfixed code
    with the exact message above;
  - a valid config must still exit 0, or a gate that always fails gets disabled;
  - a bound collab port must stay 0, guarding the distinction above.

Each fixture asserts its own premise first (`check_collab()` really is
non-empty / really is empty), so the exit-code assertions cannot pass vacuously
against a fixture that stopped being invalid.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant