fix(daemon): make doctor's exit code mean something - #818
Open
cuttlefisch wants to merge 1 commit into
Open
Conversation
`mae-daemon doctor` printed
collab config: 1 issue(s)
- collab.auth.mode = 'key' but authorized_keys is empty — no client can connect
and exited **0**. Every automated consumer therefore read that instance as
healthy. A `HEALTHCHECK` on a container is exactly an exit code, so a
misconfigured daemon that no client can reach reports itself fine.
The convention already existed and was simply incomplete: `run_doctor` returned
1 for a side-by-side resource conflict, and 0 for everything else — including
config issues it had just printed, and a collab store that failed to open.
Now it counts problems as it goes, prints a verdict line, and returns 1 if any.
Counting rather than returning early is deliberate: doctor's whole value is the
full report, so a failure must not truncate it.
**A collab port already in use is NOT counted**, and that distinction is the
whole design. Running doctor against a healthy, running instance finds its port
bound — if that counted, doctor would fail precisely when the service works.
mae_daemon's own verify play already encodes the opposite reading: it treats
`available` as the failure signal, because a bound port means the daemon is up.
The role now asserts on `rc` and quotes doctor's report on failure, instead of
only string-matching stdout. That was a workaround for this bug — Ansible's
`command` module fails on a non-zero rc for free, and could not use it. The
task keeps `failed_when: false` so the assertion can quote the report rather
than letting Ansible abort with a generic message. Asserting on rc also means a
new failure mode doctor learns to detect is caught without adding another
string match. ansible-lint passes at the production profile.
Three tests, and the middle one is the point:
- an invalid config must exit 1 — confirmed to FAIL against the unfixed code
with the exact message above;
- a valid config must still exit 0, or a gate that always fails gets disabled;
- a bound collab port must stay 0, guarding the distinction above.
Each fixture asserts its own premise first (`check_collab()` really is
non-empty / really is empty), so the exit-code assertions cannot pass vacuously
against a fixture that stopped being invalid.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
mae-daemon doctorprinted this:and exited 0.
Every automated consumer therefore reads that instance as healthy. A container
HEALTHCHECKis an exit code, so a daemon no client can reach reports itself fine.The convention already existed and was simply incomplete —
run_doctorreturned 1 for aside-by-side resource conflict, and 0 for everything else, including the config issues it had
just printed and a collab store that failed to open.
The fix
Count problems while walking the report, print a verdict line, return 1 if any. Counting rather
than returning early is deliberate: doctor's value is the full report, so a failure must not
truncate it.
A collab port already in use is deliberately not counted, and that distinction is the whole
design. Running doctor against a healthy, running instance finds its port bound — if that
counted, doctor would fail precisely when the service works. The
mae_daemonrole alreadyencodes the opposite reading: it treats
availableas the failure signal, because a boundport means the daemon is up.
The role can now use the exit code
verify.ymlasserted on scraped stdout, which was a workaround for this bug: Ansible'scommandmodule fails on a non-zero rc for free and could not use it. It now asserts onrcand quotes doctor's report on failure.
failed_when: falsestays so the assertion can quotethe report rather than letting Ansible abort with a generic message.
The wider benefit: a new failure mode doctor learns to detect is caught without adding another
string match to that file. ansible-lint passes at the
productionprofile.Tests
Each fixture asserts its own premise first — that
check_collab()really is non-empty, orreally is empty — so the exit-code assertions cannot pass vacuously against a fixture that
quietly stopped being invalid.
Found while preparing a real containerised deployment, where the missing exit code is what
stops
doctorbeing usable as a healthcheck at all.