Skip to content

Canonical schema contract tests can green-by-skip if jsonschema is unavailable #585

Description

@willhea

What

Make the canonical-diff schema contract tests fail closed when jsonschema is unavailable, so they cannot silently disappear.

Why

The canonical diff document is a machine-readable contract (ADR 0006). Three tests in tests/test_formatters_canonical.py are what prove the schema still enforces it in the direction that can actually regress:

  • test_schema_rejects_a_change_carrying_the_removed_amounts_field — the legacy changed-only money field is still forbidden;
  • test_schema_requires_amount_entries_on_every_change — the money field is still required, not merely allowed;
  • test_schema_accepts_the_unmodified_producer_output — the positive control, so neither rejection can pass by rejecting everything.

All eleven schema tests in that module open with pytest.importorskip("jsonschema"). If jsonschema is ever absent, they skip and the run stays green.

jsonschema>=4.21 is currently an explicit dev dependency, so an ordinary uv sync and CI both run them — this is not a live failure. The gap is that nothing would announce it if that stopped being true. tests/test_formatters_canonical.py is in none of the watched skip groups (CORPUS_GATE_MODULES, CI_SLOW_MODULES, FAST_GATE_MODULES), so the skip-ceiling machinery that exists for exactly this class of fail-open does not cover it.

This is the same shape as the corpus fail-opens ADR 0015 records: a gate that goes quiet is indistinguishable from a gate that passes, and the failure surfaces only when someone later trusts the green.

Done when

A missing jsonschema makes the contract-schema tests fail or error rather than skip. Any of these would satisfy it; the choice is open:

  • import jsonschema unconditionally in that module, since it is a declared dependency rather than an optional extra;
  • add the module to a watched skip group so an undeclared skip fails the session;
  • assert at session start that the contract gates are collected and running.

Not in scope

Changing the schema, the contract, or what the three tests assert.

Notes

Found while auditing ADR 0006 for #575. The ADR rewrite cites these tests as the evidence for its rejection-direction governance rule, which is what makes the fail-open worth closing rather than noting.

Refs #575.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions