test(canonical): import jsonschema unconditionally so the schema contract fails closed - #636
Open
Nitjsefnie wants to merge 1 commit into
Open
test(canonical): import jsonschema unconditionally so the schema contract fails closed#636Nitjsefnie wants to merge 1 commit into
Nitjsefnie wants to merge 1 commit into
Conversation
…ract fails closed
The six schema contract tests in tests/test_formatters_canonical.py opened
with pytest.importorskip("jsonschema"), so a missing jsonschema turned them
into silent skips and the run stayed green. jsonschema>=4.21 is a declared
dev dependency, not an optional extra, so a missing import means a broken
environment and must be loud: the module now imports it unconditionally and
a broken environment errors at collection.
The pinning test in tests/test_canonical_schema_contract_gate.py spawns a
child pytest session with jsonschema made unimportable (a sys.modules shim
plus a meta_path blocker, loaded via -p before collection) and asserts the
contract tests redden rather than skip -- the same child-session pattern as
the skip-ceiling end-to-end tests in tests/test_corpus_manifest.py. Verified
to fail on its assertion against the pre-fix tree, and to go red again when
the importorskip call sites are restored.
The issue body says eleven importorskip call sites; the tree carries six
(lines 577, 632, 645, 655, 919, 1007 on develop). The three load-bearing
tests it names are among them, so the finding and fix are unchanged.
Fixes AgoraDMV#585.
Co-Authored-By: Kimi K3 <noreply@kimi.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue
Closes #585
What does this change?
tests/test_formatters_canonical.pynow importsjsonschemaunconditionally instead of guarding each test withpytest.importorskip, so a missing dependency reddens the suite instead of quietly skipping the schema contract.jsonschema>=4.21is already a declared dev dependency — with an inline comment inpyproject.tomlnaming this exact hazard — so a skip means a broken environment, not an optional feature.Took the plain-import route rather than adding the module to a watch group. The watch-group mechanism's default-on behaviour is the open question in #610, and routing through it now would pre-empt your own undecided design call. Nothing in
CORPUS_GATE_MODULES/CI_SLOW_MODULES/FAST_GATE_MODULESis touched.Nothing the three named tests assert has changed — stripping the import lines from both revisions leaves the two files byte-identical.
One correction to the issue body, with the diagnosis rather than just the number. It says "all eleven schema tests in that module" use
importorskip. In that module there are six, at lines 577, 632, 645, 655, 919 and 1007. But the count ofimportorskip("jsonschema")across the wholetests/tree is exactly eleven — those six plus two intest_canonical_tree.pyand one each intest_corpus_tree_properties.py,test_pdf_compare.pyandtest_xml_compare.py. So "eleven" looks like a repo-wide count attributed to one module. Two rival readings do not fit: nine tests match-k schema, and the module has fifty-one tests in total. The finding stands either way; only the attribution moves. The five call sites outside this module are left alone as out of scope.How to test
tests/test_canonical_schema_contract_gate.pyis the new guard. It spawns a child pytest session withjsonschemablocked and asserts the child reddens, using the_run_child_sessionpattern the corpus-manifest tests already use for skip ceilings.Red-green, and the mutation:
assert 0 != 0— all six sites skipping withimport of jsonschema halted; None in sys.modules, which also confirms the blocker genuinely bites inside the child rather than passing for the wrong reason.importorskipcannot reintroduce the fail-open: the module-level import kills collection first. So the gate pins the class, not the six call sites that happened to exist.I have not ticked "ran the CI gates locally": the suite ran on a runner rather than on my machine. The new gate test is confirmed executed and passing — not merely collected — on 3.12, 3.12.0, 3.13 and 3.14.
Checklist
Closes #...)AI assistance
Generated by Kimi K3 (implementation, testing), Claude Opus 5 (brief, review)