Skip to content

test(corpus): floor the PDF smoke suite's collected pair count - #635

Open
Nitjsefnie wants to merge 2 commits into
AgoraDMV:developfrom
Nitjsefnie-OSC:issue-601
Open

test(corpus): floor the PDF smoke suite's collected pair count#635
Nitjsefnie wants to merge 2 commits into
AgoraDMV:developfrom
Nitjsefnie-OSC:issue-601

Conversation

@Nitjsefnie

Copy link
Copy Markdown
Contributor

Related issue

Closes #601

What does this change?

Adds the completeness floor #598 established for _DIVISION_VERSIONS, applied to
test_pdf_corpus_smoke.py's pair list.

Two assertions, pinning different failure causes:

  • len(_PAIRS) >= 15 catches the committed corpus losing PDF fixtures, or adjacent_pdf_pairs()
    ceasing to pair them. 15 sits under the live count of 23 so ordinary fixture churn does not force
    an edit, matching the precedent's ratio.
  • len(_PAIRS) == len(adjacent_pdf_pairs()) catches a slice, filter or truncation at the _PAIRS
    assignment itself, with the corpus intact.

Both read _PAIRS rather than calling adjacent_pdf_pairs() afresh. That is the load-bearing
detail: a fresh call guards the corpus, not the list the suite actually parametrizes over. My first
attempt asserted on a fresh call, and _PAIRS = adjacent_pdf_pairs()[:3] — the example from the
issue — then dropped the suite from 138 cases to 18 with the floor still green.

The floor lives in tests/test_corpus_manifest.py rather than beside the suite it guards because
tests/test_pdf_corpus_smoke.py sets pytestmark = pytest.mark.slow at module level and pytest has
no per-test un-marking, so an in-module floor would only ever run in the slow tier. That is the
placement and the reasoning recorded for the committed-fixture floor moved per your #328 review.
The cross-module import follows the existing idiom (tests/test_xml_subsection_nodes.py:52,
tests/test_assignment_classification_boundary.py:874); here it is load-bearing rather than a DRY
convenience, since decoupling it is precisely what made the first attempt guard nothing.

How to test

Red-green, both directions:

  • _PAIRS = adjacent_pdf_pairs()[:3]AssertionError: 3 >= 15 — the floor half fires.
  • _PAIRS = adjacent_pdf_pairs()[:16]AssertionError: 16 == 23 — above the floor, so only the
    equality half fires. This is the case that shows the second assertion is not decorative.
  • Healthy tree → passes, collected in the fast tier under -m "not slow and not browser", with the
    smoke module's own 138 cases still deselected. Runs in 0.05s and adds no dependency
    (pypdfium2 is already a core dep).

Known gap, stated because it bounds what this buys: a coordinated slice applied to both the
@parametrize decorator and its ids= list would still escape the floor. The single-site version
of that edit errors loudly at collection, and the same gap exists in #598's floor — I have not
tried to close it here.

Checklist

  • Linked the issue above (Closes #...)
  • Ran the CI gates locally and they pass (see What CI checks) — run on a runner rather than locally; targeted tests were run locally
  • New or changed behavior has tests
  • For a bug fix: the test fails without the fix, and I ran it both ways to check
  • Disclosed AI assistance below

AI assistance

Generated by Claude Opus 5 (brief, implementation, review, testing)

Nitjsefnie and others added 2 commits August 14, 2026 15:11
test_pdf_corpus_smoke.py parametrizes straight over adjacent_pdf_pairs()
with nothing asserting the list is complete. Zero pairs is caught by the
CI_SLOW_MODULES skip ceiling, but a partial collapse just runs fewer
comparisons and stays green.

The floor lives in test_corpus_manifest.py, not beside the suite it
guards: that module is @slow at module level and pytest has no per-test
un-marking, so a floor added there would only run in the slow tier. Same
placement, and the same reason, as the committee-report fixture floor
moved here in AgoraDMV#328. 15 sits well under the current 23 pairs so unrelated
fixture churn does not force an edit.

Closes AgoraDMV#601

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rizes

The floor asserted on its own adjacent_pdf_pairs() call, not on the _PAIRS
list test_pdf_corpus_smoke.py actually parametrizes over, so it guarded the
corpus rather than the case list. `_PAIRS = adjacent_pdf_pairs()[:3]` — the
23-pairs-down-to-3 collapse the floor was added for — cut the smoke suite from
138 collected cases to 18 with the floor and the whole fast module still green.

Read _PAIRS itself and assert both halves: >= 15 still catches the corpus or
the pairing shrinking, and == len(adjacent_pdf_pairs()) catches a slice,
filter or truncation at the assignment while the corpus is intact. Imported
inside the test so the slow module's engine imports stay off the fast module's
collection-time import graph; marks come from collection, not import, so the
smoke cases remain deselected in the fast tier.

Refs AgoraDMV#601

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_pdf_corpus_smoke.py's pair list has no completeness floor

1 participant