Skip to content

Implement the typed doctest core architecture - #93

Draft
tony wants to merge 7 commits into
adr-doctest-core-architecturefrom
adr-doctest-core-architecture-spike-00
Draft

Implement the typed doctest core architecture#93
tony wants to merge 7 commits into
adr-doctest-core-architecturefrom
adr-doctest-core-architecture-spike-00

Conversation

@tony

@tony tony commented Aug 8, 2026

Copy link
Copy Markdown
Member

Depends on #92.

Summary

  • Implement the proposed typed, host-neutral doctest_core pipeline for parsing, projection, registration, and execution.
  • Refactor doctest_docutils and pytest_doctest_docutils as adapters over the core while preserving their command and collection surfaces.
  • Keep pytest shared-state groups schedulable as one item while retaining an independent doctest.DocTest and source location for each block.
  • Expand compatibility coverage and pin the CI matrix to exact supported docutils, pytest, pytest-asyncio, and pytest-rerunfailures versions.

The architecture ADR remains proposed. This draft implements it against the existing direct and pytest entry points so the contracts can be reviewed against executable behavior.

Why

The direct and pytest paths previously owned overlapping parsing and execution behavior. The pytest adapter also blocked the standard doctest plugin while depending on its implementation. A shared core separates host integration, scheduling identity, and diagnostic identity without merging source blocks or adding an xdist affinity protocol.

Impact

Documentation blocks within a group share one namespace, but each block retains its own location and failure detail. The pytest adapter coexists with the standard doctest plugin, so pytest continues to provide fixtures, checker and report options, and Python-module doctest collection.

Validation

  • uv run ruff format . --check
  • uv run py.test
  • uv run ruff check .
  • uv run mypy .
  • just build-docs

Known CI failure

The push workflow fails in mypy for Python 3.13 with docutils==0.21.2 and pytest==7.2.0. An isolated local run with the same pins reproduces strict typing errors in src/_pytest_doctest_compat.py and src/pytest_doctest_docutils.py; the equivalent pytest==8.4.2 run passes. The oldest pytest line exposes the private compatibility surface with weaker annotations, so the blocker remains unresolved.

tony added 6 commits August 15, 2026 19:21
why: Test the ADR boundaries against stock prompt semantics, typed
markup extraction, shared groups, and extended execution.

what:
- Add immutable contracts, settings, models, and registry snapshots
- Extract and project reStructuredText and MyST into group recipes
- Run prompt and exec lanes with typed results and cleanup policy
- Cover projection, runtime, registry, and Sphinx doctree behavior
why: Keep the released direct API and command shape while exercising the
same extraction, projection, and execution path as other hosts.

what:
- Adapt the finder and direct runner to typed block and group results
- Preserve stock failure and summary accounting across Python versions
- Add direct facade ordering, location, and reporting regressions
why: Preserve pytest fixtures, checker semantics, and reporting without
blocking the built-in doctest plugin or splitting shared group state.

what:
- Add one item per group with fresh attempt-local doctest materialization
- Quarantine private pytest compatibility and filter duplicate collectors
- Exercise fixtures, reruns, xdist, async fixtures, and contributions
- Package the typed core, adapters, tests, and corrected plugin entry point
why: Broad major-version constraints hid invalid dependency combinations
and did not prove the declared Python and pytest floor.

what:
- Pin docutils, pytest, asyncio, and rerun plugin matrix versions
- Select the matrix interpreter for every uv command
- Check installed dependency consistency before static and runtime gates
why: The proposed architecture contained compatibility and lifecycle
claims that changed when exercised against real hosts and version floors.

what:
- Record the typed-core and thin-adapter bakeoff result
- Narrow vanilla, diagnostics, line, and pytest compatibility promises
- Define runner and registry contracts proven by the implementation
- List deferred host lifecycle, reporting, async, and facade work
why: Public guidance must distinguish the direct command from pytest's
one-item-per-group behavior without promising full stdlib transcripts.

what:
- Describe direct doctest command and summary behavior
- Explain built-in pytest doctest plugin composition
- Clarify bare-block isolation and the Sphinx default directive group
- State the supported docutils range
@tony
tony force-pushed the adr-doctest-core-architecture-spike-00 branch from 8b887ad to 36f1a1f Compare August 16, 2026 00:30
why: The rebase onto master dropped this branch's `docutils<0.22` cap,
on the reasoning that master tests 0.22.4 successfully. That was wrong.
Sphinx declares `docutils>=0.20,<0.22`, so 0.22.4 leaves the environment
inconsistent, and this branch's `uv pip check` step fails on it. Master
has no such step, which is why it never noticed.

what:
- Restore `docutils>=0.20.1,<0.22`
- Return the matrix to docutils 0.20.1 and 0.21.2
- Drop the Python 3.10 exclusion, which only existed to keep 3.10 away
  from docutils 0.22 and cannot trigger under the ceiling

Verified `uv pip check` clean and 227 tests passing on docutils 0.20.1
and 0.21.2 against pytest 8.4.2 and 9.1.1.
@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.21596% with 286 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.93%. Comparing base (a47d332) to head (d362ae3).

Files with missing lines Patch % Lines
src/doctest_core/markup.py 73.09% 74 Missing ⚠️
src/doctest_core/runner.py 77.12% 62 Missing ⚠️
src/pytest_doctest_docutils.py 69.23% 44 Missing ⚠️
src/doctest_core/registry.py 62.72% 41 Missing ⚠️
src/_pytest_doctest_compat.py 52.45% 29 Missing ⚠️
src/doctest_core/project.py 73.62% 24 Missing ⚠️
src/doctest_docutils.py 88.99% 12 Missing ⚠️
Additional details and impacted files
@@                        Coverage Diff                        @@
##           adr-doctest-core-architecture      #93      +/-   ##
=================================================================
+ Coverage                          76.68%   84.93%   +8.25%     
=================================================================
  Files                                 15       26      +11     
  Lines                               1025     2470    +1445     
=================================================================
+ Hits                                 786     2098    +1312     
- Misses                               239      372     +133     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tony

tony commented Aug 16, 2026

Copy link
Copy Markdown
Member Author

Rebased onto current master. The stack is preserved: #92 carries the first 24 commits onto master, this PR the remaining 7 on top of it. Both are green.

Why it needed rebasing. Master gained pytest>=8.3.3 (#94) and docutils>=0.20 plus a working test matrix (#95). This branch's 9 mypy errors are gone as a result — every one of them was a pytest 7 leg, and pytest 7 is no longer supported.

Conflicts, and how they were resolved. Three files; everything else applied clean.

pyproject.toml — took master's pytest>=8.3.3 over this branch's pytest>=7.2. Kept this branch's packaging, myst-parser>=2.0.0 and docutils>=0.20.1.

.github/workflows/tests.yml — merged both sides. This branch's exact version pins, its include: mapping of pytest to pytest-asyncio/pytest-rerunfailures, and its uv pip check step, minus the pytest 7.2.0 row. Both sides had independently added UV_PYTHON and --no-sync, so those were deduplicated to one env: block.

uv.lock — regenerated rather than hand-merged.

One resolution I got wrong, and this branch caught. I initially dropped the docutils<0.22 cap, reasoning that master tests 0.22.4 successfully. That was incorrect: Sphinx declares docutils>=0.20,<0.22, so 0.22.4 leaves the environment inconsistent. The uv pip check step added on this branch failed on it — master has no such step, which is why master never noticed. The cap is restored, along with the 0.20.1/0.21.2 matrix.

State: mypy clean across 31 source files, ruff clean, 227 tests passing, uv pip check clean on docutils 0.20.1 and 0.21.2 against pytest 8.4.2 and 9.1.1. All 20 matrix legs green.

Worth deciding: src/_pytest_doctest_compat.py still carries pytest 7 branches. Under the 8.3.3 floor they are unreachable.

Pre-rebase tips are preserved locally at backup/spike-pre-rebase and backup/adr-pre-rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant