Skip to content

Let the migrations extra stand on its own - #61

Merged
EdwardPham1615 merged 1 commit into
mainfrom
fix/persistence-lazy-init
Sep 27, 2026
Merged

EdwardPham1615 merged 1 commit into
mainfrom
fix/persistence-lazy-init

Conversation

@EdwardPham1615

Copy link
Copy Markdown
Owner

Found while starting on the Candidate-additions plan: py-common[migrations] cannot import py_common.persistence.migrations.

ImportError: The SQLAlchemy asyncio module requires that the Python 'greenlet'
library is installed. In order to ensure this dependency is available, use the
'sqlalchemy[asyncio]' install target

Pre-existing on main — confirmed by reproducing it on a clean main checkout before touching anything.

Cause

Importing that submodule runs persistence/__init__.py, which imported engine and therefore sqlalchemy.ext.asyncio, whose greenlet dependency only the persistence extra's sqlalchemy[asyncio] brings. The Alembic helpers are synchronous and need none of it.

So the package __init__ was charging an importer for dependencies its own extra does not declare — the same bug 0.2.1 fixed in runtime, http and telemetry, in the one package that fix did not reach. persistence/__init__.py now resolves re-exports on first access (PEP 562). No public name moved.

How it hid, which is the interesting part

Whether it fails depends on what a fresh resolve happens to pull in. CI was green on the last dependabot PR (#60) while a local run went red — plausibly a platform or resolution difference.

scripts/check-extras-isolation.sh installs .[extra] fresh rather than from the lockfile, which is the only reason this surfaced. That is the blind spot Candidate addition #3 is about, catching something by accident.

Two gaps closed alongside

extras-check now imports py_common.testing.routes (http) and py_common.testing.tokens (security). They are public API a consumer imports in its own test suite and each carries its own extra — previously untested, leaving a hole exactly where a helper is most likely to reach across packages. I hit that hole immediately: a first pass at Candidate #5 had testing/routes.py importing py_common.security, and nothing would have caught it.

tests/test_lazy_exports.py covers the lazy-__init__ contract for all three packages — every exported name resolves, an unknown one raises AttributeError, dir() matches __all__, and importing the package does not pull its heavy submodule. The 0.2.1 mechanism had no direct test until now; it was verified only through a script that needs a throwaway environment per extra and so cannot run in this suite.

That last test runs in a subprocess on purpose: this suite has already imported everything, so a sys.modules check in-process would pass or fail on test ordering rather than on behaviour.

Verification

399 passed (from 387), coverage 92.24%, make extras-check green on all twelve extra/module pairs.

Mutation-checked twice:

  • revert persistence/__init__.py to eager → extras-check fails on [migrations] only
  • add one eager import back → test_importing_the_package_does_not_import_its_heavy_submodule[py_common.persistence] fails

Sequencing

This lands before the seven Candidate-addition PRs — it was blocking make extras-check as a verification step for all of them. Candidate #5 is parked on a local branch and will rebase onto this.

@EdwardPham1615

Copy link
Copy Markdown
Owner Author

Update: this branch's lint-test failure is not this change — it is the MinIO image becoming unpullable from every anonymous registry today, fixed in #62.

extras-isolation and audit pass here; #62's extras-isolation fails on exactly the bug this PR fixes. The two are red only on each other's fault.

Recommended order: merge #62 first (a red lint-test blocks every test), then re-run this one.

`py-common[migrations]` could not import
`py_common.persistence.migrations` -- it raised "The SQLAlchemy asyncio
module requires that the Python 'greenlet' library is installed".

Importing that submodule runs persistence/__init__.py, which imported
`engine` and therefore sqlalchemy.ext.asyncio, whose greenlet dependency
only the `persistence` extra's sqlalchemy[asyncio] brings. The Alembic
helpers are synchronous and need none of it. So the package __init__ was
charging an importer for dependencies its own extra does not declare --
the same bug 0.2.1 fixed in runtime, http and telemetry, in the one
package that fix did not reach.

persistence/__init__.py now resolves re-exports on first access. No
public name moved.

Worth recording how it hid: whether it fails depends on what a fresh
resolve pulls in, so CI was green on the last dependabot PR while a local
run went red. extras-check resolves fresh rather than from the lockfile,
which is the only reason it showed up at all -- the same blind spot
candidate #3 is about, catching something by accident.

Two gaps closed alongside it. extras-check now imports
py_common.testing.routes and .tokens: public API a consumer uses in its
own suite, each carrying its own extra, previously untested. And
tests/test_lazy_exports.py covers the lazy-__init__ contract for all three
packages -- every exported name resolves, an unknown one raises
AttributeError, dir() matches __all__, and importing the package does not
pull its heavy submodule. That last one runs in a subprocess, because this
suite has already imported everything and a sys.modules check would pass
on test ordering. The 0.2.1 mechanism had no direct test until now.
@EdwardPham1615
EdwardPham1615 force-pushed the fix/persistence-lazy-init branch from 05e01a0 to 2cdd48f Compare September 27, 2026 13:15
@EdwardPham1615

Copy link
Copy Markdown
Owner Author

Rebased onto main now that #62 has landed. Local: make check green (399 passed, 92.24%), make extras-check green on all twelve extra/module pairs — including the [migrations] case this PR fixes and the two py_common.testing.* cases it adds.

@EdwardPham1615 EdwardPham1615 self-assigned this Sep 27, 2026
@EdwardPham1615 EdwardPham1615 added bug Something isn't working documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file labels Sep 27, 2026
@EdwardPham1615
EdwardPham1615 merged commit 4958e39 into main Sep 27, 2026
3 checks passed
@EdwardPham1615
EdwardPham1615 deleted the fix/persistence-lazy-init branch September 27, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant