perf(catalog): release DB pool lease during hierarchy provider I/O

[seonghobae]

Finding

The canonical corporate-entity catalog path correctly delays its explicit PostgreSQL transaction and pg_advisory_xact_lock until after hierarchy inference / external corroboration. However, get_or_create_corporate_entity(...) still receives a borrowed asyncpg.Connection and retains that pool lease while it performs potentially long asyncio.to_thread(...) inference and relation-verification calls, including recursive parent resolution.

The process pool is currently bounded at max_size=10. With enough concurrent catalog misses, provider latency can therefore consume all DB pool slots even though no transaction or explicit lock is held, delaying unrelated buyer read/write paths. #1055 adds Customer Master as another online caller of the same catalog owner, in addition to existing Keyman/post-summary ingestion paths, so this is now a buyer-visible resilience/performance boundary rather than a leaf implementation detail.

This is not a request to move catalog semantics into Customer Master, copy contextual-orchestrator routing, or weaken ADR 0010/0012/0160. The corporate-entity catalog remains the single canonical owner.

Realistic RED

Add a deterministic concurrency regression using the real pool contract or an equivalent integration harness:

1. configure the pool at its production bound (max_size=10) or a proportionally equivalent explicit test bound;
2. start enough unresolved organization resolutions to occupy every catalog path while inference/verification is deliberately blocked on a controllable provider barrier;
3. while those provider calls are blocked, issue an unrelated DB-only read through the same pool;
4. RED if the unrelated DB operation cannot obtain a connection within a bounded local latency solely because catalog calls retain idle leases during provider I/O.

Do not fake success by enlarging the pool, shortening the provider barrier, excluding the wait from measurement, or warming a cache that bypasses the miss path.

Causal architecture

Refactor the canonical catalog service boundary so database ownership is short-lived:

• acquire briefly to load candidate/alias snapshots required for deterministic matching;
• release the DB lease before hierarchy inference, search corroboration, or other network/provider I/O;
• preserve tie fail-closed behavior, SKOS alias handling, recursion/cycle/depth invariants and parent corroboration outside an explicit transaction;
• reacquire only for the fresh candidate/alias re-check, named pg_advisory_xact_lock, idempotent creation/update, and commit;
• on recursion, do not retain a parent connection while child/parent provider work executes;
• cancellation, timeout and provider exceptions must return every lease and leave no open transaction.

If a versioned repository/service abstraction is needed to express snapshot → external decision → short commit safely, keep that abstraction in the corporate-entity catalog bounded context. Callers consume it; they must not reproduce matching/creation SQL or provider selection.

Acceptance

• the concurrency RED above becomes GREEN without increasing the pool bound;
• no provider/network call executes while a DB connection is retained solely for later mutation, and no provider/network call executes inside an explicit transaction/advisory-lock scope;
• the final write still performs a fresh under-lock catalog check so concurrent duplicate creation remains prevented;
• existing ADR 0010/0012/0160 tie, alias, hierarchy, cycle/depth and idempotency tests remain GREEN;
• Keyman, post-summary and Customer Master callers preserve their current domain contracts and consume the same canonical catalog service;
• connection cleanup is proven for success, provider timeout/error and task cancellation;
• add an async buyer-path measurement (deterministic integration test and, where practical, k6/API evidence) showing an unrelated DB-only path remains serviceable while catalog provider calls are blocked;
• record the connection-ownership decision in ARCHITECTURE/ADR/OPERABILITY/TEST_STRATEGY as appropriate.

Serialization

Do not modify #1055 merely to start this follow-on while its current exact-head Strix admission is still running. Preserve single-writer/evidence boundaries: settle #1055 first, then implement this owner-level catalog repair on the then-current protected ancestry or through an already-existing verified successor if one appears.

No pool-size workaround, provider/model override, cross-service SQL, source-neutral validation commit, warning suppression, force push, destructive rebase, or gate weakening.

[seonghobae]

Additional protected-main evidence confirms this is a connection-ownership gap, not an existing transaction-discipline regression.

keyman_ingestion.ingest_post_keymen explicitly requires organization resolution/hierarchy creation to finish before its Keyman write transaction and warns callers not to wrap the function in an outer transaction. post_summary_ingestion.persist_post_summary likewise performs organization inference/verification/catalog creation before the summary replacement transaction. Those contracts correctly prevent provider latency from extending the post-owned write transaction/advisory-lock window.

The unresolved problem is one layer earlier: both functions still receive and retain the same borrowed asyncpg.Connection while get_or_create_corporate_entity performs provider inference/search corroboration. The process-wide pool is min_size=1, max_size=10. Thus ten concurrent catalog misses can consume all pool leases while doing no DB work even though the explicit transaction discipline remains correct.

The repair should therefore preserve the existing transaction-order invariants and change the catalog API's resource ownership, not weaken or move those transactions. The deterministic RED should also exercise at least one existing online caller (Keyman/post-summary or Customer Master after #1055 settles), because a catalog-unit-only fake connection would not demonstrate pool starvation.

[seonghobae]

Protected-main source read sharpens the required ownership refactor: this cannot be fixed safely inside get_or_create_corporate_entity(...) while leaving its current caller contract unchanged.

Current main@83eba56149eb802cd63642c507c324c9976ec78e exposes get_or_create_corporate_entity(conn: asyncpg.Connection, ..., candidates, aliases=...). On an alias miss it loads aliases through that same conn, performs hierarchy inference and placement/parent verification via asyncio.to_thread(...), recursively calls itself with the same connection, and only then opens the short creation transaction/advisory lock. So the transaction ordering is correct, but the callee has no ability to return the caller-owned pool lease during provider work.

ingest_post_keymen has the same shape one level up: it receives a borrowed conn, loads the candidate/alias snapshot through it, then loops affiliations and awaits _resolve_affiliated_organization(..., conn, ...) before its later short Keyman write transaction. The protected-main backfill script is even more explicit: it enters async with pool.acquire() as conn: and invokes ingest_post_keymen(conn, ...) from inside that lease. Therefore changing only the catalog function's internal transaction scope cannot make the RED green; the caller still owns the lease.

Implementation consequence for the eventual #1077 source lane: introduce a catalog-owned lease/repository boundary, not a nested acquire on top of an already-borrowed connection. A reasonable DDD shape is a pool/connection-factory-backed repository/service operation that performs (1) short snapshot load, (2) no-connection external decision/recursive hierarchy resolution, and (3) fresh short acquire + transaction + named advisory lock + recheck/write. Existing online callers should pass that service/lease capability rather than a live connection for the enrichment phase. Their later post/Keyman write transactions remain exactly where they are. Offline backfill must also stop holding one pool lease across provider work.

The regression should therefore assert lease ownership, not merely transaction() calls: while provider barriers are blocked, pool.get_size()/acquire behavior (or an equivalent deterministic harness) must show the catalog tasks have returned their leases and an unrelated DB-only operation can acquire one. Add cancellation/provider-error cases at both child and recursive-parent stages to prove every temporary lease is returned.

This is design evidence only while #1055's exact-head Strix run is still live; no competing source branch or caller mutation was started.

[seonghobae]

Serialization update after exact-head evidence settled: #1055 Strix 34746057545 is now terminal SUCCESS and #1055 has returned to Draft, so the old “Strix still live” hold is over. However the Strix artifact exposed a higher-priority authorization defect now owned by #1078: a post_read summary GET can mutate the same global corporate catalog. Keep #1077 as the canonical connection-lease/resource-ownership repair, but do not fold #1078's permission policy into it. Security #1078 should be repaired/validated first or as a non-overlapping prerequisite; then #1077 may change the catalog lease/repository boundary without obscuring who is authorized to mutate. The Strix report's claim that its fix was already applied is itself false evidence (apply_patch missed /workspace/backend/app/main.py) and is separately owned by .github#2168.