fix(atomicmarket): bound the stats-market recompute so a backlog drains

[robrigo]

update_atomicmarket_stats_market() claimed every due row of atomicmarket_stats_markets_updates and resolved sales, auctions, buyoffers and template buyoffers in one statement, called every two minutes on the runtime pool. That pool sets a 30 second statement_timeout, and under PgBouncer transaction pooling a connection-level setting is the only one that applies, so a backlog larger than 30 seconds of work is cancelled with 57014. The cancellation rolls back the statement's own DELETE-claim, so the next tick reads the same backlog and fails the same way. The queue grows, atomicmarket_stats_markets stops advancing, and nothing recovers on its own. A reader that falls behind is enough to build the backlog, because the reader-priority gate skips the job entirely while it catches up, and the queue had no dedup key, so a hot listing added one row per write.

The recompute now claims a bounded batch, resolves it, and releases the claimed rows guarded on a captured sequence token. That is the protocol update_atomicmarket_sales_filters has used since 1.7.11 and update_atomicmarket_template_prices since 2.0.6, and this queue is the sibling 2.0.6's header names as the one still lacking it. The job moves to the max-1 long-running pool, behind the same reader-priority gate and EXISTS probe as the other two, and yields between batches rather than after a run it cannot interrupt.

The resolution carries the 1.3.23 CTE chain unchanged, with the claimed set in place of the DELETE-claim it replaces. Parity is the contract and the integration test enforces it: the reference is extracted from definitions/migrations/1.3.23/atomicmarket.sql at run time and created under a second name, so a drift in either implementation fails the comparison rather than passing silently.

Migration 2.0.10 recreates the queue with the dedup key, the sequence token and absolute autovacuum thresholds, which also deduplicates and compacts whatever backlog it holds. The dedup key carries refresh_at, which is what keeps an auction's end-time row distinct from its immediate one so the auction still resolves when its end time passes. The rebuild takes its ACCESS EXCLUSIVE lock before the copy reads a row rather than letting the DROP acquire it: a copy running under ACCESS SHARE reads its own snapshot, and an enqueue another backend commits between that snapshot and the drop would be lost with the old heap, which is a listing whose stats row never recomputes, with no error. Holding one lock across the copy and the drop closes that window and makes an overlapping filler fail the version on the 5 second lock_timeout instead, retrying on the next boot.

The function returns queue rows released where it returned stats rows written, so a batch of already-current listings still reports progress and the burn-down is not capped at one batch per tick. Nothing read the old value.

Risk

The migration does not drain the backlog; it compacts it, and the filler drains it in bounded batches once the reader is near the chain head. The deploy precondition is that the previous filler is stopped first, recorded in the migration header, UPGRADING.md and the changelog entry. A rollback needs no schema change: an earlier image calls the function with no argument, which resolves through the parameter default.

The lock-order note in the header is load-bearing. The recompute writes atomicmarket_stats_markets, which fires 2.0.6's template-prices enqueue trigger before the release touches the stats queue. That is the reverse of the order AtomicMarketHandler.deleteDB takes the two queues in, and the header records why the two never interleave within a filler.

Validation

The test database was rebuilt from scratch and the full suite run on Postgres 14 and 18, both ends of the CI matrix.

Gate	Base d4e53ab5	This branch
pnpm test:e2e:ci, Postgres 14	1116 passing, 4 pending	1141 passing, 4 pending
pnpm test:e2e:ci, Postgres 18	not run	1142 passing, 3 pending

The 25 added tests are 13 unit and 12 integration. The parity assertion was mutation-tested: altering one arm of the new recompute fails it, so the comparison bites rather than comparing two empty results. The migration was applied twice more over an already-migrated database to confirm replay, and the lock fix was verified by racing an uncommitted enqueue against the rebuild, which aborts on the lock timeout and leaves the row intact.

pnpm lint and pnpm check-types pass. pnpm knip exits 1 with findings identical to those on d4e53ab5, so that failure is pre-existing and is not a CI gate.

[Copilot]

🔵 Needs a closer look

The transactional queue rebuild and concurrency-sensitive claim/release protocol warrant final human database review.

Pull request overview

Bounds AtomicMarket stats recomputation so queued work can drain without repeatedly timing out.

Changes:

• Adds a deduplicated, sequence-guarded queue migration.
• Moves recomputation to bounded batches on the long-running pool.
• Adds unit, integration, parity, and concurrency coverage.

File summaries

File	Description
UPGRADING.md	Documents migration and deployment requirements.
stats-market-drain.test.ts	Tests batch-loop orchestration.
stats-market-drain.integration.test.ts	Tests queue and recomputation behavior.
index.ts	Implements and schedules the bounded drain.
2.0.10/database.sql	Advances the schema version.
2.0.10/atomicmarket.sql	Rebuilds the queue and bounded function.
CHANGELOG.md	Adds 2.3.4 release notes.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.