CS-697 - apply connection pool size changes without a restart

[jubrad]

Motivation

During a recent CockroachDB rolling upgrade we observed persist consensus connection pools exhausting on every node drain: each drain force-closes its share of pooled connections, deadpool silently discards them on recycle, and acquires queue while the pool re-creates connections on demand. On the busiest environments the acquire queue reached thousands of waiters, stalling every persist operation on that process and degrading freshness.

The pool cap (persist_consensus_connection_pool_max_size, pg_timestamp_oracle_connection_pool_max_size) is the natural operational lever here, but it was read once at pool construction and required a process restart to change. This PR makes it apply to live pools, so it can be raised (or lowered) fleet-wide via dyncfg ahead of planned CRDB maintenance or during an incident.

Changes

• Upgrade deadpool 0.9.5 → 0.12.3 and deadpool-postgres 0.10.3 → 0.14.1 to get Pool::resize. API migration: Manager is natively async, recycle takes a Metrics argument, pre_recycle hook errors use HookError::message. TTL-culling semantics are unchanged.
• PostgresClient::get_connection now compares the knob against the pool's current max_size and calls Pool::resize on mismatch, so both consensus and timestamp oracle pools pick up knob changes on the next acquire. Both knob implementations already read live values (dyncfg / atomics), so no plumbing changes were needed there.
• Monitoring behavior change: with deadpool ≥ 0.10, *_postgres_connpool_available is a non-negative idle count. Acquires queued on an exhausted pool were previously visible as negative available; dashboards/alerts keying on negative available will no longer fire.
• deny.toml: deadpool 0.12 requires lazy_static 1.5, so add it as a wrapper for the lazy_static ban and move the duplicate-spin skip from 0.5.2 to 0.9.9.

Tests

• New pool_resize_applies_on_acquire test in mz-postgres-client verifying grow and shrink both apply on acquire, opted in via MZ_PERSIST_EXTERNAL_STORAGE_TEST_POSTGRES_URL like the persist external-storage tests.
• Existing mz-persist consensus and mz-timestamp-oracle tests pass.
• Noticed while validating (pre-existing on main, unrelated to this change): postgres::tests::postgres_consensus fails when MZ_PERSIST_EXTERNAL_STORAGE_TEST_POSTGRES_URL points at a CockroachDB backend, because the read-committed re-run trips the get_connection serializable assertion that refuses READ COMMITTED on CRDB. The test comment expects the flag to be a no-op on CRDB, but the isolation resolver follows the flag unconditionally. Will file separately.

Checklist

• This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
• This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
• If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
• If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
• If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

🤖 Generated with Claude Code