Add lock timeout retry support to apply - #558
Conversation
Statements that fail to acquire a lock before --lock-timeout expires (SQLSTATE 55P03) can now be retried with exponential backoff via the new --lock-timeout-retries and --lock-timeout-retry-wait flags, easing migrations against busy tables (fixes #557). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011yLiZRSQzzPgpGz8ymX9SV
Greptile SummaryThe PR adds configurable retries with capped exponential backoff for PostgreSQL lock-not-available errors in both grouped and individual migration execution paths.
Confidence Score: 4/5The PR appears safe to merge, with non-blocking input and API validation gaps that should be tightened to keep retry behavior predictable. The retry paths are covered and preserve immediate handling of non-retryable errors, but malformed backoff values can disable the intended waiting behavior and direct ApplyMigration callers can bypass the lock-timeout prerequisite. Files Needing Attention: cmd/apply/apply.go Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
Apply[Apply migration group] --> Exec[Execute SQL]
Exec -->|Success| Next[Continue migration]
Exec -->|Non-55P03 error| Fail[Return error]
Exec -->|55P03 and retries remain| Wait[Wait with capped exponential backoff]
Wait -->|Context cancelled| Cancel[Return context error]
Wait -->|Timer expires| Exec
Exec -->|55P03 and retries exhausted| Fail
Reviews (1): Last reviewed commit: "Add lock timeout retry support to apply" | Re-trigger Greptile |
There was a problem hiding this comment.
Pull request overview
Adds lock-timeout retry behavior to pgschema apply so migrations can tolerate transient lock contention by retrying 55P03 failures with exponential backoff.
Changes:
- Introduces
--lock-timeout-retriesand--lock-timeout-retry-waitflags and wires them into both grouped (concatenated) and per-statement execution paths. - Implements
55P03detection and a reusableexecWithLockRetryhelper with exponential backoff capped at 30s. - Adds unit + embedded-Postgres integration tests, and documents the new flags with an example.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/cli/apply.mdx | Documents the new retry flags and provides an example invocation. |
| cmd/apply/apply.go | Adds retry configuration/flags, validation, and lock-timeout retry execution logic. |
| cmd/apply/apply_test.go | Extends CLI flag/validation tests for the new options. |
| cmd/apply/lock_retry_test.go | Adds unit tests for SQLSTATE detection and retry behavior. |
| cmd/apply/lock_retry_integration_test.go | Adds integration tests validating retries succeed/fail as expected under real lock contention. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| select { | ||
| case <-ctx.Done(): | ||
| return nil, ctx.Err() | ||
| case <-time.After(backoff): | ||
| } |
There was a problem hiding this comment.
Not making this change. time.After in a select/ctx.Done() loop only "leaks" a timer until it fires or becomes unreachable; since Go 1.23 the runtime can garbage-collect an unreferenced, unstopped Timer without waiting for it to fire, so there's no real leak here. This loop also isn't a hot path - it only re-fires on an actual lock-timeout error, bounded by --lock-timeout-retries (a handful of iterations per statement at most), and the existing wait-directive polling loop in cmd/apply/directive.go uses the exact same time.After pattern for consistency. Happy to switch both to time.NewTimer if this ever becomes a tight, high-frequency loop, but for now it'd be defensive coding against a cost that isn't there.
Generated by Claude Code
Reject negative --lock-timeout-retries and a non-positive --lock-timeout-retry-wait instead of silently treating them as no retries / a tight retry loop. ApplyMigration re-validates the same invariants so direct callers can't bypass the CLI's checks. Also reword the retry progress message from "attempt" to "retry" to match what the counter actually tracks. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FA3axbY2LQkGbN7b6d2fY8
Summary
--lock-timeout-retriesand--lock-timeout-retry-waitflags topgschema apply. When a statement fails because it could not acquire a lock before--lock-timeoutexpired (PostgreSQL error55P03), it is retried with exponential backoff (capped at 30s) instead of failing immediately.CREATE INDEX CONCURRENTLY).--lock-timeout-retriesrequires--lock-timeoutto be set, since without a lock timeout a statement blocks indefinitely instead of failing with a retryable error.This addresses #557, which asked for a retry mechanism for gracefully handling lock contention on busy tables (per the referenced zero-downtime migrations technique).
Test plan
go build ./...go vet ./...isLockTimeoutErrorandexecWithLockRetry(success-after-retries, retries-exhausted, non-retryable error, zero retries, context cancellation during backoff)55P03errorgo test ./cmd/apply/...suite passesdocs/cli/apply.mdxwith the new flags and an exampleGenerated by Claude Code