Repository navigation
ops: cutover fixes from the live run - #136
Merged
Merged
Conversation
Logical replication from Railway (PG 18) into self-hosted Supabase (supabase/postgres 17.6.1.136, pinned self-hosted/v0.8.2) so the copy runs while production writes and the cutover costs minutes of failed writes, not hours of downtime. - server/setup-server.sh: official Supabase stack plus a nichedb overlay: Postgres on 5432 with TLS, pg_hba letting only `postgres` in from outside and only over TLS (compose subnet pinned so Docker's gateway counts as outside), anon/authenticated locked out of schema public, RAM-sized tuning, pooler on loopback, secrets kept off the terminal. - nichedb-db: provision, dns, vault, check, source-prep, schema, subscribe, status, indexes, verify, cutover, rollback, cleanup, apply-migration. - test/rehearse.sh: full local rehearsal on the production schema with a live writer; passes (checksums, generated column, sequences, read-only source, TLS/role/anon rules, the app's own Bun SQL pool). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The access token lives about an hour, after which every GraphQL call is 'Not Authorized'. A mid-cutover expiry would stop the DATABASE_URL switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prepare-db creates the database inside a cluster another app already runs on the box (one Supabase stack per host), locks anon/authenticated out of that database only, and writes the connection file. dns/vault skip what belongs to the cluster's owner. Index log lives in /tmp, not their dir. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
db.nichedb.dev was invented and is gone (record deleted). DB_DOMAIN now defaults to the target host, dns is opt-in, setup-server uses hostname -f. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
app-deploy/app-cert/app-vhost stage nichedb.dev under ~/www/nichedb.dev (checkout, docker-compose.app.yml with Redis, app.env from Railway's variables, nginx vhost, cert pre-issued over Porkbun DNS-01 so the flip has no TLS gap). cutover then points Railway's app at the new DB as web only, starts the app on the box, and moves the A records at Porkbun. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merges keep landing on main while the copy runs, and the app now builds some indexes after boot; the index step must carry whatever exists then. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… backends) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three add/add conflicts, all in the ops kit: main carries the version from the earlier PR and this branch carries its successor. The branch wins all three, because it is the later work and a superset. - README.md: the branch drops the new DNS names (db.nichedb.dev, supabase.nichedb.dev) in favour of the server's existing hostname, and adds the shared-cluster runbook and the note on why subscribe hangs. - nichedb-db: the branch keeps every helper main's copy has and adds prepare-db, for becoming a database inside a Supabase stack another app already owns. - server/setup-server.sh: same lineage. Suite on the merged tree: 1,908 pass, 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dev2's 5432 is allowlisted and Railway egress is not fixed. Railway keeps serving its read-only copy for the DNS TTL instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One conflict, in nichedb-db, and all three hunks are the same decision: this branch stops repointing Railway's DATABASE_URL at the new database, which is what the PR is for. main carries the older behaviour. - cutover: Railway stays on its read-only copy for the DNS TTL and only drops to ROLES=web. dev2's 5432 is allowlisted and Railway's egress addresses are not fixed, so pointing it at the box is not reliable. - rollback: nothing to restore, because DATABASE_URL was never changed. OLD_DATABASE_URL goes with it and is referenced nowhere else. - the help text for cutover, to match. bash -n clean on both scripts; suite 1,911 pass, 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A failed CREATE INDEX CONCURRENTLY leaves an invalid empty index on the source that pg_dump omits; it blocked the cutover with 163 vs 162. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
app.env is taken from Railway's variables, which are web-only while the copy runs; the box takes both roles at cutover. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things the live cutover to dev2 hit:
Also corrects the note printed after the DNS flip.
🤖 Generated with Claude Code