Support GCP Cloud SQL with IAM Database Authentication (and Workload Identity Federation)

[akshat-kumar-singhal]

Summary

Add first-class support in GoFr's SQL datasource for connecting to GCP Cloud SQL (Postgres & MySQL) using IAM Database Authentication, with credentials resolved via Application Default Credentials so that Workload Identity Federation (WIF) works transparently — no service-account keys on disk, no static DB_PASSWORD, no Cloud SQL Auth Proxy sidecar.

Motivation

Today, pkg/gofr/datasource/sql accepts a static DBConfig (DB_USER / DB_PASSWORD) read once at startup and passed to sql.Open. That works fine for self-hosted Postgres/MySQL but leaves GCP users with three poor options:

1. Store a long-lived DB password in env/Secret Manager and rotate manually.
2. Run the Cloud SQL Auth Proxy as a sidecar and pretend the DB is on localhost — extra container, extra failure mode.
3. Drop down to raw database/sql and skip GoFr's SQL datasource entirely, losing health checks, metrics, retries, and OTel tracing.

Modern GCP deployments (GKE Workload Identity, Cloud Run, GitHub Actions via WIF) increasingly prohibit long-lived service-account keys. GoFr should make the secret-less path the easy path.

Proposal

Reuse the existing DB_DIALECT values (postgres, mysql) and introduce a single switch:

DB_DIALECT=postgres                 # or mysql — no new dialect values
DB_HOST=my-proj:us-central1:my-inst # instance connection name (in IAM mode)
DB_USER=app-sa@my-proj.iam          # SA email (IAM DB user)
DB_NAME=app
DB_IAM_AUTH=true                    # ← the switch: use Cloud SQL connector + IAM
DB_CLOUDSQL_IP_TYPE=PUBLIC          # PUBLIC | PRIVATE | PSC (default: PUBLIC)
# No DB_PASSWORD. Credentials resolved via ADC → WIF works transparently.

When DB_IAM_AUTH=true, NewSQL uses cloud.google.com/go/cloudsqlconn with WithIAMAuthN() to mint and auto-refresh OAuth2 tokens used as the DB password; DB_PORT and DB_SSL_MODE are ignored (the connector owns TLS). Health checks, retry-on-disconnect, metrics, and OTel tracing match existing NewSQL behavior.

Packaging — separate module to keep core lean

cloudsqlconn brings ~30 transitive GCP SDK deps. To avoid taxing every GoFr binary, the implementation would live in a new module pkg/gofr/datasource/sql/cloudsql/ (mirroring mongo, gcs, etc.). Core sql.go adds a small routing branch: when DB_IAM_AUTH=true, delegate driver registration to the cloudsql module. Users who don't import the module and don't set DB_IAM_AUTH see zero behavior or dependency change.

Out of scope (for the initial PR)

• AlloyDB — same pattern, separate follow-up using alloydbconn.
• Spanner / Firestore — not database/sql-shaped; needs a different datasource interface.
• Cloud SQL Proxy sidecar mode — the connector replaces it.
• PgBouncer compatibility — IAM tokens expire mid-pool-life; will be a documented limitation, not solved.

Questions for the maintainers

1. Routing location — happy with a small branch in sql.NewSQL that delegates to the cloudsql module when DB_IAM_AUTH=true? Alternative is an explicit app.AddSQL(cloudsql.New()) call (zero core change but two config surfaces).
2. Switch naming — is DB_IAM_AUTH the right name, or would DB_CLOUDSQL_IAM_AUTH (more explicit, leaves room for AWS RDS IAM later) read better?
3. Credentials model — any objection to ADC-only (no GoFr-specific key-path knob; users set GOOGLE_APPLICATION_CREDENTIALS per Google's convention if they need to pin a key)?
4. CI integration tests — appetite for a //go:build integration gate keyed on CLOUDSQL_TEST_INSTANCE env var, or keep maintainer-manual only?

Happy to open the PR off development once direction is agreed.

[aryanmehrotra]

Thanks for the detailed write-up @akshat-kumar-singhal — this is a good idea and well scoped.

Rather than branching inside core sql.NewSQL, the cleaner fit is to support this the way every other datasource already works: add an AddSQLDB method on the app that lets users plug in their own SQL implementation (e.g. a Cloud SQL IAM connection), overriding the default.

This ties into work we already have queued up:

#2113 (Separate out datasources to a new repo) — the longer-term plan is to pull SQL and Redis out of core into self-contained modules, with core keeping only the interface and a registry doing the wiring. SQL and Redis are the last two still baked into core.

#1259 (Remove datasources' dependency on GoFr) — same north star from the other side: datasources shouldn't be coupled to a specific GoFr version.

Fully removing SQL and Redis from core today (as #2113 envisions) would be a breaking change — it'd force a v2, which we don't want to do right now. AddSQLDB gets us most of the way there without breaking anything: it's purely additive, so no v2, and it's exactly the plug-in seam the #2113 registry will need anyway. So this moves us toward that refactor rather than against it.

A few things this gets us at once:

• Users can replace the default SQL DB with their own (Cloud SQL IAM being the first use case here).
• The cloudsql implementation lives in its own module, so GCP's ~30 deps never touch core.
• It paves the way to support things like SQL Server in future the same way — as a pluggable module, without it ever being in core.

[akshat-kumar-singhal]

Rather than branching inside core sql.NewSQL, the cleaner fit is to support this the way every other datasource already works: add an AddSQLDB method on the app that lets users plug in their own SQL implementation (e.g. a Cloud SQL IAM connection), overriding the default.

My proposal was along the lines of allowing users to run the same image via regular username/password on local and IAM on GCP without writing any conditions/code specific to deployed env. Also, given that this is only a change in auth process and the underlying implementation of SQL remains unchanged, should we have a a new method created for this?

[aryanmehrotra]

Yes — conceptually this is just one more auth variant and the SQL impl underneath is unchanged. But putting that auth support directly in core would bloat GoFr's dependency tree with the GCP SDK for every user, even those who never touch Cloud SQL. That's exactly why we've been pulling datasources out into their own published modules — and SQL and Redis are the last holdouts still in core, only because removing them outright would be a breaking change / v2 we're not doing yet.

So the additive path: make SQL swappable like the other datasources — an AddSQLDB seam on the app — and publish the Cloud SQL/IAM implementation as its own GoFr module (the way gofr.dev/pkg/gofr/datasource/mongo, clickhouse, etc. already ship). The default stays byte-for-byte what we ship today; this just adds an IAM auth-variant alongside it, living in its own module so the GCP deps never reach core.

It keeps your config-driven / same-image goal intact too: you wire it the same way as any external datasource — read app.Config.Get(...) and pass the values into the module's constructor — so it's password auth locally, IAM on GCP, decided entirely by env with no per-deployment code branches.

[akshat-kumar-singhal]

Implemented in #3569