Skip to content

Tag session-close, statement-close, and rate-limit errors for telemetry#424

Draft
prathamesh-db wants to merge 1 commit into
prathamesh/PECOBLR-3537-tag-remaining-sitesfrom
prathamesh/PECOBLR-3537-tag-caller-sites
Draft

Tag session-close, statement-close, and rate-limit errors for telemetry#424
prathamesh-db wants to merge 1 commit into
prathamesh/PECOBLR-3537-tag-remaining-sitesfrom
prathamesh/PECOBLR-3537-tag-caller-sites

Conversation

@prathamesh-db

Copy link
Copy Markdown
Collaborator

What

Tags the remaining live error sources so telemetry reports a specific
error_name instead of the generic error:

Category Site
session_closed failed CloseSession (connection.go, DELETE_SESSION telemetry)
statement_closed failed CloseOperation RPC (connection.go, the 3 CLOSE_STATEMENT sites)
rate_limit_exceeded HTTP 429 that survives all retries (internal/client/client.go retry errorHandler)

All tags are telemetry-only and use the existing WithCategory chaining
(same concrete pointer), so errors.Is/errors.As, Error() strings, and the
errors returned to callers are unchanged:

  • session_closed tags a separate telemetry copy; the returned error stays a
    driver.ErrBadConn (database/sql relies on that for connection-pool eviction).
  • statement_closed tags only the telemetry argument; the raw close error still
    flows to the caller/rows.
  • rate_limit_exceeded is tagged in the retry errorHandler — the only place the
    429 status is visible (net/http discards the response on a transport error).
    Scoped to 429 exactly, so 503 keeps its existing classification. This covers the
    retry-exhausted failure only; a per-attempt 429 counter metric is a TODO follow-up.

Notes for reviewers (by design, not bugs)

  • statement_closed is only live on the Thrift backend; the kernel backend's close
    never returns an error, so it can't be tagged there.
  • Because the innermost source category wins, a close that fails specifically due to
    a 429 reports rate_limit_exceeded rather than session_closed/statement_closed
    — the more specific category. Worth knowing for dashboards.

Naming note

session_closed has no exact JDBC equivalent (JDBC's nearest is CONNECTION_CLOSED);
kept Go-specific here — flagging in case cross-driver dashboard parity is preferred.


Stacked on the earlier PECOBLR-3537 PRs; base is the previous branch, auto-retargets to main as they merge.

Design doc: https://docs.google.com/document/d/12ufP1eZrgFxWt6xzINfkhfrE-NnhB29zCa5PKokVfp8/edit
Jira: PECOBLR-3537

@prathamesh-db prathamesh-db self-assigned this Jul 23, 2026
@prathamesh-db
prathamesh-db force-pushed the prathamesh/PECOBLR-3537-tag-caller-sites branch from 4a03e2a to f8afecc Compare July 23, 2026 05:11
Tags the remaining live error sources so telemetry reports a specific
error_name instead of the generic "error":

- session_closed: a failed CloseSession is tagged on a telemetry-only copy
  in conn.Close before RecordOperation(DELETE_SESSION). The returned error
  is left untouched so it stays a driver.ErrBadConn (database/sql relies on
  that for connection-pool eviction).
- statement_closed: a failed CloseOperation RPC is tagged via a
  telemetry-only helper at the three RecordOperation(CLOSE_STATEMENT) sites
  (sync exec + the two async close-callback paths). The raw error still
  flows to the caller/rows unchanged.
- rate_limit_exceeded: an HTTP 429 that survives all retries is tagged in
  the retry errorHandler, which is the only place the 429 status is visible
  (net/http discards the response on a transport error). Scoped to 429
  exactly, so 503 keeps its existing classification. This covers the
  retry-exhausted failure only; a per-attempt 429 counter metric is left as
  a TODO follow-up.

All tags are telemetry-only and use WithCategory (same concrete pointer),
so errors.Is/As, driver.ErrBadConn identity, Error() strings, and the
returned errors are unchanged. Tests assert each tagged error yields the
right category and that 503/500 are not tagged rate_limit and the returned
session-close error is still driver.ErrBadConn.

Signed-off-by: Prathamesh Baviskar <prathamesh.baviskar@databricks.com>
@prathamesh-db
prathamesh-db force-pushed the prathamesh/PECOBLR-3537-tag-caller-sites branch from f8afecc to 96c8029 Compare July 23, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant