Skip to content

Fix double retry loop with no backoff - #88

Merged
unladenSwallow merged 4 commits into
mainfrom
fix/retry-double-loop-no-backoff
Sep 22, 2026
Merged

unladenSwallow merged 4 commits into
mainfrom
fix/retry-double-loop-no-backoff

Conversation

@unladenSwallow

@unladenSwallow unladenSwallow commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Note

🤖 Generated with Claude Code

Summary

  • RunTest already retries a failing test in its own loop, but SendHTTPRequest also wrapped the request in a go-retryablehttp client configured with the same retry count/callback — every outer-loop attempt triggered its own inner retry cycle, multiplying request volume.
  • The retryablehttp.Client was constructed as a bare struct literal (never via NewClient()), so RetryWaitMin/RetryWaitMax defaulted to zero — retries fired back-to-back with no delay.
  • On exhaustion, go-retryablehttp's Do() discards the actual *http.Response and returns only a generic "... giving up after N attempt(s)" error, even when the request got a real (just non-matching) response — masking the true status code/body from test output across every repo using this tool.

Changes

  • Removed the inner go-retryablehttp client from SendHTTPRequest; it now issues a single plain http.Client request.
  • RunTest's existing retry loop is now the sole retry mechanism, with an actual 2s backoff between attempts.
  • Dropped the now-unused github.com/hashicorp/go-retryablehttp dependency (go mod tidy).

Test plan

  • go build ./...
  • go vet ./...
  • go test ./...
  • Run against a real service host to confirm retries now show real status/body on failure instead of "giving up after N attempt(s)"

Ran tests in dev service. Link available to internal users by request.

unladenSwallow and others added 4 commits September 21, 2026 07:53
RunTest already retries failed requests in a loop, but SendHTTPRequest
also wrapped the request in a go-retryablehttp client configured with
the same retry count and callback. Every outer-loop attempt triggered
its own inner retry cycle, multiplying request volume, and since the
client was a bare struct literal (never NewClient()), RetryWaitMin and
RetryWaitMax defaulted to zero, so retries fired back-to-back with no
delay.

Worse, go-retryablehttp's Do() discards the actual http.Response and
returns only a generic giving-up error once its internal retries
exhaust, even for a request that got a real (non-matching) response.
That's why failures across services using this tool showed an opaque
"giving up" error instead of the real status code or body.

Drop the inner retry client entirely, make SendHTTPRequest a single
plain http.Client call, and let RunTest's existing loop own all retry
logic with an actual backoff between attempts.

Generated with Claude Code (https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
The lint workflow used go-version: stable, which now resolves to
1.27.x. golangci-lint v2.8.0's binary is built with Go 1.25 and panics
when asked to type-check source declaring a newer go directive than
its own toolchain (file requires newer Go version go1.27, application
built with go1.25). Use go-version-file so lint runs on the same Go
version the module actually targets, matching the pattern already
used in action.yml.

Co-Authored-By: Claude <noreply@anthropic.com>
The delay between retry attempts was hardcoded to 2 seconds. Repos
with many tests and frequent retries want control over how much that
adds to CI time, so expose it as RETRY_BACKOFF_SECONDS / a
retry-backoff-seconds action input, defaulting to the previous
hardcoded value of 2 seconds when unset.

Co-Authored-By: Claude <noreply@anthropic.com>
@unladenSwallow
unladenSwallow marked this pull request as ready for review September 22, 2026 17:24
@unladenSwallow
unladenSwallow requested review from a team as code owners September 22, 2026 17:24
@unladenSwallow
unladenSwallow requested review from Aravind3515, BobbyBonaguraNYT and stephenhetterich and removed request for a team September 22, 2026 17:24

@togdon togdon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@unladenSwallow
unladenSwallow merged commit 07b17d8 into main Sep 22, 2026
3 checks passed
@unladenSwallow
unladenSwallow deleted the fix/retry-double-loop-no-backoff branch September 22, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants