Skip to content

Add host to network errors outside registry client - #16963

Merged
v-abhishekbhaskar merged 2 commits into
mainfrom
abhishekbhaskar/add-error-context-network-errors
Nov 18, 2026
Merged

v-abhishekbhaskar merged 2 commits into
mainfrom
abhishekbhaskar/add-error-context-network-errors

Conversation

@v-abhishekbhaskar

Copy link
Copy Markdown
Contributor

What are you trying to accomplish?

Cargo and Terraform make some registry requests outside Dependabot::RegistryClient, or parse registry responses without checking them first. When these fail, the unknown error has no host in its error-context, so it can't be debugged from Sentry. This is part of the work to add the host to network errors after Dependabot::ErrorReportContext.

Cargo accounts for about 95% of the events:

  • FileParser#sparse_registry_source_details: a 403 from a sparse registry's config.json returns a plain Forbidden body. That body went straight into JSON.parse, which raised JSON::ParserError with no host. RegistryClient only attaches the host on socket errors and timeouts, not on HTTP error responses.
  • Package::PackageDetailsFetcher: called Excon.get directly and only handled 404. Any other non-2xx response, often with an empty body, was parsed and failed. Its socket errors and timeouts also had no host.
  • MetadataFinder: the same direct-Excon.get and unchecked-parse pattern.

Terraform:

  • Package::PackageDetailsFetcher: called Excon.get directly.
  • RegistryClient#all_provider_versions / #all_module_versions: parsed 200 responses without guarding against non-JSON bodies.

Changes:

  • Cargo: added Dependabot::Cargo::RegistryResponse:
    • raise_for_error_status! maps 401/403 to PrivateSourceAuthenticationFailure and any other non-2xx to PrivateSourceBadResponse. These are classified errors, so they no longer count as unknown errors.
    • with_parse_error_context attaches host, http-method and http-status to JSON::ParserError and re-raises it.
    • All three call sites use these helpers. The fetcher and metadata finder now use Dependabot::RegistryClient.get, so socket errors and timeouts also carry the host.
  • Terraform: the package details fetcher uses Dependabot::RegistryClient.get. JSON parse failures there and in all_*_versions get the registry host attached.

Only the host is ever attached, never the full URL: registry paths can contain private crate or module names.

Anything you want to highlight for special attention from reviewers?

  • Classified errors are only raised for private registries. For crates.io, a non-2xx response that isn't JSON still raises JSON::ParserError, now with host/http-status. Reporting a crates.io outage as a "private source" error would be misleading.
  • The 403 case is now a classified error. In the sparse registry case, a 403 now raises PrivateSourceAuthenticationFailure instead of carrying a host as an unknown error. This takes it out of the unknown-error bucket entirely.
  • Behavior change for private registries. Non-2xx responses other than 404 now raise PrivateSourceBadResponse. Previously, error bodies that happened to be valid JSON were parsed silently and produced no versions.
  • Terraform socket errors. LatestVersionResolver rescues StandardError around every call to Terraform's PackageDetailsFetcher, so errors from that fetcher can't reach Sentry. The change there mainly adds RegistryClient defaults (timeouts, proxy, host caching). The observed Terraform Excon::Error::Socket events likely come from another code path.

How will you know you've accomplished your goal?

  • Cargo file_parser_spec: sparse config.json returns 403 Forbidden → PrivateSourceAuthenticationFailure; 502 → PrivateSourceBadResponse; 200 with non-JSON → JSON::ParserError with host.
  • Cargo package_details_fetcher_spec: crates.io 503 with an empty body → JSON::ParserError with host: crates.io; private registry 403 → PrivateSourceAuthenticationFailure; private registry 500 → PrivateSourceBadResponse.
  • Cargo metadata_finder_spec: crates.io 502 with a non-JSON body → parse error with host.
  • Terraform registry_client_spec and package_details_fetcher_spec: non-JSON registry responses → parse error with host.

Checklist

  • I have run the complete test suite to ensure all tests and linters pass.
  • I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
  • I have written clear and descriptive commit messages.
  • I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
  • I have ensured that the code is well-documented and easy to understand.

@v-abhishekbhaskar v-abhishekbhaskar self-assigned this Nov 17, 2026
@v-abhishekbhaskar
v-abhishekbhaskar requested a review from a team as a code owner November 17, 2026 22:22
Copilot AI balanced review requested due to automatic review settings November 17, 2026 22:22
@github-actions github-actions Bot added L: terraform Terraform packages L: rust:cargo Rust crates via cargo labels Nov 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Malformed JSON from a cross-host service endpoint is attributed to the discovery host rather than the server actually queried, affecting both provider and module version lookups.

1 open issue
What changed in this PR

This PR improves Cargo and Terraform registry error diagnostics by adding host context to JSON parse failures and routing direct requests through the shared registry client.

Changes:

  • Adds Cargo response helpers to classify private-registry HTTP errors and annotate JSON parse failures.
  • Uses the shared registry client for Cargo metadata and package details, and Terraform package details.
  • Adds regression coverage for HTTP failures, malformed responses, and host-only error context.
File Description
terraform/​spec/​dependabot/​terraform/​registry_client_spec.rb Tests host context on provider-version parse failures.
terraform/​spec/​dependabot/​terraform/​package/​package_details_fetcher_spec.rb Tests host-only context on malformed provider responses.
terraform/​lib/​dependabot/​terraform/​registry_client.rb Annotates provider and module version parse errors.
terraform/​lib/​dependabot/​terraform/​package/​package_details_fetcher.rb Uses shared requests and adds parse-error context.
cargo/​spec/​dependabot/​cargo/​package/​package_details_fetcher_spec.rb Covers public-registry parse failures and private-registry HTTP errors.
cargo/​spec/​dependabot/​cargo/​metadata_finder_spec.rb Tests context on malformed crates.io responses.
cargo/​spec/​dependabot/​cargo/​file_parser_spec.rb Covers sparse-registry HTTP and parse failures.
cargo/​lib/​dependabot/​cargo/​registry_response.rb Adds shared status-validation and parse-error helpers.
cargo/​lib/​dependabot/​cargo/​package/​package_details_fetcher.rb Uses shared requests and response handling.
cargo/​lib/​dependabot/​cargo/​metadata_finder.rb Uses shared requests, handles 404s, and annotates parse failures.
cargo/​lib/​dependabot/​cargo/​file_parser.rb Validates sparse-registry responses before parsing.

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 17, 2026 22:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the Cargo and Terraform registry error-reporting changes.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@v-abhishekbhaskar
v-abhishekbhaskar force-pushed the abhishekbhaskar/add-error-context-network-errors branch from 3a0c5c4 to 286f65d Compare November 18, 2026 21:24
Copilot AI balanced review requested due to automatic review settings November 18, 2026 21:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the Cargo and Terraform registry diagnostics changes.

No open issues 🎉

Review effort: Balanced

@v-abhishekbhaskar
v-abhishekbhaskar merged commit b5f324f into main Nov 18, 2026
100 checks passed
@v-abhishekbhaskar
v-abhishekbhaskar deleted the abhishekbhaskar/add-error-context-network-errors branch November 18, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L: rust:cargo Rust crates via cargo L: terraform Terraform packages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants