You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(wpcli): improve segment inspection and scan diagnostics - #419
When an Active segment has no loaded writer or discoverable local directory, scan/inspect/probe cannot select its storage instance from coordinator metadata alone. Scan hides the per-replica errors behind an unknown summary, while a local-only successful survey can overstate end-to-end readability.
This change adds paired --bucket / --root-path flags to all three commands and forwards the escaped instance location to each quorum member. Omitting both retains existing node-side discovery.
Scan now exposes per-replica local ranges, survey stop reasons, and HTTP/transport/response errors in text and structured output. All-replica confirmed local emptiness on an Active segment is distinguished from incomplete or unavailable inspection results. A missing range remains unknown while any replica is unavailable or has an incomplete survey: a short observed copy cannot rule out a healthy uninspected tail. Raw checksum failures are not labelled as local emptiness, and responses lacking a survey stop reason are rejected as unusable.
Probe no longer declares a permanent read stall solely from all replicas reporting no local data for an Active segment. Finalized-segment shortfall checks remain in place. Help, summaries and the CLI README clarify that scan surveys local coverage/blocks; it does not verify pending writes, shared compacted objects, or application progress.
Validation:
go build -o /tmp/woodpecker-wp-418 ./cmd/wpcli
go test -race ./cmd/wpcli/... -count=1 -timeout=3m
golangci-lint run --config .golangci.yml --timeout=3m ./cmd/wpcli/...
Regression tests cover paired flag validation and escaped forwarding, confirmed empty Active vs finalized absence, checksum-failed blocks, unusable responses, preserved replica error details and JSON per-replica coverage.
❌ Patch coverage is 94.84536% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.12%. Comparing base (c45123a) to head (3089e71). ⚠️ Report is 1 commits behind head on master.
Must-fix issues introduced by this PR: 0 Merge recommendation: Mergeable as-is — this review found no must-fix issue introduced by this PR.
One-sentence overview, then one item under Medium, then the pre-existing section.
Adversarial review of the --bucket/--root-path location plumbing found one path that the new flags do not reach: the orphan-segment gap probe still issues its inspect request without a location, which can turn a held orphan into a false "data is gone" conclusion.
Medium
cmd/wpcli/cmd/log_scan.go:111 — The gap-probe closure passed to reconcileScanWithGaps calls probeGapForData without forwarding the location variadic that runLogScan now accepts, so probeGapForData (line 175) builds its /admin/logstore/segment/inspect request (line 182) without bucket_name/root_path. The other three inspect/probe paths (scanSegmentNodes, inspectEachNode, probeEachNode) do forward it, and the README now documents that log scan accepts these flags to select an instance. Failure: an orphan segment id (missing from metadata) has its data directory on a node where several storage instances hold the same log_id/segment_id directory, or local auto-discovery fails, and the operator runs wp log scan <log> --bucket X --root-path Y → the server's resolveProbeInstance returns "held by N instances; pass bucket_name and root_path" as HTTP 404, probeGapForData treats non-200 as "this node has nothing" (asked=true, continue, lines 188-190), and the scan reports "segment id N is missing from metadata and no node holds data for it … the record is gone rather than the data" (log_scan_reconcile.go:410-413) for data that a node actually holds. Suggestion: add a location ...scanLocation parameter to probeGapForData, append bucket_name/root_path the same way the other three paths do, forward location... from the closure, and add a gap-probe location-forwarding assertion to segment_location_test.go (which currently covers only the regular scan/inspect/probe paths). (raised by sijie-ni-0214, xiaocai2333)
Pre-existing hazards — not introduced by this PR (follow-up only)
The following hazards are real but pre-existing and were not introduced by this PR. They should be tracked as separate follow-up work and do not block this PR.
cmd/wpcli/cmd/log_scan_reconcile.go:251 — The "behind / need resyncing" block subtracts each answered replica's coverage from the union held (built at line 210) without checking node.completeClaim(), even though scanNode.stopped (lines 49-51) and completeClaim (lines 64-66) define coverage as a claim only when the survey reached the end of the segment, and a lower bound otherwise. Failure: a Completed/Active segment with two replicas where node-1's inspect stops early on max_blocks (coverage 0-99) and node-2 reaches the end (coverage 0-199) → the scan prints "node-1 is missing 100-199, which another replica holds — those replicas are behind and need resyncing", reporting a healthy but partially surveyed replica as behind and potentially prompting an unnecessary resync. A fix would restrict the behind judgement to replicas where completeClaim() is true, or report "survey bounded" when !settled. Unrelated to this PR because the diff touches scanNode fields, confirmedLocalEmpty, the settled initialisation and replica loop, the open-segment and !settled branches, and boundedReasons, while the behind block at lines 243-260 and the construction of held are unchanged line for line, so behaviour is identical before and after this PR. No tracking issue yet. (raised by sijie-ni-0214, xiaocai2333)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When an Active segment has no loaded writer or discoverable local directory, scan/inspect/probe cannot select its storage instance from coordinator metadata alone. Scan hides the per-replica errors behind an unknown summary, while a local-only successful survey can overstate end-to-end readability.
This change adds paired
--bucket/--root-pathflags to all three commands and forwards the escaped instance location to each quorum member. Omitting both retains existing node-side discovery.Scan now exposes per-replica local ranges, survey stop reasons, and HTTP/transport/response errors in text and structured output. All-replica confirmed local emptiness on an Active segment is distinguished from incomplete or unavailable inspection results. A missing range remains unknown while any replica is unavailable or has an incomplete survey: a short observed copy cannot rule out a healthy uninspected tail. Raw checksum failures are not labelled as local emptiness, and responses lacking a survey stop reason are rejected as unusable.
Probe no longer declares a permanent read stall solely from all replicas reporting no local data for an Active segment. Finalized-segment shortfall checks remain in place. Help, summaries and the CLI README clarify that scan surveys local coverage/blocks; it does not verify pending writes, shared compacted objects, or application progress.
Validation:
go build -o /tmp/woodpecker-wp-418 ./cmd/wpcligo test -race ./cmd/wpcli/... -count=1 -timeout=3mgolangci-lint run --config .golangci.yml --timeout=3m ./cmd/wpcli/...Based directly on master; independent of #417.
Closes #418