Skip to content

[ISSUE #10878] Fix tieredstore reads across file segments - #11184

Open
beautyarbutin wants to merge 1 commit into
apache:developfrom
beautyarbutin:fix/tieredstore-multi-segment-read
Open

beautyarbutin wants to merge 1 commit into
apache:developfrom
beautyarbutin:fix/tieredstore-multi-segment-read

Conversation

@beautyarbutin

Copy link
Copy Markdown

Which Issue(s) This PR Fixes

Brief Description

FlatAppendFile.readAsync previously combined at most the segment containing the requested offset and the immediately following segment. Reads spanning three or more committed segments therefore returned a truncated buffer.

This change starts an asynchronous read for every segment covered by the request, waits for all reads to complete, and combines their buffers in segment order. The existing single-segment fast path and final-segment truncation behavior are preserved.

A regression test creates three 100-byte consume-queue segments with distinct byte values, reads 250 bytes starting at offset 50, and verifies the complete length and byte ordering across all three segments.

How Did You Test This Change?

On the unmodified implementation, the new regression failed with expected.length=250 actual.length=150.

After the fix:

mvn -pl tieredstore -am -DskipITs -Dtest=FlatAppendFileTest -Dsurefire.failIfNoSpecifiedTests=false test
  • FlatAppendFileTest: 7 tests passed
  • Checkstyle: 0 violations
  • SpotBugs: no errors or warnings

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR generalizes readAsync in FlatAppendFile to handle reads spanning any number of file segments, replacing the previous hard-coded 2-segment limit. A new test validates cross-segment reads across 3 segments.

Findings

  • [Info] FlatAppendFile.java:253 — The readLength calculation for non-last segments assumes contiguous segments (no gaps). If segments are non-contiguous, fileSegment.getCommitOffset() - readOffset could yield unexpected values. Consider adding a guard or assertion for segment contiguity.
  • [Info] FlatAppendFile.java:258future.join() is called twice per future in the thenApply callback (once for remaining(), once for the buffer). Since the futures are already complete at that point, this is safe but could be slightly cleaner with a single join per future.
  • [Info] The early-return optimization for single-segment reads (line 245) is a nice touch — avoids unnecessary future allocation.

Overall

The fix is correct and well-tested. The generalization from 2-segment to N-segment reads addresses a real limitation. LGTM with minor suggestions above.


Automated review by RockteMQ-AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] FlatAppendFile readAsync truncates reads spanning more than two segments

2 participants