Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25574 +/- ##
=======================================
Coverage 82.50% 82.50%
=======================================
Files 1140 1140
Lines 438050 438077 +27
Branches 438050 438077 +27
=======================================
+ Hits 361410 361434 +24
+ Misses 54833 54830 -3
- Partials 21807 21813 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…r-mirroring # Conflicts: # benchmarks/sql_benchmarks/asof_join/asof_join.suite # datafusion/optimizer/src/push_down_filter.rs # datafusion/sqllogictest/test_files/asof_join.slt
Xuanwo
marked this pull request as ready for review
September 24, 2026 12:57
Member
Author
|
Hi @jayzhan211 and @2010YOUY01, this follow-up is now restacked and ready for review. Would you mind taking a look when you have time? Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
When a filter selects one ASOF equality-key group on the left, rows in other right-side groups cannot match any surviving left row. The broadcast join still sorts and retains those right rows unless the same key filter is applied to the right input.
Performance
Q09 has 1M rows on each side across 10K equality groups.
WHERE l.key = 42retains one group and produces 100 rows with both revisions. On an Apple M4 Max, using release builds and two 7-iteration runs per revision, the median elapsed time across all 14 iterations was:mainwithout this optimization (b172f73f0)b3fd5f1f9)This selective keyed workload is about 7.8x faster (87% lower elapsed time). The benchmark file and command were the same for both binaries:
The branch was subsequently synced through
21a3215b6; those upstream-only commits do not touch the optimizer or ASOF paths exercised by Q09.What changes are included in this PR?
left_key = literalpredicate to the corresponding right key when both join keys are direct columns of the same type.asof_join.slt, plus a keyed selective workload as ASOF benchmark Q09.What is the testing strategy for this PR?
The SLT cases check the mirrored plan, matching and unmatched results, a right-side
IS NULLfilter, and a left disjunction that must not prune the right input. Q09 runs the user-visible query with 1M rows per side across 10K groups, selecting one group.Are there any user-facing changes?
No API or result changes. Eligible ASOF joins can sort and retain fewer right-side rows.