Skip to content

fix: restrict reverse cast constraint propagation to safe conversions - #25531

Merged
jayzhan211 merged 1 commit into
apache:mainfrom
haohuaijin:fix/25523-lossy-cast-constraints
Sep 20, 2026
Merged

jayzhan211 merged 1 commit into
apache:mainfrom
haohuaijin:fix/25523-lossy-cast-constraints

Conversation

@haohuaijin

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #25523.
Related to #25407.

Rationale for this change

A filter on a cast can incorrectly make its input appear constant and remove a required sort. For example, CAST(x AS INT) = 0 matches both -0.5 and 0.5, but casting the singleton result interval back to DOUBLE yields [0.0, 0.0]. The optimizer can then return the wrong order for ORDER BY x DESC.

What changes are included in this PR?

Gate reverse Cast constraint propagation through a common allowlist for all types. Preserve propagation for conversions recognized by check_bigger_cast, integer-to-integer casts, and Float32-to-Float64 casts. For other conversions, keep the existing input range instead of treating a cast back as an inverse.

The conservative allowlist may reduce range refinement and pruning for safe conversions that are not yet recognized. Runtime CAST behavior and forward interval evaluation are unchanged.

What is the testing strategy for this PR?

  • Extend existing cast.slt with float-to-integer, integer-to-Float32, timestamp-to-date, and string-to-integer cases. The first three reproduce incorrect sort elimination before the corresponding guards; the string case checks compatibility.
  • Add a table-driven test for permitted and rejected reverse propagation paths.
  • Check Float32 widening against actual Cast results at NaN, infinity, signed zero, subnormal, and rounding boundaries.
  • Passed 24 Cast unit tests (one existing test ignored), 46 filter tests, and four relevant SLT files.

Are there any user-facing changes?

Queries retain required sorting when cast constraints cannot safely determine the input range. No public API changes.

@github-actions github-actions Bot added physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt) labels Sep 20, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.97872% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.38%. Comparing base (b4a8c82) to head (fcbcbd5).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/physical-expr/src/expressions/cast.rs 82.97% 3 Missing and 13 partials ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #25531    +/-   ##
========================================
  Coverage   82.38%   82.38%            
========================================
  Files        1138     1138            
  Lines      434328   434582   +254     
  Branches   434328   434582   +254     
========================================
+ Hits       357824   358040   +216     
- Misses      54872    54882    +10     
- Partials    21632    21660    +28     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jayzhan211 jayzhan211 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @haohuaijin , LGTM!

@haohuaijin

Copy link
Copy Markdown
Contributor Author

Thanks for your reviews @jayzhan211

@jayzhan211
jayzhan211 added this pull request to the merge queue Sep 20, 2026
Merged via the queue into apache:main with commit 4e603e1 Sep 20, 2026
41 checks passed
@haohuaijin
haohuaijin deleted the fix/25523-lossy-cast-constraints branch September 20, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lossy cast constraint propagation can eliminate a required sort

3 participants