Fix Spark master GroupPartitionsExec after expected key API change [reduced-it]

[firestarman]

Fixes #16063.

Description

The Spark master build fails because SPARK-59289 moved the expected partition keys into the planned grouping and replaced GroupPartitionsExec.expectedPartitionKeys with the planner-only expectedKeyCount. The shared Spark 4.2/5.0 GPU implementation still accessed the Spark 4.2 field directly, so the Spark 5.0 shim no longer compiled.

This change reads the expected key count through version-specific shims: Spark 4.2 derives it from expectedPartitionKeys, while Spark 5.0 reads expectedKeyCount. The value remains planner-only summary metadata; GPU execution continues to use Spark's planned partition groups.

SPARK-59289 also made the Spark 5.0 GroupPartitionsExec factory package-private and requires a valid keyed child during construction. The shared unit-test setup now uses a package-local test shim and a valid KeyedPartitioning, preserving the existing compatibility, fallback, and planner-metadata coverage on both Spark versions.

There is no user-facing configuration or behavior change.

Validation:

• Spark 5.0 GroupPartitionsExecSuite: 17 tests passed
• Spark 4.2 GroupPartitionsExecSuite: 17 tests passed
• Spark 5.0 full verify with tests skipped: build succeeded; all-module scalastyle processed 1900 files with no errors

Commands:

• mvn -s <settings.xml> -f scala2.13/pom.xml -Dbuildver=500 -Dcuda.version=cuda13 -DwildcardSuites=com.nvidia.spark.rapids.GroupPartitionsExecSuite clean package
• mvn -s <settings.xml> -f scala2.13/pom.xml -Dbuildver=420 -Dcuda.version=cuda13 -DwildcardSuites=com.nvidia.spark.rapids.GroupPartitionsExecSuite clean package
• mvn -s <settings.xml> -f scala2.13/pom.xml -Dbuildver=500 -Dcuda.version=cuda13 -DskipTests verify

AI assistance was used to prepare the implementation and pull request description.

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
  (Please provide the names of the existing tests in the PR description.)
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required

Performance testing is not required because the change only adapts driver-side planner metadata access and test construction; task-side execution, partition grouping, and sorting are unchanged.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no actionable correctness, security, or repository-rule violations remain.

Summary

This PR restores Spark 5.0 compatibility for GroupPartitionsExec while preserving Spark 4.2 behavior.

• Moves expected partition-key count access behind version-specific Spark 4.2 and 5.0 shims.
• Adds a package-local test trampoline for the Spark 5.0 package-private factory.
• Constructs test plans with valid keyed child partitioning for both supported versions.
• The only change since the previous review is explanatory documentation for the test trampoline.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[GpuGroupPartitionsExecInfo] --> B[GpuGroupPartitionsShims]
  B -->|Spark 4.2| C[expectedPartitionKeys.map size]
  B -->|Spark 5.0| D[expectedKeyCount]
  E[Shared GroupPartitionsExecSuite] --> F[GroupPartitionsExecTestShim]
  F --> G[Version-specific Spark factory]

Loading

_{Reviews (2) · Last reviewed commit: "Explain GroupPartitionsExec test shim"}

[firestarman]

build

[res-life]

LGTM