Skip to content

[MLPerf 6.1][GRPO] Update Qwen3.5 validation and timeout rules - #598

Merged
ShriyaRishab merged 1 commit into
mlcommons:masterfrom
jepio:fix/qwen35-qualified-reference-rules
Sep 20, 2026
Merged

ShriyaRishab merged 1 commit into
mlcommons:masterfrom
jepio:fix/qwen35-qualified-reference-rules

Conversation

@jepio

@jepio jepio commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Update the qualified Qwen3.5 validation set from 256 tasks to 251 tasks and the pass@4 rollout count from 1024 to 1004.
  • Fix the training and validation agent timeout at 1,800 seconds.
  • Fix the training and validation test and command timeouts at 60 seconds.

Rationale

Dataset validation removed five invalid validation tasks. The updated NeMo-RL reference also finalizes the runtime limits that affect rollout and reward behavior.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@nvidia.com>
@jepio
jepio requested review from a team as code owners September 17, 2026 15:40
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@ShriyaRishab

Copy link
Copy Markdown
Contributor

Approving based on task force email threads.

@ShriyaRishab
ShriyaRishab merged commit d5e6bdd into mlcommons:master Sep 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants