fix: do not pass positions to RoPE when attn_backend is sdpa - #4804

Closed

ysa2215 wants to merge 22 commits into

pytorch:mainfrom

hann-wang:yue/sdpa-rope-no-positions

Commits on Feb 24, 2026

feat: add megatron dataset index helper
hann-wang
committed
5b79de9
[ForgeEngine] fix broken ForgeEngine:
hann-wang
committed
6b95d39

Commits on Feb 25, 2026

chore: move the modified main function into example_train.py
hann-wang
committed
cf2db6a
feat: add model converts to the forge engine
hann-wang
committed
21d8a4b
add model converter hooks
hann-wang
committed
0f31c75

Commits on Feb 26, 2026

feat: add more datasets
hann-wang
committed
8374dd8
feat: add llama3-1b and instella-3b model
hann-wang
committed
1e786ed
feat: option to allow partial load from hf
hann-wang
committed
72e217b

Commits on Mar 2, 2026

fix: force validation at the end of training
hann-wang
committed
62a90d9

Commits on Mar 10, 2026

add monarch to Dockerfile
hann-wang
committed
945fbb7
chore: merge upstream changes
hann-wang
committed
26c8f65
fix: llama3-1b model args
hann-wang
committed
09d90d7
chore: update dockerfile to use numpy<2
hann-wang
committed
2796021

Commits on Apr 15, 2026

chore: fetch from megatron dataset in contiguous seq
hann-wang
committed
98128cc

Commits on Apr 16, 2026

fix: instella-3b model config
hann-wang
committed
3e82232

Commits on Apr 20, 2026

Commits on Apr 21, 2026

hotfix: fsdp sharded param gradient
hann-wang
committed
bacde72

Commits on Jun 22, 2026

feat: pass the trainer to the pre_step method of modifiers
hann-wang
committed
bcabcab

Commits on Jun 25, 2026

fix: rewind the val dataloader each step
hann-wang
committed
82084e7

Commits on Sep 20, 2026

fix: do not pass positions to RoPE when attn_backend is sdpa
ysa2215
committed
9341a95
test: cover RoPE positions by attention backend
ysa2215
committed
0cfa4a3