Skip to content

[Optimizer] Support per-group inner optimizers in Muon sharding - #79844

Merged
GuoxiaWang merged 1 commit into
PaddlePaddle:developfrom
xxyux:mixed-inner-optimizer-sharding
Oct 15, 2026
Merged

GuoxiaWang merged 1 commit into
PaddlePaddle:developfrom
xxyux:mixed-inner-optimizer-sharding

Conversation

@xxyux

@xxyux xxyux commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

PR Category

Distributed Strategy

PR Types

New features

Description

MuonShardingOptimizer 只能包一个 Muon,所有参数共用一套超参;分片和存档用的状态名也写死成 AdamW 的(moment1/2、beta1/2_pow_acc)。本 PR 新增一套可插拔的内层优化器接口,外加一个新的分片壳 MixedShardingOptimizer,让一次训练里可以挂多个内层优化器、每组各用自己的超参,新增优化器类型也不需要改分片壳。MuonShardingOptimizer 与现有单 Muon 用法行为不变(本 PR 不改 muon_sharding_optimizer.py)。

内层优化器接口(内部接口:本 PR 不改 python/paddle/optimizer/__init__.py,两个模块的 __all__ 为空,按模块路径导入)

  • paddle.optimizer.inner_optimizer:InnerOptimizer / ParamInfo
    • 类属性 keep_whole 声明该优化器的更新数学支持哪种切分:True 只能 V1(整张 tensor 放在一个 owner rank 上),False 只能 V2(逐元素分片),None 表示两者都正确、默认解析为 V2;未声明(KEEP_WHOLE_UNDECLARED)则构造时报错。None 时可由 Paddle 之上的框架层用 resolve_keep_whole / set_keep_whole 收窄(我们的 trainer 在展开按组优化器配置时调用),本 PR 内没有收窄的调用方,所以 None 一律是 V2。MixedInnerOptimizer 本身是异构的,没有单一的 keep_whole,对它调用 set_keep_whole 会直接 raise TypeError 并提示去收窄对应组的子优化器。
    • optimizer_state_layout() 由优化器自己声明状态名。
    • 子类只需实现 _ensure_accumulators(按参数分配状态)和 _update_parameters(就地更新本地的整张/分片参数)。
  • paddle.optimizer.mixed_inner_optimizer:MixedInnerOptimizer + register_inner_optimizer / get_inner_optimizer
    • 按参数名把每个参数路由到所属子优化器,_apply_optimize 一次调用内部按组分派。
    • 汇总各组的 state_dict / set_state_dict、master weight、状态布局与 ZCC 融合存储;grad clip 只在外层做一次。

新分片壳 MixedShardingOptimizer(python/paddle/distributed/fleet/meta_optimizers/mixed_sharding_optimizer.py)

  • 与 MuonShardingOptimizer(Sharding Stage1 V3,整张 + 逐元素混合)共用同一套通信 / 分区 / overlap 机制:color 自定义通信组、sharding_configs.machine_balanced_2d_partition 机器均衡 owner 分配、comm overlap、enable_fuse_optimizer_states、comm_group_call_opt、use_reduce_avg 等。差别只在优化器契约:
    • 内层必须是 MixedInnerOptimizer;
    • 每个参数走 V1 还是 V2 只由 inner.keep_whole_of(name) 决定,即由拥有该参数的子优化器的更新数学决定,不由 yaml 配置;
    • set_state_dict / sharded_state_dict 的状态后缀从 inner.optimizer_state_layout() 读取,自定义状态布局不必改分片壳;
    • 面向用户配置的前置检查(TP>1、PP overlap 互斥、accumulate_steps、comm_group_call_opt 前置条件、入参类型等)一律 raise ValueError/TypeError 而非 assert,python -O 下不会被剥离。
  • 接线:HybridParallelOptimizer 在内层是 MixedInnerOptimizer 时自动选用该壳,不需要新的 strategy 开关。

其他

  • MuonParamInfo 改为继承 ParamInfo(纯继承,字段与语义不变)。

单测:

  • test/legacy_test/test_inner_optimizer.py、test/legacy_test/test_mixed_inner_optimizer.py(单进程):接口契约、路由、状态汇总、master weight、ZCC 融合存储;纯 CPU 构建上只 skip 真正要 accelerator 的两个 FusionStorage 用例。
  • test/legacy_test/test_mixed_2d_partition.py(单进程,无通信组/加速器/启动器):直接给 _partition_2d_parameters_machine_balanced 喂构造数据(pp2 x ep4 x moe_sharding2 = 16 卡跨 2 台 8 卡机、dense/expert 两类通信组、bf16/fp32 两种 dtype),覆盖 owner 不重不漏、owner 数随桶体积增长、跨机负载均衡、与合并顺序无关、机器空闲 rank 耗尽时被剔除。
  • test/collective/fleet/test_parallel_dygraph_mixed_sharding.py(2 卡,3 个 case:默认 / comm overlap / enable_fuse_optimizer_states):V1/V2 混合切分下与单卡参考实现逐步对齐、各 rank 结果一致、state_dict / set_state_dict / sharded_state_dict 往返,以及机器均衡分区的端到端链路(两次 all_gather_object、各 rank 映射一致)、align 模式下 2D 梯度用 ReduceOp.SUM、grad clip、clear_grad 等壳内链路。
  • 变更行覆盖率(本地 2 卡实测,7 个 python/paddle/ 改动文件合计):96.5%。

是否引起精度变化

否

@xxyux xxyux changed the title Support per-group inner optimizers in Muon sharding [Prototype] Support per-group inner optimizers in Muon sharding Oct 8, 2026
@xxyux

xxyux commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch from 4e06902 to a68d06c Compare October 9, 2026 16:13
@Paddle-Review-Bot

Paddle-Review-Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Paddle-Bot Review Board (review完成)

序号 位置 优先级 规则来源 状态
1 新增 MixedShardingOptimizer(约1800行分布式壳层)接入 HybridParallelOptimizer,却无任何多进程 collective 用例,单测 docstring 还声称已被 collective 覆盖 P1 仓库规则:测试质量与验证 ✅
2 set_state_dict 用精确集合匹配,multi_precision 下 moment 名带 fp32_master 中缀不命中,续训静默丢失优化器状态 P1 仓库规则:功能正确性与兼容性 ✅
3 step() 丢失 1D slice 的 _is_initialized() 守卫,空分片占位张量会进入 _apply_optimize P1 仓库规则:分布式训练与并行 ✅
4 2D buffer 以 use_reduce_avg=self.use_reduce_avg 构建,group_call_opt 路径却硬 assert _use_reduce_avg,关闭 avg 时崩溃 P2 仓库规则:分布式训练与并行 ✅
5 缺少 set_grad_clip_info,配置 ClipGradByAdaptiveNorm 时运行期 AttributeError 崩溃 P2 仓库规则:功能正确性与兼容性 ✅
6 assert 承担用户配置校验(如 TP>1),python -O 下被剥离而静默放行 P3 仓库规则:安全与错误处理 ✅
7 MixedInnerOptimizer 子优化器校验仅检查 _apply_optimize,非 InnerOptimizer 会静默丢失参数信息 P3 仓库规则:安全与错误处理 ✅
8 TestMixedFusionStorage 的两个用例已加 _skip_without_accelerator 守卫,CPU-only 构建不再触发 fusion_utils 的 assert P0 仓库规则:测试质量与验证 ✅
9 PR 描述已改为模块路径并标注内部接口,与 __all__ 为空、未改 optimizer/__init__.py 的实现一致 P3 仓库规则:PR 信息与评审评论 ✅
10 machine_balanced_2d_partition 整条分区路径已补齐,与 muon 实现逐行一致,并新增 2 卡用例 P3 仓库规则:分布式训练与并行 ✅
11 _reduce_2d_grads 删掉了 align 模式强制 ReduceOp.SUM 的分支,use_reduce_avg 默认 true 时 2D 梯度被多除一次 nranks,且与同壳内 1D 路径不一致 P2 仓库规则:分布式训练与并行 ✅
12 新增 2 卡用例只断言单 owner 落在 rank 0,active_ranks 恒为 1,crowding 与多 owner 分支未被覆盖,mixed 侧缺少 muon 那样的纯函数分区单测 P3 仓库规则:测试质量与验证 ✅
13 描述称 keep_whole=None 由配置选,但无任何配置路径调用 set_keep_whole,None 恒解析为 V2,且对 MixedInnerOptimizer 调用会被静默忽略 P3 仓库规则:PR 信息与评审评论 ✅

Powered by Nyanpasu claude with Opus 5 默认推理级别, please check the suggestions carefully.

@codecov-commenter

codecov-commenter commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.81335% with 42 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@785b2a1). Learn more about missing BASE report.

Files with missing lines Patch % Lines
.../fleet/meta_optimizers/mixed_sharding_optimizer.py 95.84% 40 Missing ⚠️
python/paddle/optimizer/mixed_inner_optimizer.py 99.20% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop   #79844   +/-   ##
==========================================
  Coverage           ?   96.81%           
==========================================
  Files              ?        7           
  Lines              ?     1318           
  Branches           ?        0           
==========================================
  Hits               ?     1276           
  Misses             ?       42           
  Partials           ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch from a68d06c to d4d617d Compare October 10, 2026 03:35
@xxyux

xxyux commented Oct 10, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch 2 times, most recently from cf12add to 9866944 Compare October 13, 2026 05:51
@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch from 9866944 to 326ae18 Compare October 14, 2026 03:40
@xxyux

xxyux commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

@Paddle-Review-Bot

@xxyux

xxyux commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

@Paddle-Bot

@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch from 326ae18 to 1caca50 Compare October 14, 2026 12:34
@xxyux

xxyux commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

@Paddle-Bot

@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch from 1caca50 to e4de196 Compare October 14, 2026 13:17
@xxyux

xxyux commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

@xxyux xxyux changed the title [Prototype] Support per-group inner optimizers in Muon sharding Support per-group inner optimizers in Muon sharding Oct 15, 2026
@xxyux xxyux changed the title Support per-group inner optimizers in Muon sharding [Optimizer] Support per-group inner optimizers in Muon sharding Oct 15, 2026
Training a model with a different optimizer per parameter group (Muon on
the 2-D backbone, AdamW on everything else) currently means forking the
sharding optimizer. This adds the two pieces needed to do it in-tree:

- InnerOptimizer: a small base class for an optimizer that only owns the
  per-parameter update (_ensure_accumulators / _update_parameters), with
  the accumulator, master-weight and grad-clip plumbing shared. V1/V2
  sharding is a declared class attribute (keep_whole) exposed through
  declared_keep_whole() / keep_whole_of(); a sub-class that leaves it
  unset fails loudly at construction instead of silently sharding the
  wrong way. weight_decay is required (no implicit 0.0), so a config
  group that forgets it raises instead of quietly training without decay.

- MixedInnerOptimizer: holds N InnerOptimizer sub-optimizers, routes each
  parameter to its owner, and aggregates their accumulators and
  optimizer_state_layout() into one state tree.

- MixedShardingOptimizer: the sharding shell for the above, next to
  MuonShardingOptimizer. HybridParallelOptimizer picks it whenever the
  inner optimizer is a MixedInnerOptimizer (no strategy flag -- the choice
  follows from the inner optimizer), and unwrap_optimizer /
  _hybrid_sync_grad accept it like its sibling. Parameters whose owner
  declares keep_whole stay whole on one owner rank (V1); the rest are
  sharded element-wise (V2). The rest of the machinery is the same as
  MuonShardingOptimizer's -- color comm groups, the machine-balanced 2-D
  owner assignment, comm overlap and fused optimizer states -- while the
  checkpoint state suffixes come from optimizer_state_layout() instead of
  AdamW's hard-coded names.

Tests: test/legacy_test/test_inner_optimizer.py and
test_mixed_inner_optimizer.py cover the single-process contract;
test_mixed_2d_partition.py feeds constructed data straight into the
machine-balanced 2-D owner assignment (16 ranks over two 8-card
machines, dense and expert comm groups, two dtypes);
test/collective/fleet/test_parallel_dygraph_mixed_sharding.py (worker
hybrid_parallel_sharding_mixed_model.py) covers the 2-card shell --
V1/V2 partition, owner map agreement, convergence against a single-card
reference, state_dict round-trip and sharded_state_dict -- under the
default, comm_overlap and fused-optimizer-states configurations.
@xxyux
xxyux force-pushed the mixed-inner-optimizer-sharding branch from e4de196 to 5720a5b Compare October 15, 2026 06:05
@xxyux

xxyux commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

1 similar comment
@xxyux

xxyux commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

@GuoxiaWang GuoxiaWang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xxyux

xxyux commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

@xxyux

xxyux commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

/re-run all-failed

@From00 From00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@GuoxiaWang
GuoxiaWang merged commit 919f119 into PaddlePaddle:develop Oct 15, 2026
136 of 147 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants