Repository navigation
Conversation
Paddle-Bot Review Board (review完成)
可维护性建议(总分 86.4 / 100,只给建议,不影响评审结论)
各维度得分(仅评本 PR 改动,点击展开)
Powered by Nyanpasu claude with Opus 5.5 默认推理级别, please check the suggestions carefully. |
| CACHE INTERNAL "gin_overlap gradient communication is built") | ||
| if(WITH_GIN_OVERLAP) | ||
| message(STATUS "Enable gin_overlap sharding-stage1 gradient communication.") | ||
| add_subdirectory(gin_overlap) |
There was a problem hiding this comment.
门控开启路径目前没有任何编译证据。PR 描述写明 CMake 未运行、四个
.cu 未编译,也不确定 H-Coverage 镜像的 NCCL 是否 ≥ 2.30.7。这个门控在 sm_90+、CUDA ≥ 12.8、NCCL ≥ 2.30.7(带 nccl_device.h)的构建上会自动打开,例如 CUDA 12.x 下 CUDA_ARCH_NAME=All 时 NVCC_ARCH_BIN 含 90/100,不需要任何显式开关。如果 kernels/*.cu 或 gin_dynload.h 有编译错误,这类构建会直接失败,而门控关闭的默认 CI 发现不了。
建议合入前补一次门控开启的构建记录:configure 日志出现 Enable gin_overlap sharding-stage1 gradient communication.,且 libgin_overlap 链接成功。也可以在 PR 描述里注明是哪个 CI job 编译了该目录、它的 NCCL 版本是多少。
There was a problem hiding this comment.
已解决:门控现在要求 CUDA ≥ 13.0,CUDA 12.x 的 CI 构建不会再自动打开这个目录。PR 描述补充了门控开启时的构建记录(CUDA 13.2、NCCL 2.31.2、CUDA_ARCH_BIN="90 100 103"):configure 日志里有 Enable gin_overlap ...,libgin_overlap.a 链接成功,libpaddle.so 也链接通过。
|
|
||
| // 专家分片求和(原地):shard = Σ_k(k == expert_group_rank ? shard | ||
| // : recv[k])× scale,k 按 ring 序从 expert_group_rank+1 起 | ||
| __global__ void k_reduce_moe_shard(float* shard, |
There was a problem hiding this comment.
💡 可维护性建议(设计一致性,不阻塞合入):k_reduce_moe_shard 和 k_reduce_dense_shard(第 107 行)是同一个 ring 求和。dense 用 (node_index + k) % num_nodes,k 取 1..N;专家用 (expert_group_rank + 1 + k) % expert_group_size,k 取 0..N-1。两者访问顺序相同,寻址都是 recv + idx * numel,float4 和标量两条路径也逐行重复。文件头要求求和顺序和 __fmul_rn/__fadd_rn 次序不能改,两份副本以后只改一份,就会悄悄破坏和 reduce_scatter 的逐位一致。建议合并成一个 k_reduce_shard(shard, recv, numel, count, self_index, scale),两个 launch_* 保留为薄包装,分别传 (num_nodes, node_index) 和 (expert_group_size, expert_group_rank)。
| set(has_arch_ge90 FALSE) | ||
| foreach(arch ${NVCC_ARCH_BIN}) | ||
| if(${arch} GREATER_EQUAL 90) | ||
| set(has_arch_ge90 TRUE) | ||
| break() | ||
| endif() | ||
| endforeach() |
There was a problem hiding this comment.
💡 可维护性建议(约定偏离,不阻塞合入):本文件第 1-2 行已经用 COMPILED_CUDA_ARCHS 算出了 ARCH_BIN_CONTAINS_GE_90,供 deep_ep 门控使用。这里又遍历 NVCC_ARCH_BIN 算了一个 has_arch_ge90。两者都来自 cmake/cuda.cmake 里同一次 select_nvcc_arch_flags,含义相同,结果同一文件里有两套 ≥ sm_90 判定,以后调整架构门槛时容易只改一处。建议删掉第 29-35 行,直接用已有的 ARCH_BIN_CONTAINS_GE_90。
| // - preload_* 在初始化时把 kernel 所在的模块提前加载,免得第一次 launch | ||
| // 落在反向的关键路径上。同一进程里有在 GPU 上自旋等待的 kernel(DeepEP / | ||
| // NCCL)时,第一次加载模块会怎样没有验证过。 | ||
| // - 返回值都是 cudaGetLastError() 的结果(0 = 成功)。 |
There was a problem hiding this comment.
💡 可维护性建议(类型安全,不阻塞合入):这里约定返回值"都是 cudaGetLastError() 的结果",但 launch_*/preload_* 声明的返回类型是 int,各 .cu 里写的是 return (int)cudaGetLastError();。转换后丢了 cudaError_t 类型,host 侧只能写 rc != 0,无法直接交给 PADDLE_ENFORCE_GPU_SUCCESS 或 cudaGetErrorString 输出可读的错误。建议把返回类型统一改成 cudaError_t,去掉 (int) 转换。
247d766 to
2f08203
Compare
Paddle links neither libnccl nor libcuda, so new code that calls them has to go through the phi::dynload dlopen wrappers. The gin_overlap gradient communication added in the following commits needs three symbols that are not wrapped yet: * ncclDevCommCreate / ncclGetLsaDevicePointer, the two host entry points of NCCL's device (LSA/GIN) API. They are declared in nccl_device.h, which also carries the device-side inline implementations of the whole device API, so including it from nccl.h would land all of that in every translation unit that reaches for any NCCL symbol. The declarations it needs are repeated here instead, the way the internal entry points at the top of the file already are, and are skipped in translation units that include nccl_device.h themselves. Same 2.30.7 floor as the zero-SM group above them: below that the feature is refused, not degraded. * cuMemcpy2DAsync, for the pitched copy that splits a fused all_gather back into per-bucket storages. It is wrapped under the name libcuda exports, cuMemcpy2DAsync_v2: the wrapper dlsym()s the name it is given but types itself from decltype(&::name), which cuda.h's macro resolves to the v2 declaration, so passing the plain name would call the v1 entry point (whose parameter is the old CUDA_MEMCPY2D_v1 layout) through a v2 signature. Call sites keep writing cuMemcpy2DAsync and let cuda.h's macro rewrite it. The host overload of ncclTeamWorld is deliberately not wrapped: it shares a name with the device-side ncclTeamWorld(ncclDevComm const&), so decltype(&::ncclTeamWorld) cannot be formed, and no host code needs it. Co-Authored-By: Claude Code <noreply@anthropic.com>
ncclDevCommCreate and ncclGetLsaDevicePointer need the same NCCL 2.30.7 as the other symbols of NCCL_RAND_ROUTINE_EACH_AFTER_23007, so they join that group instead of a separate NCCL_RAND_ROUTINE_EACH_DEVICE_API under an identical #if. cuMemcpy2DAsync_v2 has no version gate and joins CUDA_ROUTINE_EACH; CUDA_ROUTINE_EACH_MEMCPY and its two expansions go. The comment on the nccl_device.h prototypes now states what actually holds: the prototypes must match nccl_device.h exactly, because this header is usually included first and both declarations then coexist; the guard only drops the duplicate in the other include order. Co-Authored-By: Claude Code <noreply@anthropic.com>
….30.7 gate Co-Authored-By: Claude Code <noreply@anthropic.com>
2f08203 to
0b211b3
Compare
First piece of moving the NCCL-GIN sharding-stage1 gradient primitives out of PaddleFleet's extension (paddlefleet_ops/gin_overlap/csrc) into Paddle: the CUDA kernels and their launch wrappers, plus the CMake gate that decides whether the directory is built at all. Nothing calls these yet; the host-side engine that launches them comes in the next commits. * kernels/reduce.cu: the intra-node column sum, the dense shard sum across nodes and the expert shard sum, each scaled in place. * kernels/put.cu: GIN one-sided puts to the same-rank peers on other nodes, to the expert-group peers, to the 2D-bucket owner, and the GIN flush. * kernels/signal.cu: the intra-node counter / arrival writes over LSA pointers, the GIN credit signal, a single release store, and the lookup of the host-window LSA pointers. * kernels/scale.cu: an in-place scale of a send-window range. * kernels/kernels.h: the launch_* / preload_* declarations the host side is allowed to call (no <<<>>> outside kernels/). * gin_dynload.h: brings the phi::dynload wrappers (previous commit) into namespace gin_overlap under their plain names, so the host code that comes next calls NCCL / the driver through dlopen without touching call sites. The code is moved, not rewritten: the sum order, the kernels and the block counts are untouched; the code-level diff against the extension is include paths, namespaces and include guards (#pragma once). Two casts of CUDA built-ins are respelled as static_cast so that the no-int64-type-cuda-index-without-cast ast-grep rule sees them; cpplint is switched off for the directory, as it is for deep_ep. The comments say what each kernel does and what must hold; reduce.cu's header lists what keeps the result bit-identical to reduce_scatter (ring order starting at own rank + 1, __fmul_rn before __fadd_rn) and states that the block / thread count is not among them: it only decides which indices a thread walks. The caller does change the reduce kernels' block count between windows within one run, and the bit-exactness check against reduce_scatter passes with that. Build gate (CMake is written but not run in this change): * arch: only built when NVCC_ARCH_BIN contains 9.0 or newer, computed the way paddle/phi/kernels/CMakeLists.txt computes has_arch_ge80. Inside the directory the <= 8.x gencodes are stripped off CMAKE_CUDA_FLAGS, as deep_ep/CMakeLists.txt does. * NCCL >= 2.30.7 with nccl_device.h present, and CUDA >= 12.8 for cudaMemcpyBatchAsync. The result is cached as WITH_GIN_OVERLAP for the pybind layer added later. gin_dynload.h #errors below the NCCL floor as a backstop for the gate. Co-Authored-By: Claude Code <noreply@anthropic.com>
Translate every comment, docstring, error message and log line of the gin_overlap sources (C++, CUDA, CMake, pybind and the Python layer) into English, and tighten the wording on the way. The terms are defined once, in the gin_engine.h file header (C++) and in the engine.py module docstring (Python); every other file uses them as defined there. Strings only. comment_clean_check.py --ignore-strings passes on every changed file: C++ / CUDA / CMake token streams are identical except for string literals (printf conversions inside merged literals keep their order), and the Python ASTs are identical except for str constants and the literal parts of f-strings. Log lines all carry the [gin_overlap] prefix; copy_2d errors now start with "copy_2d:". Co-Authored-By: Claude Code <noreply@anthropic.com>
…the gate Both cudaMemcpyBatchAsync calls use the CUDA 13 signature (8 arguments, const void** sources). CUDA 12.8 / 12.9 declare a 9-argument version, so a build with the gate open on those toolkits would not compile. Raise the gate to CUDA >= 13.0, the only toolkit the code has been built and run with, instead of adding an untested 12.x branch; gin_dynload.h gets the matching #error backstop. The batch attribute index is named attrs_idx, which is what that argument is. The arch condition reuses ARCH_BIN_CONTAINS_GE_90, computed at the top of the same file for DeepEP, instead of a second loop over NVCC_ARCH_BIN. _BUILD_HINT in the Python layer now lists exactly the gate's conditions, and the CMake comment says to change both together. Co-Authored-By: Claude Code <noreply@anthropic.com>
…ne API
Argument checks (values come from Python):
- scale_range, gather_put_to_owner, send_window_tensor and copy_2d check
their bounds without unsigned wrap-around, and reject numel < 0.
copy_2d also rejects non-contiguous tensors (a stride-0 or transposed
view passes a numel-based bound but the copy reads contiguous bytes).
- release::setup checks every offset of the per-bucket tables and the
host window areas: non-negative, aligned (counters 8 B, data 4 B) and
inside its window. setup_gather_table checks both arrival counter
offsets of each entry the same way. CommGroup records recv_bytes for
this, and alloc_symmetric_window returns the size it actually
registered instead of the caller repeating the 4096 B minimum.
- A wrong state returns cudaErrorNotReady and a bad argument
cudaErrorInvalidValue (gather_setup, gather_put_to_owner, scale_range).
- bucket_ready rejects a bucket reported twice in a step in the RUN state
too, from a bitmap only the main thread touches.
Init failures:
- configure destroys the streams and events it created, frees its
temporary buffer and resets the globals it set when a later step fails,
so it can be retried. set_comms clears the groups it set on failure.
release::setup unpublishes its control block if the tail thread cannot
be started. The steps that cannot be rolled back (window registration,
devcomm creation, the host window) say so: restart the process.
- The gin_engine.h constraints state that the engine lives as long as
the process; shutdown_process_group / restart_process_group are not
supported.
API:
- GIN_CHECK replaces the dozen { rc = launch_x(...); if (rc) return rc; }
blocks. launch_* / preload_* return cudaError_t.
- launch_put_to_expert_peers takes (send_offset, recv_offset) like
launch_put_to_node_peers.
- join_bucket_streams / gather_wait_bucket take kind (0 dense, 1 expert),
as bucket_ready and the gather table do, instead of is_dense.
- The six release positions are named variables, one per stage.
- The methods that do not return a code (record_window_mark, tail_mark,
bucket_ready, tail_all_posted, tail_wait, tail_abort, release_activity,
the version getters) document what they return.
- tail_wait / tail_abort release the GIL while they block.
No kernel, reduction order, release condition or credit target changes.
Co-Authored-By: Claude Code <noreply@anthropic.com>
kernels/release.cu has no kernel and no <<<>>>: it is the host-side release state machine and its std::thread. kernels/ holds only the kernel launch wrappers, so the file moves next to gin_engine.cc as release.cc and is compiled by the host compiler instead of nvcc. The content is unchanged apart from the file name in comments. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Remove gin_overlap/ from the cpplint exclude list and fix every finding: C-style casts become static_cast / reinterpret_cast, long long and unsigned long long become int64_t / uint64_t, non-const reference out-parameters become pointers (call sites updated), and C system headers come before C++ ones. atomicAdd_system keeps its unsigned long long prototype through one aliased type. No change to values, types seen by Python, or arithmetic. Co-Authored-By: Claude Code <noreply@anthropic.com>
…+ sources only The four warning options are wrapped in $<$<COMPILE_LANGUAGE:CXX>:...>, so nvcc no longer receives them for kernels/*.cu. Co-Authored-By: Claude Code <noreply@anthropic.com>
0b211b3 to
fe0edcc
Compare
| const int k = blockIdx.x * blockDim.x + threadIdx.x; | ||
| if (k >= num_peers || k == skip_rank) return; |
There was a problem hiding this comment.
💡 可维护性建议(契约一致性,不阻塞合入):k_signal_credit 是一个线程对应一个 peer(k = blockIdx.x * blockDim.x + threadIdx.x),没有 grid-stride 循环。num_blocks * 128 < num_peers 时,多出的 peer 会被 k >= num_peers 直接跳过,对端收不到 credit,现象只会是等待挂住。kernels.h 第 162 行只写了 <<<num_blocks, 128>>>,文件头约定里 block 数只影响 reduce 的并行度,调用方从接口上看不出这里的 block 数决定正确性。#79970 目前传的是 (num_peers + 127) / 128,结果正确,但这条约束只存在于调用方。
建议在 launch_signal_credit 内部按 num_peers 计算 grid 并去掉 num_blocks 参数,或改成 grid-stride 循环;至少在 kernels.h 写明 num_blocks * 128 >= num_peers 这个前置条件。
There was a problem hiding this comment.
已解决:kernels.h 第 167-168 行写明了前置条件 num_blocks * 128 >= num_peers,signal.cu 中 k_signal_credit 的注释也说明它没有 grid-stride 循环,grid 线程数必须不少于 num_peers。调用方现在能从接口上看到这条约束。
…edit Co-Authored-By: Claude Code <noreply@anthropic.com>
PR Category
Distributed Strategy
PR Types
New features
Description
[WIP] gin_overlap 1/3: dynload wrappers, device kernels and the build gate of the NCCL-GIN gradient-communication primitive (#79964). This is the first PR of the series; Files changed shows only this PR. #79970 (engine and release layer) depends on it, #79971 (pybind, Python layer, packaging, tests) on #79970. It replaces the earlier #79966 (dynload), which is closed; its description is merged below. Compiled with the gate on (CUDA 13.2, NCCL 2.31.2,
CUDA_ARCH_BIN="90 100 103"); not run on a GPU yet.Depends on: develop (
63c4ef64e6). Callers after merge: none; the kernels are launched by the engine, which #79970 adds.Part 1 (formerly #79966): dynload wrappers
What this part adds
Paddle links neither libnccl nor libcuda, so new code that calls them has to go through the
phi::dynloaddlopen wrappers. The gin_overlap code in the following PRs needs three symbolsthat are not wrapped yet:
ncclDevCommCreate/ncclGetLsaDevicePointer(paddle/phi/backends/dynload/nccl.{h,cc}): the twohost entry points of NCCL's device (LSA/GIN) API. They are declared in
nccl_device.h, which alsocarries the device-side inline implementation of the whole device API, so
nccl.hrepeats the twodeclarations instead of including it (the same way the internal entry points at the top of the file
are declared), and skips them in translation units that include
nccl_device.hthemselves.Same 2.30.7 floor as the zero-SM group above them.
cuMemcpy2DAsync(paddle/phi/backends/dynload/cuda_driver.{h,cc}): wrapped under the name libcudaexports,
cuMemcpy2DAsync_v2. The wrapperdlsym()s the name it is given but types itself fromdecltype(&::name), whichcuda.h's macro resolves to the v2 declaration; passing the plain namewould call the v1 entry point through a v2 signature. Call sites keep writing
cuMemcpy2DAsync.The host overload of
ncclTeamWorldis not wrapped: it shares its name with the device-sidencclTeamWorld(ncclDevComm const&), sodecltype(&::ncclTeamWorld)cannot be formed, and no hostcode needs it.
Callers after merge: none in this part. The first caller is
gin_overlap/gin_dynload.h(Part 2),used from #79970 (Part 3) on.
What CI can show: the
cuMemcpy2DAsync_v2wrapper is compiled on every CUDA build, so CIshows it compiles. The two NCCL wrappers sit behind
#if NCCL_VERSION_CODE >= 23007; they areonly compiled where the NCCL headers are 2.30.7 or newer. I have not checked which CI images
have such headers (
setup.pypinsnvidia-nccl-cu12at 2.25.1 to 2.28.3), so CI may not compilethem at all. No CI job calls any of the three.
How correctness is established: the wrappers are only exercised by the gin_overlap engine
(#79970 (Part 3) onwards). The bitwise evidence so far is for earlier, out-of-tree states of that engine
(4 nodes / 32 cards, 10 steps, identical to Paddle's
reduce_scatterbaseline; see Part 2), not forthis tree. The acceptance gate is the same 10-step AdamW and Muon run on a build of the whole
series. The whole series has been compiled (see Part 2); nothing has been run on a GPU yet.
Changes after the first review
ncclDevCommCreate/ncclGetLsaDevicePointerjoinedNCCL_RAND_ROUTINE_EACH_AFTER_23007andcuMemcpy2DAsync_v2joinedCUDA_ROUTINE_EACH; the two separate groups and their expansions in the.ccfiles are gone.nccl_device.hprototypes now says what holds: they must matchnccl_device.hexactly, because this header is usually included first and both declarations then coexist; the guard only drops the duplicate in the other include order.Changes after the second review
NCCL_RAND_ROUTINE_EACH_AFTER_23007says thatncclDevCommCreate/ncclGetLsaDevicePointerare the host entry points of the device (LSA/GIN) API, also first in 2.30.7, so they share the gate of the zero-SM entry points. Comment only.Part 2 (formerly #79967): device kernels and the build gate
What this part adds
The CUDA kernels and their launch wrappers, and the CMake gate that decides whether the new
directory
paddle/fluid/distributed/collective/gin_overlap/is built at all.kernels/reduce.cu: the intra-node column sum, the dense shard sum across nodes and theexpert shard sum, each scaled in place. The file header lists what keeps the result
bit-identical to
reduce_scatter: ring order starting at own rank + 1, and__fmul_rnbefore__fadd_rnper element. The block / thread count is not on that list; it only decides whichindices a thread walks.
kernels/put.cu: GIN one-sided puts to the same-rank peers on other nodes, to the expert-grouppeers and to the 2D-bucket owner, and the GIN flush.
kernels/signal.cu: the intra-node counter / arrival writes over LSA pointers, the GIN creditsignal, a single release store, and the lookup of the host-window LSA pointers.
kernels/scale.cu: in-place scale of a send-window range.kernels/kernels.h: thelaunch_*/preload_*functions; host code calls only these(no
<<<>>>outsidekernels/).gin_dynload.h: brings thephi::dynloadwrappers of Part 1 intonamespace gin_overlapundertheir plain names.
#errorbelow NCCL 2.30.7 as a backstop for the gate.collective/CMakeLists.txt: the gate. The directory is built only withWITH_DISTRIBUTE,WITH_GPU,WITH_NCCL, an arch ≥ 9.0 (ARCH_BIN_CONTAINS_GE_90, the variable the DeepEP gate in the same file uses), CUDA ≥ 13.0(the
cudaMemcpyBatchAsyncsignature) and NCCL ≥ 2.30.7 withnccl_device.hpresent. The result is cached asWITH_GIN_OVERLAPfor [WIP][Distributed] gin_overlap 3/3: pybind, Python layer, packaging and tests #79971.gin_overlap/CMakeLists.txt: strips the ≤ 8.x gencodes (asdeep_ep/CMakeLists.txtdoes) andbuilds
cc_library(gin_overlap)from the four kernel files;DEPS phi common..pre-commit-config.yaml: no change any more (cpplint runs ongin_overlap/).The comments use terms (window, LSA, credit, …) that are defined in the
gin_engine.hfile header,which arrives in #79970 (Part 3).
Callers after merge: none. The kernels are launched by the engine (#79970 (Part 3) / 4), and the library
is linked into libpaddle only in #79971.
What CI can show: on a build where the gate is off (CUDA below 13.0, no arch ≥ 9.0, or NCCL below
2.30.7), that the gate evaluates and the build is otherwise unchanged.
H-Coveragebuilds with CUDA 12.9, sowith the gate at CUDA 13.0 it does not build this directory; no CI job is known to. The build with the gate on
is described below. No test runs the kernels.
How correctness is established: the kernels are moved from PaddleFleet's extension
(
paddlefleet_ops/gin_overlap/csrc); the code-level diff against it is include paths, namespaces,include guards, and two casts respelled as
static_castfor theno-int64-type-cuda-index-without-castast-grep rule. The bitwise evidence so far is for earlierstates of this code, not for this tree: 4 nodes / 32 cards, 10 steps, loss, every norm, every
bucket before / after communication and all parameters identical to Paddle's
reduce_scatterbaseline (with the baseline made to sum in the same order). AdamW was last verified with the
code still in PaddleFleet (PaddleFleet
a50e8f5b); Muon before the narrow-callback change of#79958. Since then the release decision was moved into C++ and the code was moved into Paddle.
The acceptance gate is the same 10-step AdamW and Muon run on a build of the whole series.
The whole series has been compiled (see Part 2); nothing has been run on a GPU yet.
Changes after the first review
cudaMemcpyBatchAsynccalls use the CUDA 13 signature; 12.8 / 12.9 declare a different one, so those toolkits would not compile. There is no 12.x branch: 13.2 is the only toolkit the code has been built and run with.gin_dynload.hhas a matching#error.ARCH_BIN_CONTAINS_GE_90from the top of the same file instead of a second loop.launch_*/preload_*returncudaError_tinstead ofint.launch_put_to_expert_peerstakes(send_offset, recv_offset), the order oflaunch_put_to_node_peers.k_reduce_moe_shardandk_reduce_dense_shard. It is a kernel change and touches the summation code, so it waits for the bitwise acceptance run.Build with the gate on (P2 of the review): source tree = this series on top of #79958,
cmakewithCUDA_ARCH_BIN="90 100 103",WITH_DISTRIBUTE=ON, CUDA 13.2, NCCL headers 2.31.2. The configure log hasand the incremental
make -jafter these review fixes finished with exit 0 in 109 min (nice -n 10 make -j180; the dynload headers changed, so most ofphi_gpuwas recompiled; the log has noerror:);Linking CXX static library libgin_overlap.a/Built target gin_overlap, thenlibpaddle.solinks; the wheel containspaddle/distributed/communication/gin_overlap/. No CI job is known to build this directory.Changes after the second review
gin_dynload.h: the comment no longer says libcudart is linked statically. Runtime API calls such ascudaMemcpyBatchAsyncgo straight to libcudart, not throughphi::dynload; on Linux libcudart is linked dynamically (--cudart shared, top-levelCMakeLists.txt:113-117), so libpaddle.so needs a libcudart with these entry points (CUDA >= 13.0, the#errorbelow it). Comment only.cpplint:
gin_overlap/.*is removed from the cpplint exclude list in.pre-commit-config.yaml, and every finding of cpplint 1.6.0 (pre-commit arguments; 377 findings, now 0 forgin_overlap/andpybind/gin_overlap_api.*) is fixed in code: C-style casts become thestatic_cast/reinterpret_castwith the same meaning;long long/unsigned long longbecomeint64_t/uint64_t; non-const reference out-parameters (release::advance/status/stuck_report/constants/get_gather_sent, the matchingGradCommEnginemethods,set_comms'serr,fma_rn4's accumulator) become pointers, with every call site (includinggin_overlap_api.cc) updated; C system headers come before C++ ones. OneNOLINT(runtime/int)remains:signal.cualiasesunsigned long longforatomicAdd_system, whose CUDA prototype takes that type (uint64_tisunsigned longon Linux). NoCPPLINT.cfg. The Python side sees the same types (pybind mapsint64_t/uint64_tand their vectors toint/list[int], as before). Checked: a token-level comparison (comments removed, casts / integer types / pointer forms normalized) of every changed file shows no other change except the added#include <cstdint>inkernels.h. SASS check:cuobjdump -sassof the fourkernels/*.cu.obefore and after the change, per kernel and arch (sm_90 / sm_100 / sm_103): reduce 9/9, put 12/12, signal 15/15, scale 3/3 functions with identical instruction bodies (reduce / put / scale byte-identical; in signal only the mangled names ofk_store_release/k_signal_intra_arrivechange,y->m, fromunsigned long long->uint64_t). Incremental build of exactly this tree (no re-cmake,make -j180, 18:18-19:16, exit 0, noerror:in the log):Built target gin_overlap,libpaddle.solinks (its symbols carry the pointer signatures), the wheel containspaddle/distributed/communication/gin_overlap/.The four
-Wno-*options oftarget_compile_options(gin_overlap ...)now apply to C++ sources only (each wrapped in$<$<COMPILE_LANGUAGE:CXX>:...>, P0 of the first review): inflags.makeof the build,CXX_FLAGShas them andCUDA_FLAGSdoes not; incompile_commands.jsonthe four.ccfiles carry them and the fourkernels/*.cudo not. The rebuild (cmake re-run by make, 0error:lines, no warning from gin_overlap files,libgin_overlap.aandlibpaddle.solink, wheel built) recompiled the kernels;cuobjdump -sassbefore / after is identical per kernel and arch (reduce 9/9, put 12/12, signal 15/15, scale 3/3 functions, SASS text byte-identical).Not verified: the precision acceptance run (4 nodes, bitwise) is still in progress, and the changes of this round have not been through it. The behavior changes of this round are the argument check in
release::setup; the cpplint fixes change casts, integer type spellings and reference -> pointer parameters only.是否引起精度变化
否
🤖 Generated with Claude Code