Repository navigation
Barrier between mark reads and compaction writes in moe_topk_softmax_mi300 - #12
Open
titoatwork wants to merge 2 commits into
Open
titoatwork wants to merge 2 commits into
titoatwork wants to merge 2 commits into
Conversation
…mi300 The compaction loop reads active_expert_ids[i] and atomicAdd-compacts into the same array with no ordering between them. Compaction writes land on indices 0..k-1, so an inactive expert below the compacted count can be read after its cleared slot is overwritten and look active (phantom expert, inflating the active set nondeterministically). Stage every mark thread-locally, __syncthreads, then compact. No behavior change at current sizes: at 64 experts with blockDim 256 only threads 0..63 run, a single wavefront, which already issues all loads before any store.
MARKS_PER_THREAD assumes one stride per blockDim.x == 256 threads (see Block size above); state it so future readers cannot silently break the bound. Tighten the latency note to the verified trigger: inactive experts below the compacted count plus cross-wavefront skew (phantom-only).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes:
The compaction loop in
moe_topk_softmax_mi300_task_implreadsactive_expert_ids[i]and atomicAdd-compacts into the same array with no ordering between them (moe_topk_softmax_mi300.cuh, 'Compact active expert marks' block). Compaction writes land on indices 0..k-1 while an inactive expert below the compacted count may not have been read yet, so a cleared slot can be made to look active: a phantom expert inflates the active set nondeterministically. This change stages every mark thread-locally, issues__syncthreads(), then compacts.Scope notes, stated precisely:
Evidence (CPU ordering model of the loop, N=512 experts, std::atomic visibility, relaxed ordering):
Verification: diff is +21/-5 in one file, no new clang-format v15 violations introduced (file has pre-existing ones, verified identical before/after outside this hunk), change compiles by inspection (constant-expression array bound, uniform barrier). No MI300X access here, so device execution is unverified. The litmus and the code-structure argument above are the evidence.
Related Issues:
Linked Issues: none filed (reporting directly).