Scale node table shard count with search threads to cut match/selfplay memory peak - #30
Closed
ChinChangYang wants to merge 1 commit into
Closed
Scale node table shard count with search threads to cut match/selfplay memory peak#30ChinChangYang wants to merge 1 commit into
ChinChangYang wants to merge 1 commit into
Conversation
…y memory peak Every Search allocated a SearchNodeTable with a fixed 2^16 = 65536 shards by default, regardless of how small the search actually is. Each shard costs a mutex plus a std::map, and Search keeps a second mutex pool of the same size, so a single Search reserved ~8 MiB of empty tables before searching anything. In the match and selfplay subcommands every game thread holds its own Search (and in match, potentially one per side), so this fixed overhead is multiplied by numGameThreads and dominates the peak memory footprint. With, e.g., 256 game threads at one search thread each, that is ~2 GiB of empty hash tables. The shard count only needs to be large enough to keep lock contention low across the search threads of a single Search, which in match/selfplay is typically one. Default nodeTableShardsPowerOfTwo based on numSearchThreads (~128 shards per thread), clamped to [8, 16], so lightweight single-threaded searches use 256 shards (~32 KiB) while heavily-threaded searches keep the historical 65536-shard behavior. An explicit nodeTableShardsPowerOfTwo in the config still overrides this. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UgRZ525h6fWmDfE6G9vin8
ChinChangYang
marked this pull request as draft
June 24, 2026 16:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every Search allocated a SearchNodeTable with a fixed 2^16 = 65536 shards by
default, regardless of how small the search actually is. Each shard costs a
mutex plus a std::map, and Search keeps a second mutex pool of the same size,
so a single Search reserved ~8 MiB of empty tables before searching anything.
In the match and selfplay subcommands every game thread holds its own Search
(and in match, potentially one per side), so this fixed overhead is multiplied
by numGameThreads and dominates the peak memory footprint. With, e.g., 256 game
threads at one search thread each, that is ~2 GiB of empty hash tables.
The shard count only needs to be large enough to keep lock contention low
across the search threads of a single Search, which in match/selfplay is
typically one. Default nodeTableShardsPowerOfTwo based on numSearchThreads
(~128 shards per thread), clamped to [8, 16], so lightweight single-threaded
searches use 256 shards (~32 KiB) while heavily-threaded searches keep the
historical 65536-shard behavior. An explicit nodeTableShardsPowerOfTwo in the
config still overrides this.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01UgRZ525h6fWmDfE6G9vin8