Skip to content

[memory] load, parse: a categorical model loads in under half the heap, and the loaded model keeps no category lists or spare capacity - #65

Merged
victor-ab merged 7 commits into
mainfrom
victor-ab/load-memory
Nov 7, 2026
Merged

victor-ab merged 7 commits into
mainfrom
victor-ab/load-memory

Conversation

@victor-ab

@victor-ab victor-ab commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Loading a model with many categorical splits took a heap several times the loaded model's, and the loaded model kept data that production never reads. On a large private model with many categorical splits, 3.2.2's heap peaked at 2.4 times the loaded model when loading from a file or a reader, and the loaded model held twice what this PR leaves it. The peak was set by the categorical build's intermediates, mostly copies of the category lists:

  • category_slots held every (code, chain, rank) triple, then a counting-sorted copy, before merging each code's ranks by word into the exceptions. The exceptions table grew by doubling, so the loaded model kept up to half of it unused.
  • heavy_sets copied every split side's list as codes, then each distinct set's list again, so a list was held once for each of its sets: listed and complemented, with and without missing.
  • The decoders interned category lists in a map keyed by a copy of each list, one small allocation per list.
  • The loaded model kept every split's category list, though production tests categorical splits through the rank tables only.

Design

  1. load: write the rank tables' exceptions in place, counted exactly. A code's ranks arrive in word order, so a first pass over the chains counts each code's words, its exceptions, and a second writes and merges them in place, in a table of exactly that size. The cap is checked before the table grows, and the prefix sums are checked.
  2. load: free the categorical build's scratch sooner. The heavy sets, which live from the first pass to the second, are trimmed; split sides are recomputed rather than stored (16 bytes each); the first pass's distinct categories are deduplicated as they grow; the splits' words are allocated for the pass that writes them.
  3. parse: intern category lists by hash and pool offset. The intern maps a list's hash to its offset in the pool and compares the pool's list before sharing it; a different list with the same hash is stored again, unshared, as with interning disabled. The hash is the map's own RandomState, keyed per load as in 3.2.2, so a model cannot force collisions; a test loads lists that collided under the old fixed hash.
  4. load: free the category lists once loaded, in production. Production always precomputes masks by the sweep and runs passes without a scan, so the lists' readers (research's scans and brute-force masks, the tests' parsed-node walks) never run there. Research and the tests keep the lists; Forest's Debug shows their bytes, zero in production.
  5. load: hold each category list of a feature's heavy sets once. Split sides are sorted and deduplicated by their decoded lists, compared as their codes would compare (the codebook keeps the categories' order, and missing's code is below every category's), and each distinct list's codes are held once, after missing's; Sets takes a range per set rather than adjacent starts, so a list's sets share it. The cap, the covers' order and the sets are unchanged.
  6. load: look categories up by table where a feature's are dense. Where a feature's categories are at most eight times as many as the entries they replace, and 8 MiB, codes come from a table by category and the distinct categories from bits; otherwise the search and the sort as before. The eight is picked, not swept: on the private model every categorical feature's largest category is under three times its distinct categories, so any factor from three up takes the same path there.
  7. changelog, docs.

Not done, measured and dropped: counting every feature's exceptions before one allocation (a synthetic model with 200 to 1,000 categorical features loads as fast with the per-feature exact reservation, and the global count keeps every feature's heavy sets alive longer), freeing the category pool inside the build's first pass, skipping the pool's repacking in production (neither lowers the peak, which is in the first pass), and skipping the sort of category lists that already ascend (load time within noise: +1.1%, rounds −2.2% to +4.3%).

Evidence

On the private model, against 3.2.2, whose load memory and outputs equal 3.2.1's (dhat-rs requested bytes; malloc zone statistics; load times on an Apple M4 Pro, median of six rotated rounds of three loads, with the rounds' range):

3.2.2 this PR
Heap peak, Forest::load or from_reader 100% 45%
Heap peak, from_bytes with the file in memory 100% 61%
Loaded model's heap (one predictor) 100% 48%
malloc in use after loading 100% 38%
Load time, from_reader 100% 72% (rounds 70% to 75%)

Per commit: the share of 3.2.2's from_reader heap peak and loaded heap that each removes (exact), and its load time against the commit before it:

Commit Heap peak Loaded heap Load time
1. exceptions in place −30.2% −21.5% −3.9% (−9.5% to 0.0%)
2. scratch freed sooner −19.3% +7.7% (+0.8% to +10.5%)
3. intern by hash and offset −0.2% −12.6% (−14.0% to −8.5%)
4. lists freed in production −30.9% −0.5% (−3.4% to +3.6%)
5. heavy-set lists held once −5.7% +20.3% (+17.7% to +24.1%)
6. dense category tables −33.6% (−35.5% to −30.4%)

Commits 2 and 5 cost load time but take a quarter off 3.2.2's peak; commit 6 more than pays for them.

  • The layout is unchanged: layout_stats are equal (chains, chain slots, rank-table bytes, packing). Outputs are bitwise equal to 3.2.2's on synthetic rows (grouped and single) and on generated requests (each candidate, and the requests as served by predict_fixed). Request latency at 1 candidate is unchanged (p50 101% of 3.2.2's, rounds within ±3%).
  • cargo test --release --features research passes. cargo clippy with -D warnings for the workspace, with treewalker-bench/research, and for the library alone (-p treewalker-gbdt --lib, research off, which is the only build that compiles the production-only drop in commit 4); every commit passes the last on its own. cargo fmt --check.
  • What the tests can't see: the crate's tests always enable research (its self dev-dependency), so commit 4's production path is exercised only by a production build: a separate harness crate depending on this branch, which produced the outputs above.

category_slots collected every (code, chain, rank) triple of a feature, then
counting-sorted them into a second vector before merging each code's ranks by
word into the exceptions. A feature with many listed categories held both at
once: for a large model with many categorical splits, most of the load's
heap peak. The exceptions table also grew by doubling, so the
loaded model kept up to half of it unused.

A code's ranks arrive in word order, so a first pass over the chains now
counts each code's words, its exceptions, and a second writes and merges them
in place in a table of exactly that size. The cap is checked before the table
grows rather than after, the prefix sums are checked, and the tables are
unchanged.
Every categorical feature's heavy sets stay alive from the first pass of
bin_map to the second, so their codes and starts are now trimmed, once the
per-side lists they were taken from are freed, rather than kept at the
capacity that doubling left. The split sides are recomputed from the
feature's splits rather than stored, 16 bytes each. The first pass's
distinct categories are deduplicated as they grow rather than collected from
every split's list first, and that scratch is freed before the covers. The
splits' words are allocated for the second pass, which writes them, rather
than before the first.
The decoders interned category lists in a map keyed by a copy of each
distinct list, so a model's lists were held twice while it decoded, the
second time in one small allocation per list: a large part of the decoders'
heap for a model with many categorical splits. The intern now maps each
list's per-load hash to its offset in the pool and compares the pool's list
before sharing it. A different list with the same hash is stored again,
unshared, as with interning disabled.

The map's RandomState hashes the lists as well as indexing their hashes,
with keys drawn per load, so neither the fingerprints nor the lookup have
a fixed hash a model can collide under. Reusing the map's hasher needs no
second seed or hash implementation. Tests check the per-load fingerprints,
a collision's pool comparison, and loading lists that collided under the
old fixed hash.
A loaded model kept every categorical split's list, as many bytes as the
model's category entries, though production tests categorical splits through
the layout's rank tables only: it always precomputes masks by the sweep and
runs passes without a scan, so the lists' readers, research's scans and
brute-force masks, and the tests' parsed-node walks, never run there.
Production now frees the lists after the build; research and the tests keep
them; Forest's Debug shows their bytes, zero in production.
heavy_sets copied every split side's list into a vector of codes, then the
distinct sets' lists into another, so a feature with many categorical splits
held its lists twice while its sets were built, and kept a list once for each
of its sets: listed and complemented, with and without missing. That set the
load's heap peak for a large model with many categorical splits.

The sides are now sorted and deduplicated by their decoded lists, compared as
their codes would compare (the codebook keeps the categories' order, and
missing's code is below every category's), and each distinct list's codes are
held once, after missing's, so that its sets are ranges of them: Sets takes a
range for each set rather than adjacent starts. The cap, the covers' order and
the sets themselves are unchanged, so the layout is too.
Building a categorical feature's heavy sets looked every listed category's
code up by binary search in the feature's codebook, and the first pass found
the feature's distinct categories by sorting every split's list together:
most of the load's time for a model with many categorical splits.

Where a feature's categories are dense enough, at most eight times the
entries they replace and 8 MiB, the build now takes codes from a table by
category and collects the distinct categories in bits up to the largest;
otherwise it searches and sorts as before. Neither table outlives the
feature's first pass, and the layout is unchanged. The fixtures' small
categories take the tables, and a feature listing category u32::MAX the
search.
@victor-ab
victor-ab removed this pull request from stack #70 November 6, 2026 23:40
@victor-ab
victor-ab force-pushed the victor-ab/load-memory branch from 2496558 to 2f9df0a Compare November 6, 2026 23:40
@victor-ab
victor-ab added this pull request to stack #76 November 6, 2026 23:41
@victor-ab
victor-ab requested a review from ukaratay November 7, 2026 01:16

@ukaratay ukaratay left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The in-place exception write and the hash intern check out. A hash hit compares the pool's list before sharing it, so a collision costs sharing, not correctness. Freeing the lists in production is safe: their only readers are behind ABLATE (kernel.rs:88) or in the research scan.

One ask: which commits carry the 55% lower peak and the 26% faster from_reader? If any of them lands within noise, I'd drop it rather than carry its code path.

Comment thread src/load/layout/build.rs

/// A table by category, or the distinct categories' bits, is used only when it has
/// at most this many entries per list entry it replaces; else a search and a sort.
const DENSE_SPREAD: usize = 8;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was 8 measured or picked? This adds a second path for codes and for the distinct categories. If the dense table isn't a visible share of the load-time win, I'd keep only the search and the sort.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Picked, not swept. The table is most of the load-time win, though: commit 6 is −33.6% in every round (−35.5% to −30.4%), and without it this PR loads 8.7% slower than 3.2.2. On the private model, every categorical feature's largest category is under three times its distinct categories, so any factor from three up takes the same path there. Eight leaves room for sparser features, and the table stays under 8 MiB.

The unreleased section records what the load-memory changes save and how, and the loading docs say what a model with categorical splits holds while it loads and once loaded.
@victor-ab
victor-ab force-pushed the victor-ab/load-memory branch from 2f9df0a to 81e1b12 Compare November 7, 2026 18:29
@victor-ab

Copy link
Copy Markdown
Collaborator Author

Measured per commit on the private model: the heap figures are exact (dhat), and load time is from_reader on an M4 Pro, six rotated rounds, each commit against the one before, with the rounds' range.

Commit Heap peak Loaded heap Load time
1. exceptions in place −30.2% −21.5% −3.9% (−9.5% to 0.0%)
2. scratch freed sooner −19.3% +7.7% (+0.8% to +10.5%)
3. intern by hash and offset −0.2% −12.6% (−14.0% to −8.5%)
4. lists freed in production −30.9% −0.5% (−3.4% to +3.6%)
5. heavy-set lists held once −5.7% +20.3% (+17.7% to +24.1%)
6. dense category tables −33.6% (−35.5% to −30.4%)

The heap columns are shares of 3.2.2's peak and loaded heap. The 55% lower peak is commits 1, 2 and 5. The 26% faster load is commits 3 and 6, net of 2 and 5: those two cost load time but take a quarter off 3.2.2's peak.

Commit 7 (skipping the sort of lists that already ascend) was within noise: +1.1% (−2.2% to +4.3%), and −1.5% (−3.7% to +4.3%) applied to commit 5 alone. I've dropped it. Without it, this PR loads in 72% of 3.2.2's time (rounds 70% to 75%). The description has the table.

@victor-ab
victor-ab requested a review from ukaratay November 7, 2026 18:30
@victor-ab
victor-ab merged commit e50042f into main Nov 7, 2026
6 checks passed
@victor-ab
victor-ab deleted the victor-ab/load-memory branch November 15, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants