Skip to content

layout file: key on the schema version and target, with a typed mismatch - #79

Merged
victor-ab merged 6 commits into
mainfrom
victor-ab/layout-mismatch
Nov 11, 2026
Merged

victor-ab merged 6 commits into
mainfrom
victor-ab/layout-mismatch

Conversation

@victor-ab

@victor-ab victor-ab commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

A layout file loaded only in the build that wrote it, so models had to be converted again for every server build, and a file from another build failed like a corrupt one. This PR keys a layout file on its schema version and target, gives a mismatch its own error, and stops storing the mask plan that validation rebuilds anyway.

Loads at the head on an Apple M4 Pro (load_peak, medians of six interleaved rounds; max RSS from /usr/bin/time -l). The public models are those of experiments/layout-file.

binary text load_layout map_layout
public numeric model load time 18.2 ms 43.8 ms 4.6 ms 4.2 ms
heap peak / held 7.85 / 6.39 MiB 6.96 / 6.39 MiB 6.37 / 6.34 MiB 0.15 / 0.13 MiB
max RSS 10.5 MiB 9.8 MiB 8.4 MiB 8.5 MiB
public categorical model load time 262 ms 322 ms 108 ms 107 ms
heap peak / held 68.7 / 38.2 MiB 68.7 / 38.2 MiB 48.3 / 38.2 MiB 36.2 / 26.1 MiB
max RSS 112.9 MiB 112.6 MiB 44.0 MiB 45.1 MiB
a large private model with many categorical splits, as shares of its binary load load time 100% 161% 5.6% 4.9%
heap peak / held 100% / 100% 100% / 100% 31% / 100% 0.05% / 0.2%
max RSS 100% 100% 21% 21%

At the head, all four loads predict 20,160 generated rows bit-identically, alone and in full-width groups, and as the first commit does.

Design

One commit each:

  1. Typed error. A file of another schema version or target is LoadError::LayoutMismatch, with the file's header bytes escaped in its message.
  2. Mask plan built at load, not stored. Schema version 2, so 3.3.0's files are a mismatch. The categorical model's file goes from 37.7 to 13.2 MiB, load_layout's heap peak falls by 24.6 MiB, and it loads in 7% less time. The builder also returns its lists' spare capacity, so a source build of that model holds 38.2 MiB, from 46.9.
  3. Fixed 16 KiB packing threshold instead of the loading machine's L1. On an M4, models whose decision changes run groups of 64 up to 10% faster and single rows up to 9% slower; the commit message has the table.
  4. Key on schema version and target, not the build, guarded by committed .twl files for x86-64 and AArch64 and an arm64 CI job.
  5. convert example, which verifies a saved file before renaming it into place. load_peak also loads or maps a layout file.
  6. Docs and changelog.

Testing

  • Swapping two codec fields without a version bump fails the committed files' test, and the packing test fails with the probe on the M4.
  • Each commit passes the checks on macOS AArch64 and, under Rosetta, x86-64. The head also passes the layout-file tests on Linux AArch64.

@victor-ab
victor-ab requested a review from ukaratay November 10, 2026 01:36

@ukaratay ukaratay left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typed error is right. Before the recipe is written, I'd settle what a layout file is keyed on, and take three things out. Requesting changes.

Context: we will convert at training time and serve from one process with threads.

Settle first: stop comparing the build ID

  • The header check (src/layout_file/mod.rs:22) keys a file on the exact source hash, so conversion is per model and per server build. That is a deploy-time step, and it is why this PR needs the lockfile rule, regeneration on every upgrade, and a fallback during rolling deploys.
  • Compare the schema version and target fields only. The codec already specifies the format, and validation does not rely on the build ID for safety.
  • Guard it with committed .twl fixtures per architecture that must load and predict bit-identically, so a forgotten schema bump fails CI. docs/layout-file.md:162 lists this as open; I'd close it here.

Remove

  • The stored mask plan. Validation rebuilds it and checks equality (src/layout_file/validate.rs:269). On the public categorical model it is 24.6 MiB of the 37.7 MiB file, and the rebuild is 107 ms of the 122 ms load. Keep the rebuilt plan and stop storing it: the file drops to about 13 MiB and the peak by about 24.6 MiB, with load time unchanged. Bound the rebuild by the codes and exceptions in the file. Make it its own commit, before the build ID change, while a format change is still free.
  • The cache probe in the packing decision (src/rules/pack.rs:80). It is the only machine input that reaches the file. With a fixed threshold, conversions are reproducible; I checked that two runs, and text against binary, already give identical bytes. Measure predict time with packing forced each way first.
  • map_layout from the recipe. load_layout takes 4.7 ms against 4.2 ms on the numeric model and 124 ms against 122 ms on the categorical one, with the same or lower peak resident memory in one process (68 MiB against 94 MiB on the categorical model). Add a load_layout column to the table.

Wrong

  • The recipe renames the file into place (docs/layout-file.md:101) and then compares predictions (:102). Verify first, then rename.

Missing

  • Held heap and max RSS in the table. Source build against load_layout: 6.39 → 6.34 MiB held on the numeric model, 46.9 → 38.2 MiB on the categorical one.

Taste

  • A corrupt header reads as a mismatch. Fine, but the docs say a corrupt file is always Layout.
  • The mismatch message passes raw header bytes through (src/layout_file/mod.rs:55). Escape them.
  • The parity evidence says "with 3.3.0". Rerun it on this branch.
  • A convert example would replace the prose recipe and the save side effect in load_peak.

Drafted with an agent. The timings, sizes and byte-identity checks were run on this branch with the committed model script; the peak saving from dropping the plan is an estimate.

A layout file loads only in a build of its crate version and sources, for its
target, and the reader rejected any other with LoadError::Layout, as it rejects
a corrupt file. A server that ships layout files beside their source models
must tell the two apart: rebuild from the source after a treewalker upgrade or a
rollback, fail on a corrupt file. A header that starts as a layout file's but
differs from this build's is now LoadError::LayoutMismatch, whose message gives
the file's header fields and this build's, the file's bytes escaped but for
printable ASCII; a file that does not start as a layout file stays
LoadError::Layout. LoadError is non-exhaustive, so the variant is an addition;
only the variant of a mismatch changes.
A layout file stored the precompute's mask plan, and its loader built the plan
again from the validated bins and predicates to compare the two, because the
precompute reads the plan's ranges unchecked. The comparison made the stored copy
redundant: the file now stores no plan, and the loader keeps the one it builds,
as a source build does. Its flip lists are bounded by the file's own codes and
exceptions rather than by the stored plan's size: 4 bytes per code of a varying
categorical feature, and at most 8 per exception for each chain of the
exception's word, of which a word holds at most 32. The schema version goes to 2.

The builder returns the capacity its lists reserved as they grew, which the
forest would otherwise hold, as a source build's did.

On the public categorical model of experiments/layout-file, the layout file goes
from 37.7 to 13.2 MiB. For a process that only loads the model (Apple M4 Pro,
medians of 40 runs, the two builds alternating):

                          before     after
  load_layout  time       118 ms     110 ms
               heap peak  72.9 MiB   48.3 MiB
               heap held  38.2 MiB   38.2 MiB
               max RSS    67.9 MiB   44.1 MiB
  map_layout   time       117 ms     109 ms
               heap peak  60.8 MiB   36.2 MiB
               heap held  26.1 MiB   26.1 MiB
               max RSS    93.3 MiB   45.1 MiB
  source build heap held  46.9 MiB   38.2 MiB

The numeric model and a large private model with many categorical splits, whose
plans are a negligible part of their files, load as before. Predictions are
bit-identical before and after on all three, from a source build, load_layout
and map_layout, for single rows and full-width groups.
The packing rule packs a model's categorical rank fields when its unpacked bins
would leave L1, and for bins past 16 KiB, the share of a thread on a core of two
with 32 KiB, it asked the machine building the layout for its own L1. That was
the only input of a layout that came from the machine rather than from the
model, its configuration and the target, and a layout saved to a file serves on
other machines than the one that built it. The rule now packs every model whose
unpacked bins pass 16 KiB, so a model converts to the same layout anywhere. The
rule is the loader's, so a load from a source model follows it too.

Nothing changes on a machine with 16 KiB of L1 a thread. On one with more, a
model whose unpacked bins lie between 16 KiB and that share, and which the rule's
count of the fills that packing saves left unpacked, now packs. Measured on an
Apple M4 Pro, whose L1 holds 128 KiB, with packing forced each way on LightGBM
models trained to lie there: one-against-the-rest splits on six categorical
features of 200 to 5,000 categories drawn by a Zipf law, or of 200 to 8,000
drawn evenly, all of which the rule left unpacked. Packed time over unpacked,
medians of 20 alternating rounds, run twice:

  categories   trees   unpacked bins   groups of 64   single rows
  Zipf         1,200   17.3 KiB        0.978, 0.987   1.072, 1.069
  Zipf         3,200   19.5 KiB        0.965, 0.966   1.091, 1.061
  even           400   20.3 KiB        0.993, 0.991   1.085, 1.085
  even           800   33.9 KiB        0.955, 0.962   1.084, 1.083
  even         1,600   49.6 KiB        0.904, 0.902   1.065, 1.077
  even         3,200   64.5 KiB        0.905, 0.907   1.010, 1.019

So groups of such models run up to 10% faster and their single rows up to 9%
slower. The public models of experiments/layout-file and a large private model
with many categorical splits pack or not as they did.
A layout file loaded only in the build that wrote it: the reader compared the
whole header, with the writer's crate version and a hash of its sources, so a
file had to be converted for each server build and regenerated on every upgrade.
The codec specifies the format, and validation never relied on the build ID for
safety. The reader now compares the header's schema version and target fields
(architecture, pointer width, endianness and link stride) and nothing else. The
crate version and build ID stay in the header, for a mismatch's message to say
which build wrote a file. LoadError::LayoutMismatch is now a file of another
schema version or target; another build's file of the same ones loads.

What a build must then keep is the format under its schema version. Layout files
of six models, committed for x86-64 and AArch64 and written by an earlier build,
guard that: the production tests load them and compare their predictions with
their source models', so a change to the format, or to what the kernel makes of
it, that forgets the version fails. Between them the models hold both threshold
precisions, postprocessors and leaf plans, numeric and categorical splits on
constant, varying and increasing features, rank fields a word each and packed,
and a prefix group. `record_layouts` writes the files again after a bump. A CI
job runs the layout-file tests on AArch64, where no other run loads that
architecture's files.

A corrupt byte in the compared fields reads as a mismatch, because they are read
before anything is checked; one in the writer's fields fails the checksum, as
LoadError::Layout. The layout file guide says so, and no longer lists
compatibility across source changes as open.
The convert example is how a model becomes a layout file for serving. It loads
the model with its configuration, saves the layout to a new file, loads that file
with Forest::load_layout, compares its predictions with the forest's, bit for
bit, on generated rows alone and in groups, and only then renames the file into
place: loading validates that a layout is safe to run, not that it predicts as
its source model does.

load_peak, given a layout file, loads it with load_layout, or with --map maps
it, and prints the heap peak, held bytes and seconds it prints for a source
model, so that the three loads can be compared.
The layout file guide gains a section on converting models for serving: once for
each target, with the convert example, which checks a saved file's predictions
before it takes its place; loaded with load_layout, beside the source model to
rebuild from on a mismatch; and when mapping pays. The changelog's Unreleased
section gives the layout file's new key, error and schema version, with the
public categorical model's file and load figures; the packing rule's fixed
threshold; and the mask plan's capacity.
@victor-ab
victor-ab force-pushed the victor-ab/layout-mismatch branch from f51bc33 to 397bf59 Compare November 11, 2026 16:54
@victor-ab victor-ab changed the title layout file: a typed error for a layout from another build, and converting models for serving layout file: key on the schema version and target, with a typed mismatch Nov 11, 2026
@victor-ab

Copy link
Copy Markdown
Collaborator Author

Reworked as you asked; the description has the new table and the parity rerun.

  • Build ID: a3bed47, with the committed .twl files, the arm64 CI job and the docs on a corrupt header
  • Mask plan: c8f7150
  • Cache probe: b9e0a80, measured first; the table is in the commit message
  • Recipe and convert: 7884891, 397bf59
  • Escaped header bytes: c67c2aa

@victor-ab
victor-ab requested a review from ukaratay November 11, 2026 16:55
@victor-ab
victor-ab merged commit e0d055e into main Nov 11, 2026
3 checks passed
@victor-ab
victor-ab deleted the victor-ab/layout-mismatch branch November 11, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants