Voxel planet: chunk octree, ray-guided residency, per-voxel materials, material contract

[Tuafo]

Nine changes to the voxel planet pass, plus a merge of main (crates/passes/3d/helio-pass-voxel-planet). The design and every measurement are in docs/chunk-octree.md.

Pulsar pins this branch in Far-Beyond-Pulsar/Pulsar-Native#1395 (draft). Merge this PR first, then re-pin Pulsar to Helio main.

3D chunk octree with ray-guided residency (f866cbb6)

• Octree replaces the tile quadtree. Nodes are 64³ chunks (face, level, cx, cy, cz). Classification accounts for what a node's descendants hold (caves, brushes at any size, baked bricks), so a coarse node never hides finer detail.
• Residency is a cut of the octree. Rays report the node their pixel wants (one pixel per 8x8 block per frame). Buried nodes draw as rock until a ray or the eye reaches them. LRU eviction runs when slots or leaf pages run short.
• Footprint pool. Footprint columns stay on the GPU across frames in 2048 slots. A chunk now costs 19–22 µs to generate, down from 27–30.
• The old tile, layer, gap and deep-tile paths are deleted.

Docs: corrupt captures are an NVIDIA driver fault (955e7cf5)

The occasional captures with 8x8 haze blocks are not an engine race. Only NVIDIA's Vulkan driver shows them (616.64, RTX 3060): D3D12 never does, and synchronization validation finds no hazard. Reported to NVIDIA as bug 6939141. The docs list what was tried and what changes it.

Per-voxel materials for edited cells, and a brush index (cdaf7714)

• Per-voxel materials. Cells whose material edits set (add, paint, baked cells) keep their own material in 4 KiB detail pages. A leaf page gets one only when it holds such a cell. Other leaves keep their surface and subsurface pair.
• Drawing reads the cell's own byte first. A per-slot flag skips the lookup in chunks with no detail pages.
• PickHit has a new material field. Pulsar's PickHit constructor in voxel_backend.rs needs material: 0 when its Helio submodule is bumped.
• BrushIndex (a loose grid per face) replaces the scan of every brush per chunk in queue_footprint.

Measurements

	Before	After
mountain_slope hold, primary rays (mean of 4 rounds)	2.861 ms	2.861 ms
Cruise flight: chunk columns, per chunk	5.6 µs	5.5 µs
Cruise flight: chunk builds, per chunk	2.8 µs	2.9 µs
Cruise flight: draw p50	3.79 ms	3.80 ms

Without the per-slot flag, the detail lookup cost +0.9% on primary rays.

Generation materials match Planet::material (e5398492)

The new report test natural_cells_against_the_leaf_rule (ignored by default) compared drawn materials with Planet::material on natural mountain ground and in dug pit walls. Before this commit, 40.5% of pixels differed (0% on flat grassland), mostly from a CPU/GPU mismatch.

• The mismatch. The generation kernel evaluated ground_material at the column's domain point; Planet::material uses the cell's volume point. The snowline's noise decorrelates 2 km up, so whole views flipped (snow drawn as stone). The slope estimates also differed.
• Point. Each evaluation now uses its cell's volume point and the surface byte.
• Slope. Levels 0–3 use Planet::material_slope. Each computed footprint evaluates a lattice of level-4 heights, through three dispatches, and keeps the block slopes in the footprint pool.
• New gate, uniform_leaves_draw_canonical_materials. On leaves whose 16 columns all top out in them with one material, the top faces draw the canonical material exactly: 31,710 pixels, none wrong (779 wrong before).
• What's left is the leaf rule itself: 9.35% of pixels. That's 8.8% for cells under air in mixed leaves, and 33.5% for cells under ground, mostly in dug walls.
• Cost: generation goes from about 20 µs to 22.4–22.8 µs a chunk (+10%), all of it from the lattices at levels 1–3. Drawing is unchanged.

Dug walls draw their cells' own materials (c0f1779c)

• The problem: a leaf's subsurface byte follows its centre's depth, so about a third of a pit's wall pixels drew rock where the cells are soil a cell deep.
• The fix:
  • Every chunk under an edit now gets a per-cell scratch slot.
  • Each exposed solid cell gets its edit's material, or else its natural Planet::material.
  • Cells the leaf's pair already draws correctly are dropped, so only leaves that need it get detail pages.
• New gate, dug_walls_draw_canonical_materials: all 57,050 wall and floor pixels of pits at three varied spots draw Planet::material, at about 170 detail pages per pit.
• The natural-cells report: 3.56% of pixels wrong (9.35% before), none under ground. What's left is the surface vote in unedited mixed leaves.

GPU test worlds sized to fit VRAM (4f11cf98)

• The problem: each GPU test built a game-sized world, about 2.9 GB of VRAM with its device. Eight parallel tests overflowed a 12 GB GPU, and the suite crawled or stalled.
• The fix: ChunkWorld::with_capacity takes a Capacity (chunk slots, detail pages, footprint pool slots). new keeps the game default and its environment variables. Tests use 16,384 chunk slots, about 1.1 GB each.

Large brushes into data: edits on data, step 1 (acfcf426)

At 10,000 large digs 8 m across, one more edit cost 3.8 ms of CPU sync and 4.35 ms of GPU generation:

• every edit re-hashed, re-indexed and re-uploaded every large brush;
• every coarse chunk under them evaluated all of them.

The fixes:

• Large brushes are baked into the edit store at their bake level, the first level where they span at most 32 cells. Coarser levels follow by majority. Below that level they stay analytic, and at it every cell reads as the analytic brush did.
• Snapshots are now version 3. Older snapshots bake their large brushes on load.
• The brush index is split by the levels a brush applies at, so coarse chunks skip baked brushes.
• Edit sync is incremental. It reuses the large-brush prefix, diffs only the changed tail, and writes the buffer tail in place.
• A GPU brick cache (64 MB). Regenerated chunks no longer re-upload the same bricks.

At 10,000 large digs, one more edit now costs 0.51 ms of sync, 0.48 ms of update CPU and 0.37 ms of GPU generation, against 0.34, 0.30 and 0.20 ms at 100.

Edited chunks kept across eviction: edits on data, step 2 (725f4c88)

• Packing. When the plan evicts an edited chunk, the GPU packs its 64-tree (about 12 KB on average), reads it back, and the CPU keeps it.
• Restoring. When that chunk is wanted again, it's restored by copy: no footprint, no terrain program and no brushes.
• Empty chunks. An edited chunk that comes out empty is remembered as empty. Before this, the plan kept re-requesting dug chunks it had just found empty, and each one was regenerated from the whole history.
• Invalidation. An edit over a kept chunk drops the kept copy.

New gate, edited_chunks_come_back_by_copy: a site under 2,000 large digs, left for a far view and then seen again.

On return	Edited chunks restored / generated	Generation GPU	Pixels as before
Keeping off	0 / 534	308 ms	all
Keeping on	1041 / 0	58 ms	all

An edit over restored chunks draws exactly, and it comes back with them after another round trip.

Kept chunks in saves (41bdb45e, 1dcc418f)

• Saving. keep_resident() also packs the resident edited chunks. kept_chunks() then takes every kept chunk; it shares the packs, so it's cheap on the render thread. KeptChunks::to_bytes() deflates them (about 1 KB per chunk) on the saving thread.
• Not in the snapshot. The chunks are saved next to the edit snapshot, not inside it. A host folds only its oldest edits into a snapshot (Pulsar keeps its newest 4,096 listed), while the chunks reflect every edit. The version-4 snapshot section from 41bdb45e is removed; snapshots stay at version 3.
• Journal. VoxelEditJournal stores a renderer's chunks for its edits as VoxelKeptChunks: the bytes, plus the journal length and hash they were made for. They're saved as kept, and any change to the edits drops them.
• Loading. KeptChunks::from_bytes() reads the chunks back and load_kept() refills a new world's cache. It checks a fingerprint of the generation code, terrain and world, and refuses chunks saved for a different edit history.
• Through the pass. PlanetFrame.kept goes into any world built for that planet; ChunkPass::kept_loaded reports how many loaded, or why they were refused. PlanetFrame.export (KeptExport) is a save's request: the pass packs the resident edited chunks over the next frames, asking for frames meanwhile, then answers with the planet shown and its kept chunks.

Gate a_loaded_world_restores_its_edited_chunks: the dug site is saved (313 chunks, 316 KiB on top of the snapshot's 260) and loaded into a new world:

• edited chunks: 1,172 restored and none generated, against 313 generated without the kept chunks;
• generation GPU: 90 ms against 234 ms;
• every pixel matches the saved world;
• damaged bytes and other histories are refused.

New saved_edited_chunks_come_back_through_the_frame (voxel_pass_graph): 111 chunks exported in 6 frames. Once loaded, 84 edited chunks are restored and none generated. Far-Beyond-Pulsar/Pulsar-Native#1395 wires this into the editor's save and load.

Merge of main (b451b343)

main's voxel debug views were written against the old tile-quadtree chunks/mod.rs, which this branch replaced. They're ported onto the octree files:

• the renderer's debug mode forwarded to the voxel shaders, with local diagnostics in bits 0x200–0x800;
• the trace-cost view (mode 32). The cost sits in the visibility word's high bits and the material in five bits, so PickHit.material now reads five;
• the UV reference grid (mode 1). Its texture and sampler use group 1 bindings 25 and 26, because 24 is the ray-feedback buffer.

Tests

• cargo test --release -p helio-pass-voxel-planet at default threads: 75 unit tests and 24 GPU tests pass.
• helio-default-graphs: voxel_pass_graph (9 tests) passes. environment_join and scene_join's camera-relative tests fail exactly as they do on main; this branch changes nothing outside the voxel crate and its voxel_flight example.
• New: edited_cells_draw_their_own_materials. All 16,306 edited-cell pixels match; with the lookup disabled, 14,610 of them are wrong.
• New: the_brush_index_finds_what_a_scan_finds, kept_chunks_stay_with_exactly_their_edits (helio-voxel-data, 23 tests pass).
• New: uniform_leaves_draw_canonical_materials, dug_walls_draw_canonical_materials, edited_chunks_come_back_by_copy, a_loaded_world_restores_its_edited_chunks and sealed_large_brushes_read_the_same_at_their_bake_level (unit).

🤖 Generated with Claude Code