Skip to content

Specify EntityLocation alignment to improve codegen - #26042

Merged
alice-i-cecile merged 2 commits into
bevyengine:mainfrom
dylansechet:fix_unsafe_world_cell_regression
Oct 7, 2026
Merged

alice-i-cecile merged 2 commits into
bevyengine:mainfrom
dylansechet:fix_unsafe_world_cell_regression

Conversation

@dylansechet

@dylansechet dylansechet commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Objective

Fixes #25839.

Solution

#25632 changed the memory layout of UnsafeEntityCell, which caused some codegen downstream to hit a store-to-load forwarding failure. (I hadn't heard about those before working on this PR, this blogpost has a nice explainer. TLDR is that it's slow).

Adding a repr(align(8)) to EntityLocation seems to tip LLVM codegen the right way and we get to avoid that store-to-load forwarding failure, which lets us recover the pre-#25632 performance.

The PR also speeds up World::entity, which already had a similar forwarding failure before #25632.

I want all the details!

EntityLocation gets loaded from the entity table into three registers: archetype_row (4 bytes), table_row+archetype_id (8 bytes), and table_id (4 bytes). Since #25632, it then gets written as 8+4+4 into UnsafeEntityCell: archetype_row+table_row, then archetype_id, then table_id. That first 8-byte chunk needs archetype_row plus the half of the second register, and LLVM builds it by writing both registers to the stack and reading 8 bytes back. That read spans two pending stores and causes the store-to-load forwarding to fail. This path gets hit in most places where we build an UnsafeEntityCell from a location, and so ends up affecting multiple benchmarks.

Putting a repr(align(8)) onto EntityLocation makes LLVM load it as archetype_row, table_row, and archetype_id+table_id (4+4+8), and it can then join archetype_row and table_row directly in a register without the round-trip to the stack.

Testing

Ran the benchmarks mentioned in #25839.

We get a bunch of nice improvements, with observe/observer_custom/10000_entity being the only one that's still worse than before #25632. Not sure what's going on there, and will probably leave it as future work.

benchmark Before #25632 #25632 #25632 + align(8)
world_entity/50000_entities 259.52 µs (x1.00) 233.33 µs (x0.90) 66.48 µs (x0.26)
event_propagation/four_event_types 296.68 µs (x1.00) 350.13 µs (x1.18) 310.61 µs (x1.05)
observe/observer_custom/10000_entity 461.24 µs (x1.00) 513.55 µs (x1.11) 510.81 µs (x1.11)
ecs::resources::insert_remove 75.7 ns (x1.00) 76.7 ns (x1.01) 76.9 ns (x1.02)
despawn_world_recursive/1_entities 297.8 ns (x1.00) 315.6 ns (x1.06) 305.4 ns (x1.03)
despawn_world_recursive/100_entities 12.72 µs (x1.00) 12.90 µs (x1.01) 12.49 µs (x0.98)
despawn_world_recursive/10000_entities 1.23 ms (x1.00) 1.25 ms (x1.02) 1.21 ms (x0.99)
query_get_many_2/50000_calls_table 410.63 µs (x1.00) 412.17 µs (x1.00) 401.98 µs (x0.98)
query_get_many_2/50000_calls_sparse 307.48 µs (x1.00) 308.31 µs (x1.00) 291.51 µs (x0.95)
query_get_many_5/50000_calls_table 960.35 µs (x1.00) 957.31 µs (x1.00) 974.13 µs (x1.01)
query_get_many_5/50000_calls_sparse 715.22 µs (x1.00) 714.31 µs (x1.00) 679.33 µs (x0.95)
query_get_many_10/50000_calls_table 1.75 ms (x1.00) 1.75 ms (x1.00) 1.76 ms (x1.01)
query_get_many_10/50000_calls_sparse 1.63 ms (x1.00) 1.62 ms (x1.00) 1.59 ms (x0.98)
get_entity_mut_slice/size/20 102.2 ns (x1.00) 152.1 ns (x1.49) 100.9 ns (x0.99)
get_entity_mut_slice/size/200 1.96 µs (x1.00) 2.43 µs (x1.24) 1.90 µs (x0.97)
get_entity_mut_slice/size/2000 18.95 µs (x1.00) 24.44 µs (x1.29) 18.95 µs (x1.00)

I started by pointing a LLM at the issue and asked it to inspect the assembly before and after a716a99 to try and find a cause. It directed me towards the store-to-load issue. I'd never heard of that so I did a bunch of reading to understand what was going on, then spent way too much time staring at the assembly, and finally ran the benchmarks.

@alice-i-cecile alice-i-cecile added A-ECS Entities, components, systems, and events C-Performance A change motivated by improving speed, memory usage or compile times D-Straightforward Simple bug fixes and API improvements, docs, test and examples S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Oct 6, 2026
@alice-i-cecile alice-i-cecile added this to the 0.20 milestone Oct 6, 2026

@hymm hymm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it was a fun hole to jump down.

@alice-i-cecile alice-i-cecile added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Oct 6, 2026
@alice-i-cecile
alice-i-cecile added this pull request to the merge queue Oct 7, 2026
Merged via the queue into bevyengine:main with commit a144d28 Oct 7, 2026
73 of 75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ECS Entities, components, systems, and events C-Performance A change motivated by improving speed, memory usage or compile times D-Straightforward Simple bug fixes and API improvements, docs, test and examples S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Pref Regressions for "Use NonNull in UnsafeWorldCell (#25632)"

3 participants