Repository navigation
fix: handle aliased operands in the in-place operators - #76
Merged
Merged
Conversation
`x *= x` and `x /= x` compute derivatives from data they have already
overwritten, because b and *this are the same storage:
DDScalar::operator*= the Hessian loop reads b.m_data[1 + j] after the
gradient loop has replaced it
DDScalar::operator/= binary() writes into m_data, which is also its b
SScalar::operator*= the second loop reads b.m_d after the first has
scaled it
SScalar::operator/= same
Measured on x = 3 with dx = 1:
result expected
DD x *= x h00 = 12 h00 = 2 wrong
DD x /= x 0 0 accidentally right
S x *= x dx = 12 dx = 6 wrong
S x /= x dx = 0.222 dx = 0 wrong
Only the second-order part of DDScalar::operator*= is affected — the
gradient loop reads b.m_data[i] in the same expression that writes it, so
it still sees the original value.
DDScalar::operator/= is not observably broken. When both operands alias, the
gradient loop zeroes every slot before the Hessian loop reads them, and the
correct Hessian of x / x is zero as well, so the stale data happens to give
the right answer. It is guarded anyway: that only holds because aliasing
forces the trivial result, and a future change to the loop order would break
it silently.
`+=` and `-=` are correct in both classes. They read and write the same
element in one expression, which yields 2a and 0 as it should.
All four now delegate to the binary operator, which builds a fresh result and
therefore cannot read overwritten data. The extra copy only happens for the
aliased call. Cost of the pointer comparison on the hot path, 2M iterations of
`a *= b; a /= b`, -O3 -DNDEBUG, three alternating runs, static order 2 size 8:
177.3/177.0/179.5 ns/op with the guard against 178.5/175.8/174.2 without.
Tests use the binary operator as the oracle — the invariant is that in-place
and binary agree — plus analytic anchors: x^2 at x = 3 with g0 = 1 has
h00 = 2, and x / x is 1 with all derivatives zero.
This was referenced Jul 26, 2026
oberbichler
added a commit
that referenced
this pull request
Jul 27, 2026
A multiply of two single-entry values cost six allocations, eval copied the
name on every iteration, and binary and ternary copied the whole map. The
storage becomes a vector of name and derivative sorted by name, so combining
two values is a linear merge rather than a sequence of hash lookups and node
allocations. Measured on an expression chain, -O3:
names before after
2 416 ns, 12 allocs 86 ns, 2 allocs 4.8x
4 2029 ns, 53 allocs 280 ns, 6 allocs 7.2x
12 14271 ns, 375 allocs 1546 ns, 22 allocs 9.2x
This replaces the design proposed for this step. Interning the names behind a
shared table was going to be the approach, and the allocation accounting did
not survive scrutiny: with different name sets the merge and the remap of both
operands cost no less than the map, and in (x*y)/(x-y) the tables are equal in
content but not in identity, so the fast path would never have fired. Both
designs were prototyped and measured before choosing.
The sorted vector is not a detour for second order either. The position of a
name in the vector is its index, which is what a dense triangular Hessian would
be laid out over, and the merge already produces the union in sorted order.
Data stays the map. It is the interface type for construction and eval, which
is what callers and in particular a Python dict naturally provide; the vector
is internal. A second constructor taking the internal type would have made
SScalar(f, {{"x", 1.0}}) ambiguous between the two, so the operations go through
a named factory instead.
Three simplifications fall out. The four in-place operators delegate to their
binary form, which puts the merge in one place and makes aliased operands
correct without the guards added in #76. The derivatives are now iterated in a
defined order, so operator<< and repr are deterministic instead of unspecified
-- the tests and the README are tightened accordingly. And clang-tidy drops
from 50 findings to 41, because the braceless single statements it complained
about were in the code that went away.
Also replaces the int coefficients in operator+ and operator- with Scalar,
which matters for a float scalar type.
The public API is unchanged: 35 bound methods as before, SScalar(f=, d={...}),
size, d and eval identical. That the characterization from #84 carries the
rewrite without a single expected value being touched is the evidence that
doing it first was right.
C++ 112 test cases, 1421 assertions, Debug and Release with -Werror and
under clang with ASan+UBSan
Python 248 passed
clang-format and ruff format clean
oberbichler
added a commit
that referenced
this pull request
Jul 27, 2026
…ap (#85) A multiply of two single-entry values cost six allocations, eval copied the name on every iteration, and binary and ternary copied the whole map. The storage becomes a vector of name and derivative sorted by name, so combining two values is a linear merge rather than a sequence of hash lookups and node allocations. Measured on an expression chain, -O3: names before after 2 416 ns, 12 allocs 86 ns, 2 allocs 4.8x 4 2029 ns, 53 allocs 280 ns, 6 allocs 7.2x 12 14271 ns, 375 allocs 1546 ns, 22 allocs 9.2x This replaces the design proposed for this step. Interning the names behind a shared table was going to be the approach, and the allocation accounting did not survive scrutiny: with different name sets the merge and the remap of both operands cost no less than the map, and in (x*y)/(x-y) the tables are equal in content but not in identity, so the fast path would never have fired. Both designs were prototyped and measured before choosing. The sorted vector is not a detour for second order either. The position of a name in the vector is its index, which is what a dense triangular Hessian would be laid out over, and the merge already produces the union in sorted order. Data stays the map. It is the interface type for construction and eval, which is what callers and in particular a Python dict naturally provide; the vector is internal. A second constructor taking the internal type would have made SScalar(f, {{"x", 1.0}}) ambiguous between the two, so the operations go through a named factory instead. Three simplifications fall out. The four in-place operators delegate to their binary form, which puts the merge in one place and makes aliased operands correct without the guards added in #76. The derivatives are now iterated in a defined order, so operator<< and repr are deterministic instead of unspecified -- the tests and the README are tightened accordingly. And clang-tidy drops from 50 findings to 41, because the braceless single statements it complained about were in the code that went away. Also replaces the int coefficients in operator+ and operator- with Scalar, which matters for a float scalar type. The public API is unchanged: 35 bound methods as before, SScalar(f=, d={...}), size, d and eval identical. That the characterization from #84 carries the rewrite without a single expected value being touched is the evidence that doing it first was right. C++ 112 test cases, 1421 assertions, Debug and Release with -Werror and under clang with ASan+UBSan Python 248 passed clang-format and ruff format clean
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
x *= xandx /= xcompute derivatives from data they have already overwritten, becauseband*thisare the same storage:DDScalar::operator*=b.m_data[1 + j]after the gradient loop has replaced itDDScalar::operator/=binary()writes intom_data, which is also itsbSScalar::operator*=b.m_dafter the first has scaled itSScalar::operator/=Measured on
x = 3withdx = 1:DDScalarx *= xh00 = 12h00 = 2DDScalarx /= x00SScalarx *= xdx = 12dx = 6SScalarx /= xdx = 0.222dx = 0Two corrections to what I claimed when filing this:
DDScalar::operator*=is affected. The gradient loop readsb.m_data[i]in the same expression that writes it, so it still sees the original value and produces the correct2·f·g. That is why the first-order types come out right.DDScalar::operator/=is not observably broken. When both operands alias, the gradient loop zeroes every slot before the Hessian loop reads them — and the correct Hessian ofx / xis zero as well, so the stale data happens to give the right answer. It is guarded anyway: that only holds because aliasing forces the trivial result, and a change to the loop order would break it silently.+=and-=are correct in both classes and are left alone. They read and write the same element within one expression, which yields2aand0as it should.Fix
All four delegate to the binary operator, which builds a fresh result and therefore cannot read overwritten data:
The extra copy only happens for the aliased call. Cost of the pointer comparison on the hot path — 2M iterations of
a *= b; a /= b,-O3 -DNDEBUG, static order 2 size 8, three alternating runs:Noise. Dynamic size 8 and 32 likewise (152.4 vs 152.2, 754.0 vs 758.3), with identical accumulated results.
Tests
Written first. The oracle is the binary operator — the invariant being that in-place and binary agree — plus analytic anchors so the test cannot pass by both paths breaking the same way:
x²atx = 3withg0 = 1hasf = 9,g0 = 6,h00 = 2, andx / xis1with every derivative zero.C++ —
Self multiplication,Self division,SScalar Self multiplication,SScalar Self division. RED:Python —
test_self_multiplicationandtest_self_divisionover all four{order 1, 2} × {static, dynamic}contexts, plus the twoSScalarcases. RED failed on exactly the four combinations predicted above, which is what confirmed the two corrections:test_self_multiplicationfailed for 2nd order only, andtest_self_divisionpassed for all fourDDScalarvariants.After: C++ 46/46 test cases, 456 assertions · Python 176 passed ·
clang-format --Werrorandruff format --checkclean.Notes
Based on
mainwith #73 and #74 merged. Independent of #75 (still open) — and once #75 lands, this code is covered by the sanitizer job as well.Last item from the review's P0 list:
resize()silently scrambles the Hessian layout for order 2 (pad_right()does it correctly). fix: validate indices and sizes instead of relying on asserts #73 only stopped it from accepting a negative size.