Repository navigation
fix: restrict Hessian API to second-order DDScalar - #72
Merged
Merged
Conversation
The Hessian accessors index into m_data behind the gradient block. On a
first-order scalar that storage does not exist, so every one of them read
or wrote past the end of the object.
The C++ header exposed h/set_h/hm/set_hm unconditionally, and common.h
bound them for both orders — only from_gradient/from_arrays were guarded
by order. Reachable from pure Python:
v = hj.D3Scalar.variable(0, 1.0) # data length 4
v.hm() # reads 6 doubles past the array
hj.DScalar.variable(0, 1.0, 3).set_hm(np.full((3, 3), 7.0))
# writes 6 doubles past the heap buffer
The asserts in h() are compiled out in release wheels, so nothing caught
it. It also surfaced through the documented API: hj.dd() probes
hasattr(v, 'hm') and therefore returned foreign memory for first-order
values instead of an empty Hessian.
Constrain the accessors with `requires (order() == 2)` in the header and
bind them under `if constexpr (T::order() == 2)`. The constraint on the
header is what makes the guarantee — the binding can no longer drift away
from it, and misuse in C++ is now a compile error rather than silent UB.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Hessian accessors index into
m_databehind the gradient block:A first-order scalar has no such storage — its data length is
1 + size. The header exposedh/set_h/hm/set_hmunconditionally, andcommon.hbound them for both orders (onlyfrom_gradient/from_arrayswere guarded by order). So every Hessian access on a first-order type read or wrote past the end of the object, reachable from pure Python:The
asserts inh()are compiled out in release wheels, so nothing caught it.It also surfaced through the documented API:
hj.dd()probeshasattr(v, 'hm')(main.cpp:69) and therefore returned foreign memory for first-order values instead of an empty Hessian.Fix
requires (order() == 2)onh(i),h(i, j),set_h(i, v),set_h(i, j, v),hm(mode),hm(mode, out),set_hm(value).defs moved underif constexpr (T::order() == 2).Putting the constraint on the header is what makes the guarantee: the binding can no longer drift away from it, and misuse in C++ becomes a compile error instead of silent UB. It also replaces one more
static_assert-style guard with a real constraint (cf.TODO.md→ Concepts).Tests
Both were written first and observed failing.
C++ (
test/src/test.cpp) — concepts probe the constraint directly at the header level, independent of the bindings. Before the fix all 8 first-order checks reportedtrue:Python (
python/tests/test_DDScalar.py) —test_hessian_accessassertsAttributeErrorfor first-order types (matching the existingtest_pad_rightconvention) and keeps asserting real values for second order;test_dd_of_first_orderpinshj.dd()tonp.empty((n, 0, 0)), consistent withtest_dd_of_scalar. Thehj.ddfailure showed the leak plainly:After: C++ 40/40 test cases, 400 assertions · Python 147 passed ·
clang-format --Werrorandruff format --checkclean.Second-order behaviour is unchanged — the existing
test_values/test_ndarray/test_ddassertions onh,set_h,hm,set_hmall still pass untouched.Note
Found while reviewing the whole project. Related follow-ups, in rough order of severity, that this PR deliberately does not touch:
h(i, j)/set_h(i, j, v)are still unbounded in release builds —v.h(-5, -5)reads arbitrary memory from Python. The binding layer needs a real range check;assertis the wrong mechanism at a language boundary.DDScalar<1, double, Dynamic>{1.0, 2.0, 3.0}segfaults (m_datais an empty vector whenstd::copywrites into it).x *= xandx /= xproduce wrong derivatives (self-aliasing) in bothDDScalarandSScalar.resize()silently scrambles the Hessian layout for order 2;pad_right()does it correctly.