Skip to content

fix: make the array overload of variables() deducible - #78

Merged
oberbichler merged 1 commit into
mainfrom
fix/variables-array-deduction
Jul 26, 2026
Merged

oberbichler merged 1 commit into
mainfrom
fix/variables-array-deduction

Conversation

@oberbichler

Copy link
Copy Markdown
Owner

Problem

template <index T>                                  // index = std::ptrdiff_t
static std::conditional_t<...> variables(const std::array<Scalar, T> &values)

std::array is sized by std::size_t, so deducing a ptrdiff_t non-type parameter from the argument always fails:

error: no matching function for call to 'variables'
note: candidate function not viable: no known conversion from 'std::array<double, 2>'
      to 'const std::vector<Scalar>' for 1st argument
note: candidate template ignored: substitution failure: deduced non-type template
      argument does not have the same type as the corresponding template parameter
      ('unsigned long' vs 'index' (aka 'long'))

The C++ usage example in the README did not compile. The overload was only ever reachable from the bindings, which passed the size explicitly and therefore never exercised deduction:

T::template variables<T::static_size()>(values)

Fix

Declare T as std::size_t. Three small consequences:

  • The explicit workaround in common.h goes away — T::variables(values) now deduces.
  • static_assert(index(T) == TSize) compares as index, so both sides keep the same signedness (relevant now that -Wall -Wextra is on since ci: add a sanitizer job and turn on compiler warnings #75).
  • The local array is sized by T instead of TSize, which the assert has just established to be equal, and avoids a signed→unsigned conversion in a template argument.

Verification

The README snippet now compiles verbatim. Rather than retyping it, I extracted the ```cpp block straight out of README.md and built it with -Wall -Wextra -Wpedantic -Werror:

f   = -6
df/dx = -4
df/dy = 1
d²f/dx² = -2.66667

which is exactly what its inline comments claim.

  • C++ 45/45 test cases, 449 assertions — Debug, Release, and the clang ASan+UBSan job, all with -Werror
  • Python 166 passed; D3Scalar.variables([1,2,3]) and DDScalar.variables([1,2]) verified to still produce the right gradients through the simplified binding
  • clang-format --Werror and ruff format --check clean

Tests

TEST_CASE("README example") pins the example's values, so it cannot silently stop compiling or start lying again. Its Hessian is the same computation the Python quickstart shows in the README ([[-2.667, 1.333], [1.333, -0.667]]), so those numbers are covered by the same test.

TEST_CASE("Variables from an array") exercises both branches of the conditional return type — std::array<Type, T> for static and std::vector<Type> for dynamic. Neither had any C++ coverage before, which is how the deduction failure survived: the only caller passed the size explicitly.

Notes

A durable version of the README check would extract and compile the snippets in CI rather than duplicating them in a test. Worth doing once there are more of them; the pinned test case covers this one.

Based on main with #75 merged. Independent of #76 and #77 (both still open).

The non-type parameter was declared `index` (ptrdiff_t) while std::array is
sized by std::size_t, so deduction from the argument always failed:

    error: no matching function for call to 'variables'
    note: candidate template ignored: substitution failure: deduced non-type
          template argument does not have the same type as the corresponding
          template parameter ('unsigned long' vs 'index' (aka 'long'))

Which means the C++ usage example in the README did not compile. It only
worked from the bindings, because those passed the size explicitly:

    T::template variables<T::static_size()>(values)

Declaring T as std::size_t makes it deducible, and the explicit workaround in
common.h goes away with it. The static_assert compares as index so the two
sides stay the same signedness, and the local array is now sized by T rather
than TSize, which the assert has just established to be equal.

The README example now compiles verbatim — extracted straight from the file
with -Wall -Wextra -Wpedantic -Werror — and prints what its comments claim:

    f   = -6
    df/dx = -4
    df/dy = 1
    d²f/dx² = -2.66667

Tests: the example becomes a test case with its values pinned, so it cannot
silently stop compiling or start lying again. Its Hessian is the same
computation the Python quickstart shows in the README, so those numbers are
covered too. A second case exercises both branches of the conditional return
type, static and dynamic, neither of which had any C++ coverage.
@oberbichler
oberbichler merged commit b14be3c into main Jul 26, 2026
17 checks passed
@oberbichler
oberbichler deleted the fix/variables-array-deduction branch July 26, 2026 20:59
oberbichler added a commit that referenced this pull request Jul 26, 2026
The C++ suite ran almost entirely on DDScalar<2, double, 3>. Templates that
are never instantiated are never compiled, so neither the tests, nor the
warnings, nor the sanitizer job from #75 could see the first-order or dynamic
code paths at all. That is how the deduction failure in #78 survived, and why
turning on -Wall -Wextra produced zero warnings over a header that had five.

Adds a second layer in its own translation unit: 14 TEST_CASE_TEMPLATEs over
{order 1, 2} x {static, dynamic}, covering accessors and the linear Hessian
index, the factories, all arithmetic in both binary and in-place form with
both operand orders, one function per arity (unary, binary, ternary), eval,
and padding. 56 test cases, 853 assertions.

The expected values were derived symbolically from quadratic Taylor
polynomials, so they do not share formulas with the header. One set of numbers
serves both orders: the first-order part of every operation depends only on
the first-order parts of its inputs, so truncating a second-order expectation
to [f, g...] gives the first-order one -- the same trick the Python tests use.

These pass on the first run, which on its own proves nothing, so they were
validated by mutation. Two deliberate faults, counting failures in the old
suite against the new one:

    unary() skips the gradient loop at order 1     old 0    new 4
    data_length() one short for dynamic            old 1    new 22

The first is invisible to the old suite. It would break every math function
for every first-order type and still ship green.

Kept separate from test.cpp rather than converting it: that file holds
hand-verified numbers for one variant in full detail, which is worth keeping
as-is, and a separate TU compiles in parallel (1.8 s next to 3.2 s).

No production changes. Verified in Debug, Release and under clang with
ASan+UBSan, all with -Werror: 101 test cases, 1302 assertions. Python 172
passed, unchanged with the file stashed.
@oberbichler oberbichler mentioned this pull request Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant