Skip to content

refactor: drop the runtime size from statically sized scalars - #82

Merged
oberbichler merged 1 commit into
mainfrom
refactor/drop-static-size
Jul 27, 2026
Merged

oberbichler merged 1 commit into
mainfrom
refactor/drop-static-size

Conversation

@oberbichler

Copy link
Copy Markdown
Owner

Problem

m_size only means anything when TSize is Dynamic. For every other variant it was eight bytes that nothing reads — and the constructor taking Data never set them, which is what clang-tidy reported in #80:

hyperjet.h:322: 1 uninitialized field at the end of the constructor call
                [clang-analyzer-optin.cplusplus.UninitializedObject]
  322 |   DDScalar(const Data &data) : m_data(data) {
   56 |   index m_size;   <- uninitialized

Nothing read the field, so nothing was visibly wrong. But every copy of such an object copied an indeterminate value, which is UB to read and would trip MSan.

Change

The field gets an empty stand-in type for the static variants, marked [[no_unique_address]] ([[msvc::no_unique_address]] on MSVC, which still needs the vendor spelling). A static scalar is now exactly its data:

before after
DDScalar<1, double, 3> 40 32
DDScalar<2, double, 3> 88 80
DDScalar<2, double, 8> 368 360
DDScalar<2, double, 16> 1232 1224
DDScalar<2, double, Dynamic> 32 32

The initializer-list constructor is the one place that runs for both variants and set the field from its member initializer list. It now computes the size into a local and only stores it when there is somewhere to store it.

Why this matters beyond the lint

This settles the layout question a NumPy dtype asks of its element type. A static scalar is now payload-sized and trivially copyable; the dynamic one is neither — which is the concrete reason only the static variants could ever back a dtype. Eight bytes per element of dead weight would otherwise have been an uninitialized byte range inside every array element, plus 10 % wasted bandwidth on the hot path for size 3.

Not a performance change — measured, not assumed

Five alternating runs of a * b + a.sin() over 20k elements, -O3 -DNDEBUG:

main this branch
size 3 10.81 · 10.85 · 10.86 · 10.92 · 13.27¹ 10.74 · 10.81 · 10.91 · 10.93 · 11.00
size 8 52.04 · 52.11 · 52.22 · 52.47 · 52.89 51.60 · 51.84 · 51.87 · 51.91 · 52.41

¹ first run, cold start

Size 3 unchanged; size 8 consistently a little faster in all five runs — the smaller object pays off marginally. Worth recording: my first measurement showed 17.96 → 20.12 ns and looked like a 12 % regression. It was cold-start noise, and repeating it is what showed that.

Verification

  • C++ 102 test cases, 1310 assertions — Release with -Werror, and under clang with ASan+UBSan
  • Python 172 passed; exported API dump identical against a build of the previous state
  • clang-tidy 51 → 50 findings, the uninitialized field gone, 0 errors

TEST_CASE("Static scalars are exactly their data") pins it: sizeof(T) == sizeof(T::Data) for sizes 0, 3 and 8 across both orders, trivial copyability and standard layout for the static variants, and CHECK_FALSE(is_trivially_copyable_v<...>) for the dynamic one. RED was 4 of its 8 checks failing.

Notes

Based on main. Independent of #81 (binding codegen, still open) — that one touches python/, this one include/.

Next in the sequence we discussed: the NumPy dtype itself, once #81 lands.

m_size only means anything when TSize is Dynamic. For every other variant it
was eight bytes that nothing reads, and the constructor taking Data never set
them, which is what clang-tidy reported in #80:

    hyperjet.h:322: 1 uninitialized field at the end of the constructor call
                    [clang-analyzer-optin.cplusplus.UninitializedObject]
      322 |   DDScalar(const Data &data) : m_data(data) {
       56 |   index m_size;   <- uninitialized

Nothing read the field, but every copy of such an object copied an
indeterminate value, which is UB to read and would trip MSan.

The field now has an empty stand-in type for the static variants, marked
[[no_unique_address]], so a static scalar is exactly its data:

                                  before   after
    DDScalar<1, double, 3>            40      32
    DDScalar<2, double, 3>            88      80
    DDScalar<2, double, 8>           368     360
    DDScalar<2, double, 16>         1232    1224
    DDScalar<2, double, Dynamic>      32      32

The initializer list constructor is the one place that ran for both variants
and set the field from its member initializer list, so it computes the size
into a local and only stores it when there is somewhere to store it.

This also settles the layout question a NumPy dtype would ask of the element
type: a static scalar is now payload-sized and trivially copyable, and the
dynamic one is neither, which is why only the static variants could ever back
a dtype.

Not a performance change, but measured rather than assumed. Five alternating
runs of `a * b + a.sin()` over 20k elements, -O3 -DNDEBUG: size 3 unchanged at
about 10.85 ns per element, size 8 consistently a little faster, 51.6-52.4
against 52.0-52.9. The first measurement suggested a 12 percent regression and
was cold-start noise.

  C++ 102 test cases, 1310 assertions, in Release with -Werror and under clang
  with ASan+UBSan
  Python 172 passed, exported API identical against a build of the previous
  state
  clang-tidy down to 50 findings, the uninitialized field gone
@oberbichler
oberbichler merged commit 1692918 into main Jul 27, 2026
17 checks passed
@oberbichler
oberbichler deleted the refactor/drop-static-size branch July 27, 2026 06:14
oberbichler added a commit that referenced this pull request Jul 27, 2026
Arrays of scalars were object arrays: a pointer per element and a Python
object behind each one, so every NumPy operation was an object loop. The
static variants have a fixed element size and, since #82, are exactly their
data and trivially copyable, which is what a dtype needs. Measured over 10k
elements of DD3Scalar:

                  object      dtype
    a + b        349.5 ns     2.5 ns    138x
    a * b        354.3 ns     3.9 ns     92x
    np.sqrt(a)   349.2 ns     2.8 ns    123x
    np.sin(a)    366.4 ns     7.0 ns     52x
    np.sum(a)    353.1 ns     3.8 ns     94x
    a @ b        673.0 ns     2.7 ns    246x

One dtype per static type, generated from the same template as the bindings,
which is what #81 was the prerequisite for. No parametric dtype is needed: the
size follows from the type. The dynamic variants hold a std::vector and stay
object arrays.

27 ufunc loops per type: the four arithmetic ones, negative and positive,
arctan2, hypot, matmul, vecdot, and 19 unary mathematical functions. All of
them copy through local values, which makes them safe for unaligned data at no
cost.

np.dot and np.linalg.norm do not work on these arrays and cannot be made to.
np.dot is not a ufunc but an __array_function__ dispatcher: it dispatches on
the array type rather than the dtype, and our arrays are plain ndarrays, so the
protocol never fires. It then refuses anything that is not a native or an
old-style dtype, before consulting any slot -- the legacy dotfunc slot exists
and was tried, and is unreachable. Registering as an old-style dtype would
work but builds the feature on the API NumPy is removing. `@`, np.matmul and
np.vecdot compute the same and work on object arrays too, so the three tests
that used np.dot now use `@`.

Requirements NumPy enforces at registration, none of them documented -- each
surfaced only as an error and cost an iteration: __repr__ and __str__ are
mandatory, so is a cast between the DType's own instances, that cast has to
handle unaligned data and declare NPY_METH_SUPPORTS_UNALIGNED, and the DType
needs its own tp_new. A null type object is rejected, and so is sharing one
type object between DTypes, which is why the dtype has to hang off the scalar
class and picking it up in np.array is not optional.

The loop registration allocates its spec on the heap for the lifetime of the
module. Function-local statics would have been shared by every call with the
same type and arity, and NumPy may keep pointers into the spec.

  Python 239 passed, 66 of them new in test_dtype.py
  C++ 102 test cases, 1310 assertions, Release with -Werror and under
  clang with ASan+UBSan
  all 34 generated dtypes checked for element size, boxing and arithmetic

The new tests were validated by mutation, and the first version failed that:
comparing a ufunc on an array against the same ufunc on scalars passes
whatever the loop computes, because both go through it. They now compare
against the scalar operators, and injecting subtract-computes-plus and
arccosh-computes-arcsinh fails four of them.

Build time is unchanged at 37.4 s; the module grows from 2.8 to 3.4 MB.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant