Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ns_cmsis_nn_canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@ on:
ns_cmsis_nn_ref:
description: "ns-cmsis-nn commit SHA, tag (e.g. v7.35.0) or branch to build against"
required: true
# Current repository pin: AmbiqAI/ns-cmsis-nn v7.39.3. Dispatching with
# Current repository pin: AmbiqAI/ns-cmsis-nn v7.41.0. Dispatching with
# the default is a no-op control run that should match the normal
# matrix; keep this in step with ext_libs/helia.inc.
default: "94bf3b9362874d951f9363bb0ddef72fd162e8f6"
default: "48ddbf7d863745eee108c68726f42fc82dabe0af"

permissions:
contents: read
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/smoke_cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ permissions:
env:
# Pin CI to the tested ns-cmsis-nn release, shared by every job that
# clones it so the cross-compile legs cannot drift apart.
NS_CMSIS_NN_REF: v7.39.3
NS_CMSIS_NN_REF: v7.41.0

jobs:
manifest:
Expand Down Expand Up @@ -241,6 +241,7 @@ jobs:
packs=(dist/Ambiq.helia-rt.*.pack)
PACK="${packs[0]}"
test -f "${PACK}"
python3 tools/cmsis_pack/check_pdsc_test.py
python3 tools/cmsis_pack/check_pdsc.py "${PACK}"
- name: Install consumer toolchains (Arm GNU, ATfE)
run: |
Expand Down
6 changes: 3 additions & 3 deletions astro-site/src/content/docs/getting-started/cmsis-pack.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,15 @@ In your project's component selection, use the Ambiq `Machine Learning` / `TFLM
|---|---|
| `Reference` | Portable runtime kernels without an external NN library. |
| `CMSIS-NN` | Upstream Arm CMSIS-NN kernel dependency for your target. |
| `HELIA` | The Ambiq heliaCORE (`ns-cmsis-nn`) source component. The generated PDSC requires version 7.39.2 or newer. |
| `HELIA` | The Ambiq heliaCORE (`ns-cmsis-nn`) source component. The generated PDSC requires version 7.41.0 or newer. |

Compile the runtime's C++ sources with `-fno-exceptions`: its arena objects hide `operator delete`, so its placement `new` does not compile with exceptions enabled. The example below sets it for the whole solution.

Keep runtime and kernel feature settings consistent. A HELIA component selection does not establish support for every operator, tensor type or model shape. Review [Model compatibility](/helia-rt/guide/model-compatibility/) before enabling float kernels.

## Build the example consumer

[`tools/cmsis_pack/examples/hello_world`](https://github.com/AmbiqAI/helia-rt/tree/main/tools/cmsis_pack/examples/hello_world) is a minimal CMSIS-Toolbox solution. It targets the generic `ARM::ARMCM55` device and builds one project per variant: `hello_world` selects `Reference`, and `hello_world_helia` selects `HELIA` with heliaCORE built from the `Ambiq::NS-CMSIS-NN` source pack. Each links a `MicroInterpreter` against the installed packs. Add the heliaRT pack with `cpackget`, plus the ns-cmsis-nn pack (`Ambiq.NS-CMSIS-NN.<version>.pack` from the [ns-cmsis-nn release](https://github.com/AmbiqAI/ns-cmsis-nn/releases) that heliaRT pins, 7.39.2 or newer). Then point CMSIS-Toolbox 2.13.0 at each compiler's `bin` directory with a variable named after its version, for example `GCC_TOOLCHAIN_14_3_1` for Arm GNU 14.3.rel1 and `CLANG_TOOLCHAIN_22_1_0` for ATfE 22.1.0. Then build:
[`tools/cmsis_pack/examples/hello_world`](https://github.com/AmbiqAI/helia-rt/tree/main/tools/cmsis_pack/examples/hello_world) is a minimal CMSIS-Toolbox solution. It targets the generic `ARM::ARMCM55` device and builds one project per variant: `hello_world` selects `Reference`, and `hello_world_helia` selects `HELIA` with heliaCORE built from the `Ambiq::NS-CMSIS-NN` source pack. Each links a `MicroInterpreter` against the installed packs. Add the heliaRT pack with `cpackget`, plus the ns-cmsis-nn pack (`Ambiq.NS-CMSIS-NN.<version>.pack` from the [ns-cmsis-nn release](https://github.com/AmbiqAI/ns-cmsis-nn/releases) that heliaRT pins, 7.41.0 or newer). Then point CMSIS-Toolbox 2.13.0 at each compiler's `bin` directory with a variable named after its version, for example `GCC_TOOLCHAIN_14_3_1` for Arm GNU 14.3.rel1 and `CLANG_TOOLCHAIN_22_1_0` for ATfE 22.1.0. Then build:

```bash
cd tools/cmsis_pack/examples/hello_world
Expand All @@ -62,7 +62,7 @@ cbuild hello_world.csolution.yml --toolchain CLANG --update-rte --packs

`--packs` installs the pinned public `ARM::CMSIS` and `ARM::Cortex_DFP` packs from the public pack index on first use. The example links `--specs=nosys.specs` on GCC and `-lsemihost` on ATfE to satisfy the C library's I/O and exit hooks; a board application supplies its own retarget layer instead. The packaging workflow builds this example on both compilers. It proves the `Reference` and `HELIA` variants compile and link together with their dependencies. It does not build the `CMSIS-NN` variant or run a model.

If a `HELIA` build fails with `Include/arm_nnfunctions.h: No such file or directory`, the installed ns-cmsis-nn pack is older than 7.39.2, the first release that exports its root as an include path. CMSIS-Toolbox reports the unmet PDSC requirement only as a warning. Install ns-cmsis-nn 7.39.2 or newer.
If a `HELIA` build fails with `Include/arm_nnfunctions.h: No such file or directory`, the installed ns-cmsis-nn pack is older than 7.39.2, the first release that exports its root as an include path. CMSIS-Toolbox reports the unmet PDSC requirement only as a warning. Install ns-cmsis-nn 7.41.0 or newer to satisfy the current HELIA source requirement.

Your consumer project still provides the device/board support, compiler settings, startup code, memory layout and application sources. Add the model and inference sequence from [First inference](/helia-rt/getting-started/first-inference/), build with your CMSIS toolchain, then verify known inputs and outputs on the board.

Expand Down
2 changes: 1 addition & 1 deletion astro-site/src/content/docs/getting-started/source.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ set(HELIA_RT_NSCMSISNN_TARGET "ns-cmsis-nn" CACHE STRING "Kernel dependency")
add_subdirectory(third_party/helia-rt)
```

Link `rt_app` to `helia_rt::helia` instead of `helia_rt::reference`. Supply the CMSIS Core headers and target compiler flags required by your heliaCORE checkout. Enable float features before adding that dependency when the model needs them. For the adapters in this runtime release, use heliaCORE v7.35.0 or newer.
Link `rt_app` to `helia_rt::helia` instead of `helia_rt::reference`. Supply the CMSIS Core headers and target compiler flags required by your heliaCORE checkout. Enable float features before adding that dependency when the model needs them. For the current HELIA adapters, use heliaCORE v7.41.0 or newer.

CMake links heliaCORE transitively into the final application; it does not physically merge two static libraries. For the upstream Arm backend, use `HELIA_RT_ENABLE_CMSIS_NN=ON`, provide its target through `HELIA_RT_CMSISNN_TARGET`, and link `helia_rt::cmsis_nn` instead. Do not substitute heliaCORE for upstream CMSIS-NN.

Expand Down
2 changes: 1 addition & 1 deletion astro-site/src/content/docs/getting-started/zephyr.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Add these projects to the `manifest.projects` list in your workspace's west mani
path: modules/helia-rt
- name: ns-cmsis-nn
url: https://github.com/AmbiqAI/ns-cmsis-nn
revision: v7.39.3
revision: v7.41.0
path: modules/ns-cmsis-nn
```

Expand Down
9 changes: 6 additions & 3 deletions cmake/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ When consuming via the top-level `CMakeLists.txt` (or NSX/Zephyr source-mode she
|---|---|---|
| `HELIA_RT_BUILD_TYPE` | `release_with_logs` | `debug` / `release_with_logs` (defines `NDEBUG`) / `release` (also `TF_LITE_STRIP_ERROR_STRINGS`) |
| `HELIA_RT_ENABLE_CMSIS_NN` | `OFF` | Materialize `helia_rt::cmsis_nn`. Requires consumer-provided `cmsis-nn` target. |
| `HELIA_RT_ENABLE_HELIA` | `OFF` | Materialize `helia_rt::helia`. Requires consumer-provided `ns-cmsis-nn` target. |
| `HELIA_RT_ENABLE_HELIA` | `OFF` | Materialize `helia_rt::helia`. Requires a consumer-provided `ns-cmsis-nn` v7.41.0 or newer target. |
| `HELIA_RT_GLOBAL_KERNEL_OPTIMIZE` | `SPEED` | HELIA backend kernel profile: `SPEED` or `SIZE`. |
| `HELIA_RT_CONV_OPT` | (empty) | Optional HELIA conv-family override: `SPEED` or `SIZE`; empty inherits `HELIA_RT_GLOBAL_KERNEL_OPTIMIZE`. |
| `HELIA_RT_FC_OPT` | (empty) | Optional HELIA fully-connected override: `SPEED` or `SIZE`; empty inherits `HELIA_RT_GLOBAL_KERNEL_OPTIMIZE`. |
Expand All @@ -72,8 +72,11 @@ add_subdirectory(third_party/helia-rt)
target_link_libraries(my_app PRIVATE helia_rt::reference)
```

For the HELIA backend, select the source-build kernel profile before adding
the repo:
For the HELIA backend, provide a released ns-cmsis-nn v7.41.0 or newer
target before adding the repo. Its exported FP32/FP16 definitions must match
the library. The build checks the canonical header version; the application
remains responsible for supplying the matching library. Select the
source-build kernel profile before adding the repo:

```cmake
set(HELIA_RT_ENABLE_HELIA ON CACHE BOOL "" FORCE)
Expand Down
8 changes: 8 additions & 0 deletions cmake/helia_rt_core_version.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// Copyright 2026 Ambiq
// SPDX-License-Identifier: Apache-2.0

#include "Include/arm_nn_types.h"

#if !defined(NS_CMSIS_NN_VERSION) || NS_CMSIS_NN_VERSION < 7041000
#error "heliaRT requires ns-cmsis-nn v7.41.0 or newer"
#endif
4 changes: 4 additions & 0 deletions cmake/helia_rt_sources.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,10 @@ function(helia_rt_select_kernel_sources OUT_VAR)
endif()
endforeach()

if(_ARG_BACKEND STREQUAL "helia")
list(APPEND _out "${HELIA_RT_ROOT}/cmake/helia_rt_core_version.cc")
endif()

set(${OUT_VAR} ${_out} PARENT_SCOPE)
endfunction()

Expand Down
2 changes: 1 addition & 1 deletion nsx/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
# nsx_soc_hal SoC HAL (transitive deps for debug_log glue)
#
# Backend-specific upstream targets:
# helia: nsx::cmsis_nn (or alias) — ns-cmsis-nn kernels, with Ambiq's
# helia: nsx::cmsis_nn (or alias) — ns-cmsis-nn >= 7.41.0, with Ambiq's
# additive weight-sum / fast-path symbols actively referenced.
# fp32/fp16 are opt-in: set ARM_NN_ENABLE_F32/F16 before
# add_subdirectory(<ns-cmsis-nn>/nsx ...), which is where
Expand Down
2 changes: 1 addition & 1 deletion nsx/nsx-module.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ support:

summary: >-
Ambiq-optimized TensorFlow Lite for Microcontrollers (TFLM) runtime.
Built from source against the nsx-cmsis-nn (ns-cmsis-nn) kernel module.
Built from source against the nsx-cmsis-nn (ns-cmsis-nn >= 7.41.0) kernel module.

capabilities:
- inference
Expand Down
2 changes: 1 addition & 1 deletion tensorflow/lite/micro/kernels/helia/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ For more information about the optimizations, check out
[CMSIS-NN documentation](https://github.com/ARM-software/CMSIS-NN/blob/main/README.md),

The helia backend's floating-point build support is provided by the pinned
`AmbiqAI/ns-cmsis-nn` v7.39.3 release. The Make build enables the fp32 API for helia
`AmbiqAI/ns-cmsis-nn` v7.41.0 release. The Make build enables the fp32 API for helia
targets and enables the fp16 API only for `TARGET_ARCH=cortex-m55`. fp16 requires an
ARMv8.1-M MVEF-capable toolchain.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ struct LinkState {
int32_t compute_context_sizes[static_cast<int>(ComputeRoute::kCount)] = {};
bool compute_context_buffers[static_cast<int>(ComputeRoute::kCount)] = {};
int weight_query_calls = 0;
// heliaCORE's weight-sum precompute may query the size itself; only the
// kernel's own queries are counted.
bool in_precompute = false;
int precompute_calls[static_cast<int>(PrecomputeRoute::kCount)] = {};
int32_t weight_context_sizes[static_cast<int>(PrecomputeRoute::kCount)] = {};
bool weight_context_buffers[static_cast<int>(PrecomputeRoute::kCount)] = {};
Expand Down Expand Up @@ -187,9 +190,12 @@ int32_t __wrap_arm_depthwise_conv_wrapper_s16_get_buffer_size(
int32_t __real_arm_convolve_s8_get_weights_sum_size(const cmsis_nn_dims*);
int32_t __wrap_arm_convolve_s8_get_weights_sum_size(
const cmsis_nn_dims* output_dims) {
++g_link_state.weight_query_calls;
const int32_t actual =
__real_arm_convolve_s8_get_weights_sum_size(output_dims);
if (g_link_state.in_precompute) {
return actual;
}
++g_link_state.weight_query_calls;
int32_t result = actual;
if (g_link_state.override_weight_query) {
result = g_link_state.weight_query_result < 0 || actual == 0
Expand All @@ -213,8 +219,10 @@ arm_cmsis_nn_status __wrap_arm_convolve_weight_sum(
if (g_link_state.precompute_error == PrecomputeRoute::kConv) {
return ARM_CMSIS_NN_ARG_ERROR;
}
g_link_state.in_precompute = true;
const arm_cmsis_nn_status status = __real_arm_convolve_weight_sum(
sums, filter, input_dims, filter_dims, output_dims, input_offset, bias);
g_link_state.in_precompute = false;
if (status == ARM_CMSIS_NN_NO_IMPL_ERROR) {
if (sums != nullptr) {
sums[0] = input_offset * filter[0] + (bias == nullptr ? 0 : bias[0]);
Expand All @@ -237,9 +245,11 @@ arm_cmsis_nn_status __wrap_arm_depthwise_convolve_weight_sum(
if (g_link_state.precompute_error == PrecomputeRoute::kDepthwise) {
return ARM_CMSIS_NN_ARG_ERROR;
}
g_link_state.in_precompute = true;
const arm_cmsis_nn_status status = __real_arm_depthwise_convolve_weight_sum(
sums, scratch, filter, params, input_dims, filter_dims, output_dims,
input_offset, bias);
g_link_state.in_precompute = false;
if (status == ARM_CMSIS_NN_NO_IMPL_ERROR) {
if (sums != nullptr) {
sums[0] = input_offset * filter[0] + (bias == nullptr ? 0 : bias[0]);
Expand Down
140 changes: 140 additions & 0 deletions tensorflow/lite/micro/kernels/helia/tests/quantize_s16_test.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
/* Copyright 2026 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/

#include "Include/arm_nnfunctions.h"
#include "tensorflow/lite/micro/c/common.h"
#include "tensorflow/lite/micro/kernels/kernel_runner.h"
#include "tensorflow/lite/micro/test_helpers.h"
#include "tensorflow/lite/micro/testing/micro_test_v2.h"

namespace {

#if defined(HELIA_QUANTIZE_S16_LINK_WRAP)
struct RequantizeCall {
int calls = 0;
int32_t size = 0;
int32_t multiplier = 0;
int32_t shift = 0;
int32_t input_zero_point = 0;
int32_t output_zero_point = 0;
};

RequantizeCall g_requantize_call;
#endif

} // namespace

#if defined(HELIA_QUANTIZE_S16_LINK_WRAP)
extern "C" {
arm_cmsis_nn_status __real_arm_requantize_s16_s16(const int16_t*, int16_t*,
int32_t, int32_t, int32_t,
int32_t, int32_t);

arm_cmsis_nn_status __wrap_arm_requantize_s16_s16(
const int16_t* input, int16_t* output, int32_t size, int32_t multiplier,
int32_t shift, int32_t input_zero_point, int32_t output_zero_point) {
++g_requantize_call.calls;
g_requantize_call.size = size;
g_requantize_call.multiplier = multiplier;
g_requantize_call.shift = shift;
g_requantize_call.input_zero_point = input_zero_point;
g_requantize_call.output_zero_point = output_zero_point;
return __real_arm_requantize_s16_s16(input, output, size, multiplier, shift,
input_zero_point, output_zero_point);
}
}
#endif

namespace tflite {
namespace testing {
namespace {

template <int N>
void ExpectS16Requantize(const int16_t (&input)[N], float input_scale,
int32_t expected_shift, const int16_t (&expected)[N]) {
#if !defined(HELIA_QUANTIZE_S16_LINK_WRAP)
(void)expected_shift;
#endif
int dims_data[] = {1, N};
TfLiteIntArray* dims = IntArrayFromInts(dims_data);
constexpr int16_t kSentinel = 0x5a5a;
int16_t output[N + 1] = {};
output[N] = kSentinel;
TfLiteTensor tensors[] = {
CreateQuantizedTensor(input, dims, input_scale, 0),
CreateQuantizedTensor(output, dims, 1.0f, 0),
};
float input_scales[] = {1, input_scale};
float output_scales[] = {1, 1.0f};
int zero_points[] = {1, 0};
TfLiteAffineQuantization quantization[2] = {};
quantization[0].scale = FloatArrayFromFloats(input_scales);
quantization[1].scale = FloatArrayFromFloats(output_scales);
for (int i = 0; i < 2; ++i) {
quantization[i].zero_point = IntArrayFromInts(zero_points);
tensors[i].quantization = {kTfLiteAffineQuantization, &quantization[i]};
}
int inputs_data[] = {1, 0};
int outputs_data[] = {1, 1};
const TFLMRegistration registration = Register_QUANTIZE();
micro::KernelRunner runner(registration, tensors, 2,
IntArrayFromInts(inputs_data),
IntArrayFromInts(outputs_data), nullptr);
#if defined(HELIA_QUANTIZE_S16_LINK_WRAP)
g_requantize_call = RequantizeCall{};
#endif
EXPECT_EQ(kTfLiteOk, runner.InitAndPrepare());
#if defined(HELIA_QUANTIZE_S16_LINK_WRAP)
EXPECT_EQ(0, g_requantize_call.calls);
#endif
EXPECT_EQ(kTfLiteOk, runner.Invoke());
#if defined(HELIA_QUANTIZE_S16_LINK_WRAP)
EXPECT_EQ(1, g_requantize_call.calls);
EXPECT_EQ(N, g_requantize_call.size);
EXPECT_EQ(1 << 30, g_requantize_call.multiplier);
EXPECT_EQ(expected_shift, g_requantize_call.shift);
EXPECT_EQ(0, g_requantize_call.input_zero_point);
EXPECT_EQ(0, g_requantize_call.output_zero_point);
#endif
for (int i = 0; i < N; ++i) {
EXPECT_EQ(expected[i], output[i]);
}
EXPECT_EQ(kSentinel, output[N]);
}

} // namespace
} // namespace testing
} // namespace tflite

TEST(HeliaQuantizeS16Test, LargeScalePositiveSaturation) {
const int16_t input[] = {1000};
const int16_t expected[] = {32767};
tflite::testing::ExpectS16Requantize(input, 2097152.0f, 22, expected);
}

TEST(HeliaQuantizeS16Test, LargeScaleBodyAndTail) {
const int16_t input[] = {1000, -1000, 0, 1, -1, 32767, -32768, 2, -2};
// Exact scale ratio 2^21: every nonzero input saturates with its sign.
const int16_t expected[] = {32767, -32768, 0, 32767, -32768,
32767, -32768, 32767, -32768};
tflite::testing::ExpectS16Requantize(input, 2097152.0f, 22, expected);
}

TEST(HeliaQuantizeS16Test, OrdinaryScalePreservesValues) {
const int16_t input[] = {1000, -1000, 0, 32767, -32768};
tflite::testing::ExpectS16Requantize(input, 1.0f, 1, input);
}

TF_LITE_MICRO_TESTS_MAIN
4 changes: 2 additions & 2 deletions tensorflow/lite/micro/tools/make/ext_libs/helia.inc
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ ifneq (,$(filter $(TARGET_ARCH), x86_64))
# NS-CMSIS-NN optimizations not supported
endif

# Tracks AmbiqAI/ns-cmsis-nn tag v7.39.3. Consumed by ns_cmsis_nn_download.sh
# Tracks AmbiqAI/ns-cmsis-nn tag v7.41.0. Consumed by ns_cmsis_nn_download.sh
# via the environment.
NS_CMSIS_NN_COMMIT ?= 94bf3b9362874d951f9363bb0ddef72fd162e8f6
NS_CMSIS_NN_COMMIT ?= 48ddbf7d863745eee108c68726f42fc82dabe0af
export NS_CMSIS_NN_COMMIT

ifneq (,$(NS_CMSIS_NN_LIBS))
Expand Down
10 changes: 10 additions & 0 deletions tensorflow/lite/micro/tools/make/ext_libs/helia_tests.inc
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ MICROLITE_KERNEL_SIMPLE_TEST_SRCS += \
$(TENSORFLOW_ROOT)tensorflow/lite/micro/kernels/helia/tests/prepare_contract_test.cc \
$(TENSORFLOW_ROOT)tensorflow/lite/micro/kernels/helia/tests/fp16_pad_rank_test.cc

$(eval $(call microlite_test,kernel_quantize_s16_test,\
$(TENSORFLOW_ROOT)tensorflow/lite/micro/kernels/helia/tests/quantize_s16_test.cc))

ifneq (,$(filter gcc atfe,$(TOOLCHAIN)))
$(kernel_quantize_s16_test_BINARY): CXXFLAGS += \
-DHELIA_QUANTIZE_S16_LINK_WRAP=1
$(kernel_quantize_s16_test_BINARY): LDFLAGS += \
-Wl,--wrap=arm_requantize_s16_s16
endif

# Every float golden relies on the near and float-equal checks failing on a
# NaN. see AmbiqAI/helia-rt#335
MICROLITE_KERNEL_SIMPLE_TEST_SRCS += \
Expand Down
Loading
Loading