[mnajafian-nv]

Overview

Harnesses report token counts differently. Some include cached tokens in input; others report them separately. Missing usage can also appear as zero. Comparing these numbers directly can make incomplete runs look cheaper.

Add shared accounting rules to NVIDIA NeMo Fabric, with the existing Deep Agents adapter as the first collector. SDK users get normalized token counts and reasons for unknown values through RunResult.model_usage, without enabling recording or Relay.

Fabric returns a total only when the usage evidence accounts for the relevant work. When it cannot establish a total, that field is null. Available counts remain separately accessible in observed, with reasons in gaps. A reported zero stays zero.

Details

Normalize input to include cache reads and writes, output to include reasoning tokens, and total to equal input plus output. Each field can be unknown independently. A per-model breakdown is included when the source establishes which model performed the work.

Model work, including subagent work, belongs to the Fabric invocation that started it. Counts are finalized when that invocation returns. Late usage cannot change its result or move into a later invocation. A failed invocation can still have known usage. Missing usage or unfinished model work leaves the affected totals unknown. Values in observed are not guaranteed lower bounds.

The Deep Agents collector combines LangChain callbacks with HTTP-attempt hooks on adapter-created ChatOpenAI clients. This includes summarization and local subagents using those clients. Unmanaged model overrides and dispatched remote Agent Protocol work leave the affected totals unknown. For clients whose usage is available only through LangGraph messages, existing RunResult.usage.input_tokens and RunResult.usage.output_tokens can now be null. Their graph-message subtotals remain in RunResult.output.usage as diagnostics and must not be treated as complete invocation totals.

The contract keeps provider-reported cost separate from estimates. The Deep Agents collector does not collect provider-reported cost. Catalog estimates follow in #478. The run journal and readable report are added separately in #423.

If structurally valid accounting cannot be normalized, Fabric keeps the execution outcome and raw usage and adds an accounting diagnostic. Extensions that violate the adapter schema still fail validation.

Consumers with older strict result decoders must upgrade before reading model_usage. Rust struct literals that list every RunResult field must also add model_usage.

Validation

The Rust workspace tests, native Python build and focused Python accounting checks passed. Shared fixtures check normalization across Rust, Python and TypeScript. Regression cases cover independent dimensions, reported zero, omissions, invalid evidence and schema-copy parity. Deep Agents cases cover retries, summaries, parallel in-process subagents, concurrent runtimes and late callbacks.

With the checkout built and Python dependencies installed, run the focused checks from the repository root:

python -m pytest -p no:cacheprovider tests/python/test_model_usage.py tests/python/test_usage_reducer_parity.py tests/adapters/test_deepagents_accounting.py -q

Where should the reviewer start?

Review the shared contract first:

1. Check crates/fabric-core/src/model_usage.rs, usage_normalization.rs, and schemas/extensions/usage-accounting.schema.json for when totals are known, when they stay null, and which observations remain available. Compare those rules with the shared conformance fixtures and Python and TypeScript reducers.
2. Check adapters/python/deepagents/src/nemo_fabric_adapters/deepagents/usage.py and tests/adapters/test_deepagents_accounting.py for retries, subagents, and usage that arrives after an invocation returns.

The files under docs/reference/api/ are generated API references. Spot-check them against the public types after reviewing the source.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Relates to: none

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.

• I searched existing issues and open pull requests, and this does not duplicate existing work.

[github-actions]

Fern docs preview: https://nvidia-preview-pull-request-442.docs.buildwithfern.com/nemo/fabric

[mnajafian-nv]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[AjayThorve]

Reviewed the shared contract, independent normalization dimensions, schema parity, runtime failure preservation and Deep Agents invocation ownership at 1007bcb. Local validation: Rust workspace 316 passed; native Python build; 307 focused accounting/Deep Agents/contract checks; Python suite 2,621 passed, 274 skipped with tests/e2e excluded; TypeScript contract/package tests passed. Current product CI and required DCO are green. The NVSkills request job lacks validation evidence and still needs maintainer follow-through. Draft status is unchanged. This is accounting approval, not a B200 task qualification: the retained Deep Agents filesystem/timeout failures remain separate.

[mnajafian-nv]

The focused Harbor fixes are merged, and the broader usage needs are documented for team review. I’m closing this contract branch so future implementation can follow a shared design decision.