Skip to content

Report Claude result cost per result and expose the cumulative session cost - #47

Merged
andrew merged 2 commits into
alpha-omega-security:mainfrom
abhinavgautam01:fix/1159-claude-incremental-result-cost
Oct 12, 2026
Merged

andrew merged 2 commits into
alpha-omega-security:mainfrom
abhinavgautam01:fix/1159-claude-incremental-result-cost

Conversation

@abhinavgautam01

Copy link
Copy Markdown
Contributor

Part of alpha-omega-security/scrutineer#1159

Note

This builds on #46, so its commit appears here until #46 merges. The change in this PR is only the top commit, 48efe32: abhinavgautam01@48efe32. I'll rebase onto main once #46 is in.

Problem

Claude's total_cost_usd is the session's cumulative cost, not the cost of the result line that carries it. Every result's usage and num_turns cover only that result, so callers sum them, and they sum CostUSD the same way. That overcounts in two cases (details and reproductions in scrutineer#1159):

  1. Several results in one invocation. A background subagent can make the CLI print a second result after the main loop finishes. Both lines carried total_cost_usd 0.04407415, so summing gave $0.0882 for a $0.0441 run.
  2. --resume. The resumed invocation's result includes the earlier invocation's cost. A resume reported $0.015707 when its own tokens price at $0.0046134, which is exactly the $0.0110936 from the first run added back.

The parser sees only one invocation, so it can fix the first case on its own. The second needs the caller to know what it already recorded.

Change

  • Claude results report what they added. The per-stream state from Emit per-call usage events with list-price estimates #46 now keeps the latest total_cost_usd. Each result's CostUSD is the increase over it, so summing results within one invocation gives the session total. Tokens and turns pass through unchanged, since they already cover only their own result.
  • New Event.SessionCostUSD. It carries the raw cumulative total, so a caller that resumes a session can subtract the total it recorded for the earlier invocation from the first result. That subtraction is the scrutineer half of #1159.
  • Copilot sets SessionCostUSD from its billing checkpoint. session.usage_checkpoint is already documented as session-cumulative on a resume, so the field means the same thing on both backends. Copilot's CostUSD is unchanged. A list-price estimate without a checkpoint leaves SessionCostUSD at zero.
  • The Event doc and README describe both fields and the resume rule.

OpenCode and Codex already report each result's own cost, so they are untouched.

Verification

  • Replaying the real CLI output from scrutineer#1159 through the parser:

    Stream Before After
    Two results, background subagent 0.04407415 + 0.04407415 = $0.0882 0.04407415 + 0 = $0.04407415
    Resumed session $0.015707 $0.015707 with SessionCostUSD 0.015707, so the caller subtracts its recorded 0.0110936
  • New tests:

    • Claude: the increase across three results, including a repeated total that adds zero. The sum equals the session total, tokens and turns are unchanged, and a result with no cost leaves SessionCostUSD unset.
    • Copilot: the checkpoint sets SessionCostUSD, and a list-price estimate leaves it zero.
  • Mutation checks: reporting the raw total, never advancing the stored total, or dropping Copilot's SessionCostUSD each makes a test fail.

  • gofmt, go vet (also -tags integration), go test -race ./..., golangci-lint v2.14.0 (0 issues) and go mod tidy -diff all pass.

Note for callers

A Claude result's CostUSD used to be the session total so far. It is now what that result added. Callers that sum CostUSD over result events become correct within an invocation. Callers that read a single result's CostUSD as the session total should read SessionCostUSD instead.

@abhinavgautam01
abhinavgautam01 force-pushed the fix/1159-claude-incremental-result-cost branch 2 times, most recently from 784c710 to cc332fc Compare October 10, 2026 09:23

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resume rule in the Event doc and the new README paragraph says to subtract the SessionCostUSD recorded for the earlier invocation unconditionally. That breaks for Copilot when the earlier invocation saw a session.usage_checkpoint and the resumed one did not: the resumed CostUSD is then a per-invocation list-price estimate with SessionCostUSD 0, and subtracting the earlier total gives a negative or undercounted cost. Please say to subtract only when the resumed result also sets SessionCostUSD.

In TestClaudeResultCostIsIncrementalWithinStream the comment says the values are from a real CLI run, but the first result's 0.03 is invented; in the run from scrutineer#1159 both results carried 0.04407415. Either use that real pair, which also covers a first result that is already the full total and a second that adds 0, or drop the claim.

@abhinavgautam01
abhinavgautam01 force-pushed the fix/1159-claude-incremental-result-cost branch from cc332fc to cd1f6a0 Compare October 10, 2026 10:16
@abhinavgautam01

Copy link
Copy Markdown
Contributor Author

Thanks, fixed. The Event doc and README now say to subtract only when the resumed result also sets SessionCostUSD and a resumed Copilot result without a checkpoint is used unchanged. The test now uses the real 0.04407415 pair from scrutineer#1159 and the third result is marked as synthetic so a non-zero increase is still covered.

@andrew
andrew force-pushed the fix/1159-claude-incremental-result-cost branch from cd1f6a0 to 05a48bc Compare October 12, 2026 08:46

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both earlier findings are addressed. Claude results report incremental cost within an invocation, and the documented resume rule handles Copilot results without billing checkpoints.

@andrew
andrew merged commit d2d4efa into alpha-omega-security:main Oct 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants