Repository navigation
Report Claude result cost per result and expose the cumulative session cost - #47
Conversation
784c710 to
cc332fc
Compare
andrew
left a comment
There was a problem hiding this comment.
The resume rule in the Event doc and the new README paragraph says to subtract the SessionCostUSD recorded for the earlier invocation unconditionally. That breaks for Copilot when the earlier invocation saw a session.usage_checkpoint and the resumed one did not: the resumed CostUSD is then a per-invocation list-price estimate with SessionCostUSD 0, and subtracting the earlier total gives a negative or undercounted cost. Please say to subtract only when the resumed result also sets SessionCostUSD.
In TestClaudeResultCostIsIncrementalWithinStream the comment says the values are from a real CLI run, but the first result's 0.03 is invented; in the run from scrutineer#1159 both results carried 0.04407415. Either use that real pair, which also covers a first result that is already the full total and a second that adds 0, or drop the claim.
cc332fc to
cd1f6a0
Compare
|
Thanks, fixed. The |
cd1f6a0 to
05a48bc
Compare
andrew
left a comment
There was a problem hiding this comment.
Both earlier findings are addressed. Claude results report incremental cost within an invocation, and the documented resume rule handles Copilot results without billing checkpoints.
Part of alpha-omega-security/scrutineer#1159
Note
This builds on #46, so its commit appears here until #46 merges. The change in this PR is only the top commit, 48efe32: abhinavgautam01@48efe32. I'll rebase onto
mainonce #46 is in.Problem
Claude's
total_cost_usdis the session's cumulative cost, not the cost of the result line that carries it. Every result'susageandnum_turnscover only that result, so callers sum them, and they sumCostUSDthe same way. That overcounts in two cases (details and reproductions in scrutineer#1159):resultafter the main loop finishes. Both lines carriedtotal_cost_usd0.04407415, so summing gave $0.0882 for a $0.0441 run.--resume. The resumed invocation's result includes the earlier invocation's cost. A resume reported $0.015707 when its own tokens price at $0.0046134, which is exactly the $0.0110936 from the first run added back.The parser sees only one invocation, so it can fix the first case on its own. The second needs the caller to know what it already recorded.
Change
total_cost_usd. Each result'sCostUSDis the increase over it, so summing results within one invocation gives the session total. Tokens and turns pass through unchanged, since they already cover only their own result.Event.SessionCostUSD. It carries the raw cumulative total, so a caller that resumes a session can subtract the total it recorded for the earlier invocation from the first result. That subtraction is the scrutineer half of #1159.SessionCostUSDfrom its billing checkpoint.session.usage_checkpointis already documented as session-cumulative on a resume, so the field means the same thing on both backends. Copilot'sCostUSDis unchanged. A list-price estimate without a checkpoint leavesSessionCostUSDat zero.OpenCode and Codex already report each result's own cost, so they are untouched.
Verification
Replaying the real CLI output from scrutineer#1159 through the parser:
SessionCostUSD0.015707, so the caller subtracts its recorded 0.0110936New tests:
SessionCostUSDunset.SessionCostUSD, and a list-price estimate leaves it zero.Mutation checks: reporting the raw total, never advancing the stored total, or dropping Copilot's
SessionCostUSDeach makes a test fail.gofmt,go vet(also-tags integration),go test -race ./...,golangci-lintv2.14.0 (0 issues) andgo mod tidy -diffall pass.Note for callers
A Claude result's
CostUSDused to be the session total so far. It is now what that result added. Callers that sumCostUSDover result events become correct within an invocation. Callers that read a single result'sCostUSDas the session total should readSessionCostUSDinstead.