Skip to content

fix(protocols): preserve native cache usage counters - #175

Open
sernote wants to merge 1 commit into
Continuum-AI-Corp:mainfrom
sernote:codex/native-usage
Open

sernote wants to merge 1 commit into
Continuum-AI-Corp:mainfrom
sernote:codex/native-usage

Conversation

@sernote

@sernote sernote commented Oct 26, 2026 •

Copy link
Copy Markdown

Orca-Code-Review — push 1

Severity Count
P0 0
P1 0
P2 0
P3 0

✅ no blocking findings

Native Anthropic responses exposed LiteLLM's inclusive prompt total as ordinary input_tokens and omitted the cache read/write breakdown. Native Gemini responses also dropped the available cached-token count. Both blocking responses and terminal stream usage were affected.

Project known cache counters into each native protocol: subtract known read/write components once for Anthropic, and retain Gemini's inclusive prompt count while adding cachedContentTokenCount. Preserve explicit zero, omit unknown counters, and reject inconsistent or out-of-bounds breakdowns. The documented fallback retains the legacy inclusive total when a precise ordinary-input count cannot be established.

Validation: 842 tests passed in the offline Python 3.12 suite. Independent review reran 130 targeted tests plus 12 additional controls using actual LiteLLM usage normalizers and stream parsers. Native SDK terminal-counter behavior was checked against official SDK source; native SDK packages were not installed. Ruff and git diff --check passed. No live provider or billing measurements.

Discovered during a repository audit using audit-prompt-caching, then independently reproduced and reviewed.

@orcacode-review orcacode-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐳 OrcaCode Review

✅ No findings — nothing to flag in this PR. Great work!

OrcaCode Review — Route Smarter. Ship Safer. Spend Less.
Engine-reported: 198 calls · 7.7M tokens · 94% cached

❤️ Share · Install OrcaCode Review

Free on GitHub — the review runs on your own OrcaRouter key. If it helped, a shout-out goes a long way.

Share: X · Reddit · LinkedIn
Follow: X · Discord · LinkedIn · OrcaRouter

@sernote
sernote marked this pull request as ready for review October 26, 2026 20:56

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant