Repository navigation
feat(ai): try OpenAI before Gemini when Anthropic is capped - #29
Merged
Merged
Conversation
The chain added in #25 was anthropic -> gemini. This puts OpenAI between them, because it is the closer substitute: the prompts and the deterministic gates were built around Claude, and a same-shaped model is likelier to produce a draft that still passes them. Gemini stays as the last link rather than the only one. `OpenAIModel` is plain HTTP for the reason `GeminiModel` gives — one POST with a documented body is less to keep working than another SDK. Three things in it are not obvious and each is a bug avoided: - `max_completion_tokens`, never `max_tokens`. The reasoning models reject the latter outright, and a fallback that 400s on every call is worse than no fallback at all. - The same reasoning-token headroom Gemini needed. Sizing the cap to the caller's `maxTokens` lets the model reason up to the limit and return an empty string, which reads downstream as a model with nothing to say. - A refusal arrives as HTTP 200 with `content: null` and `refusal` set. Read without checking it, that is a successful blank draft. The default is `gpt-5.5` rather than a dated snapshot. There is no moving `-latest` alias for the reasoning line to use the way Gemini has one, so the generation is bumped by hand and `OPENAI_MODEL` overrides it. Verified against the live API: the id resolves, and the key in the vault answers 429 `insufficient_quota` / `credit_balance_exhausted` — which `isBudgetExhausted` already classifies as budget, so the chain steps past it to Gemini rather than failing the request. That body is now a test. Also documents `GEMINI_API_KEY`, which #25 never added to `.env.example`, and corrects the 503 that told operators to set `ANTHROPIC_API_KEY` as though it were the only key that enables drafting. 510 pass, 0 fail; format and typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # docs/prd-implementation-map.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/setuphas been dying on this:#25 built the chain for exactly this and made it
anthropic -> gemini. This addsOpenAI in between, so the order becomes anthropic → openai → gemini.
OpenAI goes second rather than last because it is the closer substitute: the
prompts and the deterministic quality gates were built around Claude, and a
same-shaped model is likelier to produce a draft that still passes them.
What's here
packages/ai/src/openai.ts—OpenAIModelbehind the existingTextModelinterface, plain HTTP for the reason
GeminiModelgives.apps/server/src/index.ts, keyed onOPENAI_API_KEYexactly like the other two. Absent keys just shorten thechain; nothing refuses to boot.
GEMINI_API_KEY/GEMINI_MODELdocumented in.env.example— feat(ai): fall back to Gemini when a provider runs out of budget #25 neveradded them.
ANTHROPIC_API_KEYnow names all three.Three non-obvious things in the adapter
max_completion_tokens, nevermax_tokens. The reasoning models rejectthe latter outright, and a fallback that 400s on every call is worse than none.
GeminiModelalready paid for.Sizing the cap to the caller's
maxTokenslets the model reason up to thelimit and return an empty string, which reads downstream as a model that had
nothing to say.
content: nullandrefusalset. Read withoutchecking it, that is a successful blank draft.
Default model is
gpt-5.5, undated. There is no moving-latestalias for thereasoning line the way Gemini has one (
*-chat-latestis the chat-tuned line,not the same model renamed), so the generation is bumped by hand and
OPENAI_MODELoverrides it without touching this file.Verified against the live API
The id resolves. The key currently in the vault answers
429 insufficient_quota/credit_balance_exhausted— whichisBudgetExhaustedalready classifies as budget, so the chain steps past it to Gemini instead of
failing the request. That exact body is now a test.
The billing gate rejects before request validation, so the body shape itself
could not be exercised end to end on a credit-less key; it is covered by unit
tests against the documented format.
Deploying this changes nothing on its own
Production currently has only
ANTHROPIC_API_KEYset. That is why theGemini fallback from #25 has been inert since it merged — there was never a
second link. Merging this adds a third link that is also unset.
To actually restore drafting:
GEMINI_API_KEYis the one that matters right now: Anthropic is capped until2026-09-01 and the OpenAI key has no credits, so Gemini is the only link that
can currently answer. The boot log prints the resolved chain
(
model chain: anthropic -> openai -> gemini), so it is checkable after deploy.510 pass, 0 fail; format and typecheck clean.
🤖 Generated with Claude Code