Repository navigation
🌐 feat: Translate Docs With Claude Haiku 5.5 and Batch Prefetch - #836
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be56f38ebb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Key order matters: OpenRouter rejects a body whose `requests` comes first. | ||
| const body = { | ||
| endpoint: '/v1/chat/completions', | ||
| model: `${opts.model}:batch`, |
There was a problem hiding this comment.
When the workflow uses --batch, this sends anthropic/claude-haiku-5.5:batch as the batch-level model. OpenRouter's Batch API documentation specifies the ordinary model slug here and selects its eligible :batch endpoint internally; supplying the variant suffix causes submission to be rejected. Because the surrounding catch silently falls back to live requests, scheduled runs will lose the intended discount and can spend the remaining job time translating the entire corpus live.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in a8b57ef. The batch-level model is now the plain slug (opts.model), which matches the batch quickstart; OpenRouter picks the :batch endpoint itself. The submit test asserts body.model === 'anthropic/claude-haiku-5.5' and that requests is serialized last.
| if (opts.dryRun) { | ||
| opts.onMiss?.({ text, locale, kind, context }) | ||
| stats.translatedBlocks++ | ||
| return text |
There was a problem hiding this comment.
When one file repeats the same source text in different contexts or kinds, this dry-run branch calls onMiss for every occurrence because it returns without adding a placeholder to ctx.staged. Those requests have different system prompts or context and therefore survive requestKey deduplication, but the real run stages the first translation by hashText(text) and never makes the later model calls. The extra batch rows are still processed and billed even though their answers cannot be consumed, which can multiply translation costs on repetitive pages.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed, finally in b901749. A dry-run dedupe alone (a8b57ef) still left two files processed concurrently each sending the same text with a different context. prefetchMisses now buys one answer per string per locale, keyed by locale and text (b894c50), and gives every occurrence its own copy, the same way the TM already shares translations by text. The test buys each string once per locale and serves every occurrence, so nothing runs live uses a frontmatter description that repeats a paragraph, plus a second page sharing that paragraph. It asserts one submitted request per distinct string and zero live calls.
Switch the default translation model from gemini-3.1-flash-lite to anthropic/claude-haiku-5.5, which lists at $0.10/$0.50 per million tokens. Drop the Google AI Studio provider pin and flex service tier, which no Anthropic endpoint can satisfy, stop sending temperature (Haiku 5.5 rejects non-default sampling values), and turn reasoning off so thinking tokens are not billed on every block.
Add a --batch mode to the translate script. A silent dry run collects every request the real run would send, those go to the model's :batch endpoint at half the live price, and the real run consumes each answer once before falling back to the live model for validation retries, truncation, and anything a batch did not answer within TRANSLATE_BATCH_WAIT_MINUTES. Batch-side failures never fail the run; they only cost the live price. The translate workflow now runs in batch mode.
…y-run misses OpenRouter selects the :batch endpoint from the plain model slug and rejects the suffixed one, which silently sent every request live. A dry run also reported a repeated source text once per occurrence, so batch rows were billed for repeats the real run serves from the staged map. Batch requests now carry the shared OpenRouter attribution headers.
OpenRouter cannot cancel a batch, so running its requests live after the wait billed them twice. Batches still running are now kept in a ledger in the translation cache; their blocks stay in English and uncached for this run, and the next run collects the answers instead of resubmitting them.
…s they are submitted Files processed concurrently can miss the same text before either commits it, with requests that differ only by context; the batch now buys one answer per string and serves a copy to every occurrence, as the TM already shares translations by text. The ledger is saved after each submit so a run that dies during the wait still hands its batches on.
… settings A retry now carries its attempt number, so the prefetched model serves only a block's first request and never hands back another copy of an answer that just failed validation. Batch size, wait, and poll interval fall back to their defaults unless positive, so a bad value cannot loop forever or poll without pause.
…them The ledger remembered each batch by the request of the string's first occurrence, whose context can differ from run to run, so a later run could resubmit a string a running batch was already answering.
…re no longer needed A run that died after collecting a batch but before caching its answers left nothing for the next run to find, so it paid for those strings again.
A submit or status call that never answered kept the wait loop from reaching its deadline.
…y the remaining wait A run scoped with --only or --locales dropped batches it did not need, so a later full run paid for them again; such entries now stay in the ledger past the 24h completion window. Each batch request is also capped by the time left in the wait so stalled calls cannot add up past it.
… be read Running with a model override wiped the ledger entries of the default model, and a batch whose status returned a permanent 4xx such as 403 stayed pending forever, keeping its strings untranslated on every run.
be56f38 to
488e4d3
Compare
📦 Next.js Bundle Analysis for librechat.aiThis analysis was generated by the Next.js Bundle Analysis action. 🤖 This PR introduced no changes to the JavaScript bundle! 🙌 |
This branch was successfully deployed
1 participant
Summary
Switches the docs translation model from Gemini 3.1 Flash Lite to Claude Haiku 5.5. At current OpenRouter pricing Haiku is cheaper on both input and output. Thinking is turned off and temperature is no longer sent, because Haiku 5.5 rejects non-default sampling values.
Also adds a
--batchmode, which the workflow now uses. A dry pass collects the uncached strings and buys one half-price batch answer per string per locale. Every occurrence of that string uses the answer on its first attempt; validation retries and failed batches run live. OpenRouter cannot cancel a batch, so one still running after the 180-minute wait is not paid for twice. Its strings stay in English for that run, andcontent/.i18n-cache/batches.json(persisted with the translation cache) lets the next run collect the answers.PROMPT_VERSIONis unchanged, so existing translations stay cached.Change Type
Testing
pnpm test: 434/434 passed. Batch tests cover once-per-string purchase across files and contexts, retries going live, deferral and collection on the next run, a ledger that survives a crash mid-wait, stalled requests, 403/404 polls, scoped runs, and model overrides.pnpm lint,pnpm lint:prettier,pnpm typecheck: clean.npx tsx scripts/translate.ts --dry-run --locales=de: no skips.batch: N occurrence(s) answered, and zh/ja/ko output should be spot checked before the translations branch is merged.Checklist