docs(openrouter-benchmarks): sync with openrouter-web changes - #164
docs(openrouter-benchmarks): sync with openrouter-web changes#164devin-ai-integration[bot] wants to merge 1 commit into
Conversation
…i/v1/benchmarks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Original prompt from Automation
|
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Summary
GET /api/v1/benchmarksnow also publishes OpenRouter's own search benchmarks (openrouter-web#33066), on the exact endpoint this skill teaches. The skill claimed the endpoint aggregates Artificial Analysis and Design Arena only, and its query-parameter tables omitted every new parameter, so an agent following it could neither request search results nor parse them.Documented, in SKILL.md and
references/benchmarks-api.md:sourceacceptsopenrouter;task_typeacceptssearch; newbenchmark_type(gpqa_diamond,tau_bench_verified_airline,search_browsecomp,search_hle,search_dsqa,search_widesearch), and the search-onlyinclude_run_config(defaultfalse),search_engine,search_surface(server-tool|plugin).accuracy:source=design-arena) returns an empty200, not a400. Search rows carry nopricingfield, so the existing per-token pricing guidance is now marked AA/DA-only.Auth and rate limits are unchanged, so those sections are untouched. Note: open PR #106 separately proposes documenting the classic
source=openrouter(GPQA/tau-bench) rows; this branch is based onmainand overlaps it textually on thesourceenum line.Link to Devin session: https://openrouter.devinenterprise.com/sessions/898c97dc4b6b45abb203df1a32f76143