Repository navigation
Program 2: cache campaigns, cache-riding extraction, compaction cleanup, plugin and dashboard (umbrella) #470
Description
TL;DR. This is the second program for Context Guru, following #433 and #466. It covers smarter cache keep-alive, cheaper and more frequent LLM extraction that reuses the session's own cache, and removing compaction that deletes content by guesswork. It also redesigns the plugin and the dashboard so users can understand them. Production is never touched; changes ship as reviewed PRs.
Current best numbers
| what | value | class | source |
|---|---|---|---|
| keep-alive defaults (#444, merged) | ≈3.5–3.9% of the bill | counterfactual + live check | #446 |
| tool-declaration filter (#449, open) | ≈1.5–2.3% of the bill | counterfactual estimate | #465 |
| new items appear here as they pass independent evaluation |
Workstreams
- Cache lifecycle. Smarter cache keep-alive (research in progress in a private repository).
- Cache-riding extraction.
extract_llmruns on the session's own model and cached prefix, so each call is mostly cache reads. It is checked on SWE-bench Verified and Terminal-Bench 2 with ≥5 seeds. - Compaction cleanup. Remove
failed_runandcollapse, plus any other component that deletes content by heuristic rather than parking it reversibly. - Plugin and status line. Answers three questions: how much am I saving, is my cache about to cost me, what should I do.
- Dashboard. Redesigned with five pages, each starting with a bottom line.
Rules
Production is read-only: analysis uses snapshot copies, and experiments run on separate proxies and ports. Merging is allowed; deployment is left to the owners. Each number states its class (observed, counterfactual, oracle, benchmark), window, population and pricing basis. Detailed research results are kept outside this public repository until the owners approve publication.
Activity
OsherElhadad commentedon Oct 12, 2026
Progress update (2026-10-12)
Product PRs are open. Each needs a maintainer's approval to merge.
ready-to-merge.extract_llm(opt-in presethouse_ride). Reviewed,ready-to-merge. Benchmark arms are running./cg:*commands, cache countdown, Codex parity). Passed design review; code review in progress.The cache-lifecycle research is in progress in the private research repo. Benchmarks (SWE-bench Verified, Terminal-Bench 2, ≥5 seeds) are running on experimental proxies. Production has not been touched.