Skip to content

feat(config): add house2 preset (house + failed_run + long-line linecap), draft - #448

Closed
OsherElhadad wants to merge 1 commit into
mainfrom
feat/preset-house2
Closed

OsherElhadad wants to merge 1 commit into
mainfrom
feat/preset-house2

Conversation

@OsherElhadad

Copy link
Copy Markdown
Collaborator

Adds a house2 preset: house plus failed_run and linecap with only its long-line cap (lines over 2,000 characters; the duplicate-line collapse is off). Part of programme issue #434 (workstream A4/A5). DRAFT: live benchmarks are pending, so this must not become a default.

Why these two and nothing more. Candidates were screened against a bound set before measuring: the agent's next 6 actions must not need removed content (retained >= 98% of quoted lines), and every removal must be content-aware and reversible (stash plus <<cg:HASH>> marker).

Offline replay through /compact (85 SWE-shaped trajectories, 2,123 requests, 39 cold turns; counterfactual, input-side dollars at deployed gateway rates, trajectory bootstrap):

tokens removed (all / warm / cold) B vs off, % of input cost paired delta vs house
house 2.09 / 2.10 / 1.43 0.66 [0.36, 1.02]
house2 2.45 / 2.44 / 3.33 0.77 [0.48, 1.11] +0.11% [+0.04, +0.19]

Warm-turn cache busting caused by cg: 0. Format integrity failures: 0 of 72,182 requests across 34 arms. cg latency p50 +1.3 ms.

Left out, with the measured needed-info retention: mask 80.0%, collapse 74.4%, linecap duplicate-line collapse 77.1%, readlifecycle stale class 78.3% (and stale_at_depth busts the warm prefix on 111 requests). The two kept components showed no miss, but there are too few units (7) to certify the 98% bound, so treat the gain as small and unproven. searchcap would join after #439 merges.

Tests: go test ./config/ ./tenant/... ./components/all/...; new TestHouse2IsHousePlusFailedRunAndLongLineCapOnly pins the pipeline delta and the linecap settings. linecap sits immediately before cachesplit as the existing ordering test requires.

house plus the two reversible, content-aware offloaders that showed no
needed-info miss in the offline screen: failed_run and linecap's
long-line cap (duplicate-line collapse off). Offline replay: +0.11% of
input cost over house [+0.04, +0.19] paired, no warm-turn cache busting,
no format failures. Provisional: live benchmarks pending, not a default.

Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
@OsherElhadad

Copy link
Copy Markdown
Collaborator Author

Closing: this preset adds failed_run, which is removed in #474 (components may no longer delete by heuristic; see #471). The long-line cap it carried is removed there too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants