feat(providers): stable MemBox prompt-cache boundary for Anthropic

[Kevin-K-W]

Summary

• Base branch: memorybox
• Add a MemBox wire marker (MEMBOX_PROMPT_CACHE_BOUNDARY_PREFIX) so the host can identify the stable current_user_request boundary during Anthropic request conversion.
• Place Anthropic cache_control on that boundary before volatile MemoryBox context, preserving the stable prefix through adjacent-user merging and follow-up turns.
• Retain incremental caching for a growing tool-loop tail while keeping volatile MemoryBox context uncached.
• Reallocate the redundant OAuth identity breakpoint and enforce Anthropic's limit of four explicit breakpoints, prioritizing tools, the full system prompt, the MemBox boundary, and then the incremental tail.
• Scope boundary: This PR changes only the shared marker helpers and Anthropic request projection. It does not change prompt text, memory retrieval, tool execution, other provider projections, configuration, or public CLI behavior.
• Blast radius: Anthropic API-key and OAuth chat/stream request serialization; MemBox callers that opt in by adding the marker. Unmarked conversations and non-Anthropic providers retain their existing behavior.
• Linked issue(s): N/A — this is the ZeroClaw side of the MemBox Anthropic cache integration.
• Labels: None currently applied on this internal memorybox-targeted PR.

Testing (required)

How you can test (when useful)

• Reviewer testing requested? Yes; a credentialed Anthropic follow-up validates the provider-reported cache metrics that unit tests cannot prove.
• Interface(s) exercised: Anthropic provider chat and streaming paths used by the MemBox host.
• Setup / preconditions: Pin MemBox to 43c5fc8, configure a valid Anthropic credential, and use a conversation containing stable current_user_request content followed by volatile MemoryBox context and at least one tool round.
• Steps to run: Send the initial request, a follow-up, and a tool-result continuation while recording Anthropic usage fields and the serialized request breakpoint locations.
• Expected on this branch (after): The marker is absent from provider-visible text; the stable request boundary receives cache_control; volatile MemoryBox context remains uncached; tool-loop tails receive an incremental breakpoint when a slot is available; the request contains no more than four explicit breakpoints; follow-ups report cache creation/read tokens when the prompt meets Anthropic's model-specific minimum.
• Prior behavior on memorybox (before): Cache placement on the changing tail invalidates the reusable request prefix; the first PR revision also suppressed incremental tool-loop tail caching whenever a MemBox boundary existed.

How I tested

• CI checks relied on and why they cover this change: GitHub's path-label and PR-title checks passed on the current head. This memorybox-targeted PR does not receive the repository's full Rust Quality Gate, so focused and full provider checks were run locally.
• Known CI coverage gap, if any: No credentialed Anthropic request was run, so cache_creation_input_tokens / cache_read_input_tokens and a live >20-content-block tool loop remain follow-up validation. The stable explicit boundary and breakpoint allocation are covered structurally by unit tests.
• Commands run and tail output:

cargo fmt --all -- --check
# passed

cargo clippy -p zeroclaw-providers --all-targets -- -D warnings
# Finished `dev` profile successfully

cargo test -p zeroclaw-providers cache --lib
# test result: ok. 49 passed; 0 failed; 0 ignored; 0 measured; 1190 filtered out

cargo test -p zeroclaw-providers --lib
# test result: ok. 1239 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

• Beyond CI, what did you manually verify? Reviewed both chat and stream_chat request construction, OAuth/system/tool/message breakpoint accounting, latest-boundary selection, adjacent-user merging, marker stripping, volatile-sibling exclusion, no-marker behavior, and the four-breakpoint priority policy. Live Anthropic cache usage was not verified.
• If any command was intentionally skipped, why: Live Anthropic traffic was skipped because this review environment does not provide a dedicated test credential.

Security & Privacy Impact (required)

• New permissions, capabilities, or file system access scope? No
• New external network calls? No
• Secrets / tokens / credentials handling changed? No
• PII, real identities, or personal data in diff, tests, fixtures, or docs? No
• Prompt injection or untrusted model-visible text introduced/changed? No
• Risk and mitigation: The marker is host-internal transport metadata and is stripped before Anthropic receives the message text. Breakpoint placement changes caching metadata only; it does not expand what prompt content is sent.

Compatibility (required)

• Backward compatible? Yes
• Config / env / CLI surface changed? No
• Rust/MSRV/toolchain floor changed? No
• Existing unmarked callers keep the prior last-message cache behavior; only callers that prepend the exported MemBox marker opt into the stable-boundary policy.

Rollback

• Risk: Low. No migration, persisted-state change, or user action is required.
• Rollback: Revert this PR and pin MemBox back to the previous ZeroClaw revision.

[sheperdh]

Review summary

The stable MemBox cache-boundary approach is structurally correct and should prevent volatile MemoryBox context from invalidating the reusable Anthropic prompt prefix. I did not find a response-correctness regression, but I recommend addressing one cache-performance regression and the validation gaps below before merging.

What looks correct

• MemBox marks only the native Anthropic current_user_request path.
• The wire marker is stripped before the request is sent to Anthropic.
• cache_control is placed on the latest marked request block, before the volatile MemoryBox context suffix.
• The breakpoint survives Anthropic's adjacent-user-message merging.
• Both chat and stream_chat use the new behavior.
• The non-Anthropic projection remains unchanged.

This matches Anthropic's guidance to place an explicit breakpoint on the last block whose prefix remains identical between requests.

Performance concern: last-message caching is disabled too broadly

apply_conversation_cache_control returns as soon as it finds the stable MemBox boundary. This disables the existing last-message cache for every marked request.

That matters during a long tool loop: the growing text / tool_result tail can no longer be cached incrementally, so each provider call may reprocess more input tokens and incur additional latency and cost.

Simply retaining the old last-message breakpoint is not sufficient because the MemBox OAuth path can already use four explicit breakpoints:

1. OAuth identity system block
2. Full system prompt
3. Last tool definition
4. Current-user stable boundary

Anthropic currently allows at most four explicit breakpoints. Adding the old tail breakpoint unconditionally could therefore produce five.

Suggested allocation

When a full system block follows the fixed OAuth identity block, the identity block's separate breakpoint is largely redundant: the full-system breakpoint already includes that prefix. I suggest freeing that slot and using this layout:

1. Last tool definition
2. Full system prompt, including the OAuth identity prefix
3. MemBox current-user stable boundary
4. Latest cacheable text or tool_result tail

If fewer components are present, allocate only the breakpoints that are needed. If no slot is available, prioritize the stable MemBox boundary over the tail cache.

Please add a test that asserts both the intended breakpoint placement and a maximum of four serialized breakpoints.

Validation gaps

The new unit test verifies content-block placement, but it does not demonstrate an actual cache read. The PR's live-cache test-plan item is also still unchecked.

Recommended coverage:

• Two real follow-up requests that assert cache_creation_input_tokens and cache_read_input_tokens.
• A multi-step tool loop showing that the stable prefix and incremental tail can both be reused.
• A conversation adding more than Anthropic's 20-block backward-lookback window between turns.
• No-marker behavior retaining the existing last-message cache.
• Serialized breakpoint count never exceeding four.

Local verification

• cargo test -p zeroclaw-providers cache --lib: 47 passed
• cargo test -p zeroclaw-providers --lib: 1235 passed
• MemBox prompt-prefix and Anthropic/non-Anthropic marker tests: 3 passed
• cargo fmt --all -- --check: failed on the three modified files

The PR checks currently visible are path labeling and PR-title validation; the full Rust Quality Gate is configured for PRs targeting master, so it did not run for this memorybox-targeted PR.

Recommendation

The core fix is sound, but I recommend updating the breakpoint-allocation policy, adding live/long-tool-loop evidence, and applying cargo fmt before merge.

[Kevin-K-W]

Thanks for the review — addressed in 43c5fc8:

Breakpoint allocation

• When a full system block follows the OAuth identity prefix, the identity block no longer takes its own cache_control (the full-system breakpoint already covers that prefix). Identity alone still gets a breakpoint when it is the only system block.
• With a MemBox stable boundary present, we still apply an incremental last-message cache on the tool-loop tail (text / tool_result), but we do not cache the volatile MemoryBox context sibling that shares the same user message as the boundary.
• After tools + OAuth + conversation cache are applied, we enforce Anthropic’s ≤4 explicit breakpoints and drop lowest-priority tail breakpoints first, preserving tools / system / MemBox boundary.

Tests

• Tool-loop tail still cached after MemBox boundary
• OAuth identity does not consume an extra slot when system follows
• OAuth + tools + MemBox boundary + tail stays at exactly 4
• No-marker path still retains last-message cache
• cargo fmt --all applied on the touched files

Live cache evidence (follow-up / MemBox-side): real cache_creation_input_tokens / cache_read_input_tokens assertions and a multi-step / >20-block lookback tool loop are not covered in this unit-test PR; happy to track those as a follow-up once we can run credentialed Anthropic traffic from CI or MemBox evals.

[sheperdh]

Review summary

Reviewed current head 43c5fc8. The updated implementation resolves the prior cache-performance concern without changing provider-visible prompt content or non-Anthropic behavior. I found no remaining code blocker.

✅ Resolved — tool-loop tail caching

The MemBox stable boundary no longer disables incremental last-message caching globally. A separate text or tool_result tail can receive its own breakpoint, while volatile MemoryBox context that shares the boundary message remains uncached.

✅ Resolved — four-breakpoint allocation

When a full system prompt follows the OAuth identity prefix, the identity block no longer consumes a redundant breakpoint. The normal tools + system + MemBox boundary + incremental-tail layout therefore fits Anthropic's four-breakpoint limit, and the final request-building step enforces that limit for both chat and streaming paths while dropping lower-priority tail breakpoints first.

✅ Resolved — regression and formatting coverage

The added tests cover the marked boundary, the unmarked fallback path, OAuth slot allocation, tool-loop tail caching, and the combined four-slot request. Formatting and strict provider Clippy checks now pass.

🟡 Warning — live cache metrics remain follow-up evidence

This review did not run credentialed Anthropic traffic, so provider-reported cache_creation_input_tokens / cache_read_input_tokens and a live >20-content-block tool loop remain unverified. This is non-blocking because the stable breakpoint is serialized directly at the unchanged boundary on every request, the incremental tail has independent structural coverage, and the live gap is now documented explicitly in the PR description.

🟢 What looks good — behavior remains scoped

The transport marker is stripped before provider delivery, only the latest marked request becomes the stable breakpoint, volatile context stays outside it, both Anthropic request paths share the policy, and unmarked/non-Anthropic flows remain unchanged.

Validation

• cargo fmt --all -- --check: passed
• cargo clippy -p zeroclaw-providers --all-targets -- -D warnings: passed
• cargo test -p zeroclaw-providers cache --lib: 49 passed
• cargo test -p zeroclaw-providers --lib: 1239 passed
• Current GitHub checks: passed; the memorybox base does not trigger the full Rust Quality Gate

Decision

Approved.