Revisit memory middleware cache boundaries across model providers

[open-swe]

Filed on behalf of Sydney Runkle (@sydney-runkle), at her request, following a discussion with the Managed Deep Agents (MDA) team.

Area

• deepagents (SDK)

Issue Content

Revisit the prompt-cache boundary in MemoryMiddleware and how it composes with provider-aware prompt caching. MDA's proposed memory middleware copies Deep Agents' Anthropic-only cache-control logic and emits ordinary text for other providers. That avoids sending Anthropic markers to unsupported providers, but does not settle caching behavior for the other models developers can select.

MDA has shared agent memory and private user memory, with per-run allow callbacks deciding which layers are mounted. Its proposed loader refreshes content before each model call and injects it into the request without checkpointing a memory_contents field. Content can therefore differ between callers/runs and after writes within a run.

Questions / desired outcome

• Evaluate cache boundaries around stable instructions versus changing memory. Quantify cache reads/writes and cost rather than assuming caching memory is always good or always bad.
• Review middleware ordering and interaction with other cache markers, including Anthropic breakpoint limits and TTL configuration.
• Account for provider differences: Anthropic explicit markers, OpenAI automatic prefix caching, and Gemini's distinct explicit-cache lifecycle. Unsupported providers should continue to receive valid requests.
• Decide which behavior belongs in a generalized caching middleware versus memory-specific assembly; document supported providers and no-op behavior clearly.
• Test unchanged memory, edited memory, switched callers, and denied/removed layers. Cache decisions must not bypass current-run memory authorization.

Related work

• Generalize caching #3239 — Generalize caching. This issue is the memory-specific behavior/design follow-up, not just the rename/dispatch scaffolding described there.
• MemoryMiddleware: add_cache_control=True (0.6) puts incorrect cache_control marker on volatile memory block #3639 — prior closed report about memory cache-marker placement; revisit the tradeoffs without assuming all claims in that report are established.
• MemoryMiddleware keeps stale memory on checkpointed threads #6122 — memory refresh policy.