Skip to content

TurnMonitor: detect stalled and degenerate turns inside the event bridge #1242

Description

@jayminwest

The gap

Between a turn starting and a turn ending, warren is blind. It cannot tell a model that is thinking from one that has stalled, and it cannot tell useful output from a reasoning stream that has degenerated into whitespace until the provider's length cap ends the turn. Two production incidents in one week:

  • A pi run on OpenRouter sat idle for 45 minutes inside one turn and burned $2.38 before the heartbeat watchdog fired.
  • A glm-5.3-flash model produced whitespace-only reasoning deltas until the length cap, on five runs in a row, each paid for in full.

Why warren cannot see either:

  • The event bridge receives every streamed delta but drops the per-delta types before persisting them. src/runs/stream/bridge.ts:206-209 continues on isPerDeltaNoiseEvent, and :454 lists message_update, tool_execution_update, and message_start as that set. The rationale at :418-424 is sound (persisting them flooded the event table), but no inspection happens before the drop.
  • The watchdog anchors on the newest persisted event (src/runs/watchdog.ts:172-178, repos.events.listTail(run.id, 1)), with a 45-minute default (:97). A turn that streams deltas for 44 minutes looks alive to it.
  • The spend cap reads usage only from turn_end (bridge.ts:238-242 into src/runs/usage-aggregate.ts:130-134), so cost accrued inside a turn is invisible until the turn ends.
  • The local provider has no in-process idle kill at all. The K8s pod has a 30-minute one (src/runtime/k8s/agent-stdin-hold.ts:196-200, default at agent-entrypoint-env.ts:104), re-armed on any child output, so a streaming spiral keeps it alive too.
  • Nothing inspects stopReason === "length". src/core/event-envelope.ts:110 already extracts stopReason from the nested message; no consumer acts on it.

The signals are all present on the wire. Every delta carries partial.usage.cost.total and its text, and whitespace-only thinking blocks pass the pi parser (src/runtime/adapters/parsers/pi.ts:153-157 drops only empty text).

What to build

A TurnMonitor in a new src/runs/stream/turn-monitor.ts, fed from the bridge at the drop site (before the continue at bridge.ts:206), that tracks three things per turn:

  1. Stall. Time since the last delta. Past a configurable budget (an env knob with a default well below the watchdog's 45 minutes, say 10 minutes), emit a turn.stalled event and cancel.
  2. Degeneration. A running count of consecutive thinking or text deltas whose content is whitespace-only. Past a threshold (by count or by bytes), emit turn.degenerate and cancel.
  3. In-turn spend. The cumulative partial.usage.cost.total, so the spend cap can trip mid-turn rather than at turn_end.

On a trip, follow the shape enforceBudgetCap uses in src/runs/stream/budget.ts:50-75: evaluate, persist what you know, emit the event, cancel the run through the same cancelBurrowRun seam, and break the bridge loop with a terminal outcome. The monitor must never persist deltas; src/runs/stream/bridge.test.ts:107 pins that they stay unpersisted and unpublished.

It works on both runtimes because the K8s provider feeds the same bridge through src/runs/stream/provider-source.ts. Keep the new file under 500 lines (check:size) and the bridge loop under the complexity ceiling; a separate module with an injected clock is the intended shape.

Scope

Out of scope: making a stalled or degenerate turn eligible for the automatic provider retry in src/runs/retry/provider-retry.ts. That needs a decision about which failure reason it carries and is a follow-up. If you add a new failure reason for the terminal outcome, it goes in RUN_FAILURE_REASONS at src/core/wire.ts:201 and nowhere else; check:wire-types enforces that, and gen:docs and gen:openapi must be re-run.

Tests: a new bridge.stall.test.ts beside bridge.oom.test.ts, driven with StreamEventView fixtures from src/runs/stream/test-helpers.ts (piTurnEnd at :100 is the turn-end builder; add a delta builder) and a fake clock. budget.test.ts is the template for the cancel-and-break assertions. The existing bridge.test.ts and watchdog.test.ts should pass unchanged.

Getting started

AGENTS.md covers setup and conventions. The bridge suites run in-process against fixture streams with no provider credentials. Run bun run check:all before pushing. Warnings count as failures.

Tracked internally as warren-74a7 (whitespace degeneration) and warren-1f85 (mid-turn stall). Their third original bullet, salvage of a timed_out run's work, already ships: the watchdog's force-fail reap runs salvage-before-destroy.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/runsRun lifecycle, events, streaming (src/runs/)effort/medium1-3 days of focused workgood first challengeLarger contributor-ready project: scoped, self-contained, gate-verifiedhelp wantedMaintainers welcome external contributions herepriority/P2Medium: default priority for normal feature worktype/featureNew user-facing capability

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions