Skip to content

bug(replay): a text-only assistant turn without a thinking block is rejected upstream #110

Description

@chrisnestrud

Summary

With thinking enabled, the provider rejects any conversation whose assistant
turn carries no thinking block:

The `content[].thinking` in the thinking mode must be passed back to the API.

The adapter path only inserts thinking for assistant turns that contain
tool_use, so a conversation whose earlier assistant turn was plain text
reaches the provider bare and is rejected. Every retry re-sends the same body,
so the client session dies instead of recovering.

Root cause

prependCachedThinking in internal/service/server/adapter_dispatch.go skips
every assistant message that has no tool_use block:

// Only tool-call assistant messages require thinking replay fallback.
if !hasToolUse {
    continue
}

The plugin path already applies the rule to any assistant turn
(DSPlugin.PrependThinkingForAssistant -> state.PrependCachedForAssistantText
-> deepseekv4.PrependRequiredThinkingForAssistantText). The adapter path never
received that treatment when internal/bridge was removed in e436a33, so
text-only assistant turns stayed bare.

Evidence

Captured with trace.enabled, then replayed against the provider. The captured
body had messages[2] = assistant[text] with no thinking and
messages[3] = assistant[thinking(0), tool_use]:

variant result
captured body as-is 400
drop the thinking signature 400
fill the empty thinking text 400
add an empty thinking block to messages[2] 200
add a filled thinking block to messages[2] 200
drop messages[2] 200
remove the thinking parameter entirely 400

So every assistant turn needs a thinking block, an empty block is accepted, and
neither omitting the block nor dropping the parameter works.

Proposed fix

PR: #111. The adapter path applies the plugin's rule to every assistant
message: the tool-call cache first, then the assistant-text cache, then an empty
boundary block, with a warning when the fallback fires.

Verification

  • A four-turn tool-call probe that failed deterministically before the fix now
    completes.
  • Zero content[].thinking 400s across 185 streamed requests afterwards, where
    the failure was previously constant.
  • The new fallback warning fired 1124 times in that window, so the condition is
    routine rather than rare.
  • go test ./... is green, with new tests covering the text-only turn, an
    existing thinking block, and tool-call cache preference.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions