Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
- **「未转义引号 + 尾部杂字符」叠在一起时围栏不再整条失守**:模型常在 JSON 根值之后追加 `</p>` 或一句解释(真实会话 539 条围栏里 240 条有尾随 `</p>`),这类正文本来靠 `parsePartialGenuiSpec` 的平衡前缀就能渲染;但一旦**同时**还有字符串内未转义的半角引号,tier-1 因为「必须整体 parse 通过」而放弃、tier-2 同样放弃——两个缺陷叠加就把整条围栏变成代码块 + 红横幅。现在 tier-2 在整体 parse 失败时回退到**平衡前缀**(根值结束处)并采用它;tier-1 保持严格(它也在流式期运行,采用前缀可能发布半截正文)。真实语料 37 条不可渲染 → 22 条(救回 15 条,占失败数 40%)。
- SKILL.md 使用规则第 3 条补充:`}` 之后不要再写任何字符(常见错误是追加 `</p>` 或解释)。
- **通用代码块的内容识别接受标点级修复**:宿主会隐去它不认识的语言(高亮器不支持 `dsh-ui`),此时同一份围栏在 DOM 里是一个「代码块 / Code block / Code」标签的普通代码块;当 ChatSnapshot 的语言来源在那一行上不可用时,内容识别是唯一出路。此前它要求 `JSON.parse(raw)` **直接**通过,于是「只差一个未转义引号(tier-1 可修)」的围栏永远不会被接管——真实会话(seq 40530)里模型终于把围栏写进正文、正文经 tier-1 修 14 处后可正常渲染,界面却始终停在代码块。现在内容识别跑与带标签路径**相同的 tier-1 修复**,并且在整体 parse 失败时**裁到第一个平衡根值**(`trimToBalancedRoot`,只裁剪、不补全结构)——真实样本里模型把工具调用模板泄漏在 JSON 之后且围栏没闭合,正是这一种;结构级 tier-2 刻意不参与(兜底不该接管只是"长得像 JSON"的普通代码),canonical 规范 / 未知字段 / 归一化等价的检查保持不变。
- **回合状态的两处漏补救**:① 失败的 `render_ui` 调用曾被算作「已交付」——只要本回合调用过 `render_ui`,回合边界就认定已回答、不再提醒,渲染失败的用户拿不到任何纠正。现在交付只由 `tool/result` 决定(无内部错误且结果块非 `isError` 才算成功),结果未到达时回合边界保持静默、不与晚到结果赛跑。② 同回合的合成上下文消息(`agent.inject()` 通知、成员消息等)会清掉本回合的验证/交付状态——它们走 `user/message` 但并非真实用户提示,回合边界检查时状态早已被清空。现在重置只发生在正式的 `turn/start` 边界与直接用户提示(`source.kind === 'user'`)上(#236 review 确认的两项)。

## [0.11.3] - 2026-09-29

Expand Down
104 changes: 79 additions & 25 deletions src/plugin/fence-feedback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -200,10 +200,17 @@ interface SessionFeedback {
*/
validatedThisTurn: boolean
/**
* The turn delivered something formal: a non-empty body text or a `render_ui`
* call. "No fence in the body" is NOT "no answer".
* The turn delivered something formal: a non-empty body text, or a
* `render_ui` call whose RESULT reports success. A failed render is not an
* answer, and a call whose result has not arrived yet proves nothing.
*/
deliveredThisTurn: boolean
/**
* `render_ui` calls of this turn whose result has not arrived yet. The
* result — not the call — decides delivery, so an outstanding call blocks
* steering at the boundary instead of being guessed about.
*/
pendingRenders: Set<string>
/** Fence fingerprints already corrected for a RENDER failure. */
correctedSpec: Set<string>
/**
Expand All @@ -229,7 +236,7 @@ export interface FenceFeedbackPlanInput {
readonly deliveryRemindedTurns?: ReadonlySet<number> | undefined
/** A `validate_dsh_ui` call happened this turn (formal GenUI signal). */
readonly validatedThisTurn?: boolean | undefined
/** The turn already delivered a body or a `render_ui` card. */
/** The turn already delivered a body or a successful `render_ui` result. */
readonly deliveredThisTurn?: boolean | undefined
readonly aborted: boolean
/** Corrections already steered in this turn (shared hard cap). */
Expand Down Expand Up @@ -257,7 +264,8 @@ export interface FenceFeedbackPlan {
* testable without a host.
*
* The decision is driven by FORMAL events only: fences in the reply body, a
* `validate_dsh_ui` call, a delivered body or `render_ui`. The reasoning block is
* `validate_dsh_ui` call, a delivered body text or a successful `render_ui`
* result. The reasoning block is
* never read here — a draft inside the thinking block is not proof that the model
* chose to deliver it, so it must not change any decision.
*
Expand Down Expand Up @@ -312,27 +320,27 @@ function textOfContent(content: unknown): string {
.join('\n')
}

/** Tool whose successful call IS a formal delivery. */
/** Tool whose SUCCESSFUL result is a formal delivery. */
const DELIVERY_TOOL = 'render_ui'

/**
* Whether one assistant message already delivered something formal.
* Whether one assistant message delivered a non-empty body text.
*
* Deliberately narrow: a non-empty text block, or a `render_ui` call. Other
* tools (validate_dsh_ui, bash, …) may succeed without producing any answer, so
* Deliberately narrow: a non-empty text block. A `render_ui` call is decided
* by its `tool/result`, not by the call appearing in a message — a call that
* failed (or whose result has not arrived) is not a delivery. Other tools
* (validate_dsh_ui, bash, …) may succeed without producing any answer, so
* they are NOT counted as delivery.
*
* @param content - assistant content blocks.
* @returns true when this message delivered a body or a UI.
* @returns true when this message delivered a body text.
*/
function deliveredSomething(content: unknown): boolean {
function deliveredBodyText(content: unknown): boolean {
if (!Array.isArray(content)) return false
return content.some(block => {
if (typeof block !== 'object' || block === null) return false
const record = block as { type?: unknown; text?: unknown; name?: unknown }
if (record.type === 'text') return typeof record.text === 'string' && record.text.trim() !== ''
if (record.type === 'tool-call') return record.name === DELIVERY_TOOL
return false
const record = block as { type?: unknown; text?: unknown }
return record.type === 'text' && typeof record.text === 'string' && record.text.trim() !== ''
})
}

Expand Down Expand Up @@ -378,6 +386,7 @@ export function installFenceFeedback(ctx: Context, enabled: boolean): void {
text: '',
validatedThisTurn: false,
deliveredThisTurn: false,
pendingRenders: new Set(),
correctedSpec: new Set(),
deliveryRemindedTurns: new Set(),
correctionsThisTurn: 0,
Expand All @@ -392,24 +401,64 @@ export function installFenceFeedback(ctx: Context, enabled: boolean): void {
sessions.delete(String(session.id))
})

const resetTurnState = (state: SessionFeedback): void => {
state.text = ''
state.validatedThisTurn = false
state.deliveredThisTurn = false
state.pendingRenders.clear()
}

ctx.on('session/event', (session, event: SessionEvent) => {
const sessionId = String(session.id)
if (event.type === 'turn/start') {
// The formal turn boundary: whatever happened in the previous turn is
// settled and this turn starts clean. The `user/message` reset below is
// only a fallback for direct prompts.
const state = sessions.get(sessionId)
if (state !== undefined) resetTurnState(state)
return
}
if (event.type === 'assistant/message') {
const content = (event.data as { message?: { content?: unknown } }).message?.content
const text = textOfContent(content)
const state = stateOf(sessionId)
// Fences are read from the BODY only: a draft in the reasoning block is not
// a delivery and must not become one.
state.text = extractDshUiFences(text).length > 0 ? text : ''
// Delivery is sticky for the turn: once a body or a render_ui card exists,
// the turn has answered and must never be corrected into publishing again.
state.deliveredThisTurn = state.deliveredThisTurn || deliveredSomething(content)
// Delivery is sticky for the turn: once a body text exists, the turn has
// answered and must never be corrected into publishing again. A
// `render_ui` call in the content proves nothing here — its result does.
state.deliveredThisTurn = state.deliveredThisTurn || deliveredBodyText(content)
return
}
if (event.type === 'tool/call') {
const data = event.data as { name?: unknown; callId?: unknown }
const state = stateOf(sessionId)
// validate_dsh_ui is the FORMAL signal that this turn is GenUI-related.
const name = (event.data as { name?: unknown }).name
if (name === 'validate_dsh_ui') stateOf(sessionId).validatedThisTurn = true
if (data.name === 'validate_dsh_ui') state.validatedThisTurn = true
// The result — not the call — decides whether render_ui delivered.
if (data.name === DELIVERY_TOOL && typeof data.callId === 'string') {
state.pendingRenders.add(data.callId)
}
return
}
if (event.type === 'tool/result') {
const data = event.data as {
message?: { content?: unknown }
error?: unknown
}
const content = data.message?.content
const block = Array.isArray(content)
? content.find(part => typeof part === 'object' && part !== null && (part as { type?: unknown }).type === 'tool-result') as { toolCallId?: unknown; isError?: unknown } | undefined
: undefined
const callId = typeof block?.toolCallId === 'string' ? block.toolCallId : undefined
if (callId === undefined || block === undefined) return
const state = stateOf(sessionId)
if (!state.pendingRenders.delete(callId)) return
// Success = no internal failure identity AND the model-facing block is
// not an error. Anything else leaves the turn undelivered so the
// boundary can remind (a failed card is not an answer).
if (data.error === undefined && block.isError !== true) state.deliveredThisTurn = true
return
}
if (event.type !== 'user/message') return
Expand All @@ -423,13 +472,14 @@ export function installFenceFeedback(ctx: Context, enabled: boolean): void {
for (const fingerprint of fingerprints) state.correctedSpec.add(fingerprint)
return
}
// A genuine user prompt starts a new turn: the previous reply is settled.
// A DIRECT human prompt starts a new turn: the previous reply is settled.
// Synthetic context that rides the same user surface — `agent.inject()`
// notices, team member messages — carries a different source kind and
// arrives MID-TURN: resetting here used to wipe the turn's validation and
// delivery state before the boundary could use it.
if (data.source?.kind !== 'user') return
const state = sessions.get(sessionId)
if (state !== undefined) {
state.text = ''
state.validatedThisTurn = false
state.deliveredThisTurn = false
}
if (state !== undefined) resetTurnState(state)
})

ctx.on('agent/turn-stopping', ({ agent, turn, signal }): void => {
Expand All @@ -438,6 +488,10 @@ export function installFenceFeedback(ctx: Context, enabled: boolean): void {
if (agent.session.header.parentSession !== undefined) return
const state = sessions.get(String(agent.session.id))
if (state === undefined) return
// A render_ui result still outstanding could deliver (or fail) after this
// boundary fires: steering now would race the late result, so stay silent
// and let the next boundary decide on settled facts.
if (state.pendingRenders.size > 0) return
const usedThisTurn = state.correctionsTurn === turn ? state.correctionsThisTurn : 0
const plan = planFenceFeedback({
text: state.text,
Expand Down
93 changes: 90 additions & 3 deletions tests/fence-feedback.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,24 @@ describe('installFenceFeedback wiring', () => {
expect(h.steer).toHaveBeenCalledTimes(1)
})

const toolCallEvent = (name: string): unknown => ({ type: 'tool/call', seq: 4, time: 2, data: { name } }) as unknown as SessionEvent
const toolCallEvent = (name: string, callId?: string): unknown =>
({ type: 'tool/call', seq: 4, time: 2, data: callId === undefined ? { name } : { name, callId } }) as unknown as SessionEvent
const toolResultEvent = (callId: string, failure?: { internal?: boolean; blockError?: boolean }): unknown => ({
type: 'tool/result',
seq: 6,
time: 3,
data: {
message: { content: [{ type: 'tool-result', toolCallId: callId, isError: failure?.blockError === true }] },
...(failure?.internal === true ? { error: { name: 'ToolError', code: 'render_failed' } } : {}),
},
}) as unknown as SessionEvent
const contextMessageEvent = (kind: string): unknown => ({
type: 'user/message',
seq: 8,
time: 4,
data: { content: [{ type: 'text', text: '文件已变更' }], source: { kind } },
}) as unknown as SessionEvent
const turnStartEvent = (turn: number): unknown => ({ type: 'turn/start', seq: 1, time: 0, data: { turn } }) as unknown as SessionEvent
const reasoningEvent = (text: string): unknown => ({
type: 'assistant/message',
seq: 5,
Expand Down Expand Up @@ -360,7 +377,7 @@ describe('installFenceFeedback wiring', () => {
expect(decisions).toEqual([1, 1, 1])
})

it('stays silent when the turn delivered a body or a render_ui card', () => {
it('stays silent when the turn delivered a body or a successful render_ui result', () => {
const delivered = harness()
delivered.emitSession(userEvent())
delivered.emitSession(toolCallEvent('validate_dsh_ui'))
Expand All @@ -369,14 +386,84 @@ describe('installFenceFeedback wiring', () => {
delivered.boundary({ agent, turn: 8, signal: new AbortController().signal })
expect(delivered.steer).not.toHaveBeenCalled()

// render_ui 只由它的 tool/result 决定交付:结果成功才算(此前仅凭调用就算,
// 失败的渲染卡也被当成已交付 — review 确认的漏补救之一)。
const card = harness()
card.emitSession(userEvent())
card.emitSession(toolCallEvent('render_ui'))
card.emitSession(toolCallEvent('validate_dsh_ui'))
card.emitSession(toolCallEvent('render_ui', 'render-1'))
card.emitSession({ type: 'assistant/message', seq: 7, time: 4, data: { message: { content: [{ type: 'tool-call', name: 'render_ui', arguments: '{}' }] } } } as unknown as SessionEvent)
card.emitSession(toolResultEvent('render-1'))
card.boundary({ agent: { session: { id: 'sess-1', header: { id: 'sess-1', version: 4 } }, steer: card.steer }, turn: 9, signal: new AbortController().signal })
expect(card.steer).not.toHaveBeenCalled()
})

it('does not count a FAILED render_ui result as delivery', () => {
for (const failure of [{ internal: true }, { blockError: true }]) {
const h = harness()
h.emitSession(userEvent())
h.emitSession(toolCallEvent('validate_dsh_ui'))
h.emitSession(toolCallEvent('render_ui', 'render-fail'))
h.emitSession(toolResultEvent('render-fail', failure))
const agent = { session: { id: 'sess-1', header: { id: 'sess-1', version: 4 } }, steer: h.steer }
h.boundary({ agent, turn: 5, signal: new AbortController().signal })
expect(h.steer).toHaveBeenCalledTimes(1)
const message = h.steer.mock.calls[0]![0] as { content: Array<{ text: string }> }
expect(message.content[0]!.text).toContain('status=nothing_delivered')
}
})

it('stays silent while a render_ui result is outstanding, then decides on the result', () => {
const h = harness()
h.emitSession(userEvent())
h.emitSession(toolCallEvent('validate_dsh_ui'))
h.emitSession(toolCallEvent('render_ui', 'render-late'))
const agent = { session: { id: 'sess-1', header: { id: 'sess-1', version: 4 } }, steer: h.steer }
// Result not arrived yet: steering here would race the late result.
h.boundary({ agent, turn: 6, signal: new AbortController().signal })
expect(h.steer).not.toHaveBeenCalled()
// The result settles as failed: the next boundary can remind.
h.emitSession(toolResultEvent('render-late', { blockError: true }))
h.boundary({ agent, turn: 6, signal: new AbortController().signal })
expect(h.steer).toHaveBeenCalledTimes(1)
const message = h.steer.mock.calls[0]![0] as { content: Array<{ text: string }> }
expect(message.content[0]!.text).toContain('status=nothing_delivered')
})

it('does not let synthetic same-turn context messages clear the turn state', () => {
// agent.inject() 通知、成员消息等同面上下文走 user/message 但 source.kind
// 不是 'user':它们到达于回合中途,不得清掉验证/交付状态(review 确认的
// 漏补救之二)。真实用户提示(kind 'user')仍开启干净的新回合。
for (const kind of ['tool', 'agent', 'member']) {
const h = harness()
h.emitSession(userEvent())
h.emitSession(toolCallEvent('validate_dsh_ui'))
h.emitSession(contextMessageEvent(kind))
const agent = { session: { id: 'sess-1', header: { id: 'sess-1', version: 4 } }, steer: h.steer }
h.boundary({ agent, turn: 3, signal: new AbortController().signal })
expect(h.steer).toHaveBeenCalledTimes(1)
const message = h.steer.mock.calls[0]![0] as { content: Array<{ text: string }> }
expect(message.content[0]!.text).toContain('status=nothing_delivered')
}
})

it('resets the turn state on the formal turn/start boundary', () => {
const h = harness()
h.emitSession(turnStartEvent(1))
h.emitSession(userEvent())
h.emitSession(toolCallEvent('validate_dsh_ui'))
const agent = { session: { id: 'sess-1', header: { id: 'sess-1', version: 4 } }, steer: h.steer }
h.boundary({ agent, turn: 1, signal: new AbortController().signal })
expect(h.steer).toHaveBeenCalledTimes(1)
// New turn: turn/start wiped the state, so a fresh validate + nothing
// delivered can be reminded again under a new turn number.
h.emitSession(turnStartEvent(2))
h.emitSession(userEvent())
h.emitSession(toolCallEvent('validate_dsh_ui'))
h.boundary({ agent, turn: 2, signal: new AbortController().signal })
expect(h.steer).toHaveBeenCalledTimes(2)
})

it('keeps the delivery ledger separate from the render-failure ledger', () => {
const h = harness()
h.emitSession(userEvent())
Expand Down
Loading