Repository navigation
feat(compaction): 以 history checkpoint 取代压缩覆盖层 - #1700
Merged
Merged
Conversation
2 of 6 tasks
ChrAlpha
force-pushed
the
feat/history-checkpoint
branch
from
October 28, 2026 09:37
350d00b to
7154862
Compare
ChrAlpha
force-pushed
the
feat/history-checkpoint
branch
from
October 28, 2026 09:54
7154862 to
c2466c4
Compare
2 of 6 tasks
ChrAlpha
marked this pull request as ready for review
October 28, 2026 15:23
chen-ran
force-pushed
the
feat/history-checkpoint
branch
from
October 29, 2026 08:58
c2466c4 to
3de5ef0
Compare
chen-ran
approved these changes
Oct 31, 2026
chen-ran
force-pushed
the
feat/history-checkpoint
branch
2 times, most recently
from
October 31, 2026 23:41
9ea6602 to
fc0bfd1
Compare
会话上下文改由 bot_sessions.history_checkpoint_id 指向的 checkpoint 表达: 一个 (turn_position, turn_message_seq) 位置,加上按序排列的条目(按引用保留的行 与摘要),请求先渲染这些条目,再渲染位置之后的上下文行。 - 0170 建表并把现有会话的有效摘要及其跨度内未覆盖的行转换为 checkpoint,同时 移除覆盖层;edit / retry / 删除在同一语句里沿父链回退指针,fork 在创建事务里 复制它所在轮次保留的 checkpoint,摘要引用的行改指 fork 的副本。 - 折叠引擎 compaction.Service.Fold 对 run 的请求做 cache-safe fork,沿用 run 的 输出设置,只读窗口在输出额度之外容得下的部分;一次 pass 只计算一次顺序 (turn.PlaceSummary),native runtime 与按行构造的请求都直接套用。 - 保留什么由引擎在第一个 pass 前按一个原文预算决定:请求余量的四分之一,至多 32000 tokens;下一次调用要回答的输入与没有行承载的消息无条件保留,其余 anchor 与最新的完整调用组按新到旧放入,放不下的随同一 pass 折叠。 - 按行读取以整个调用组为单位,调用不会与结果分在 checkpoint 两侧;按行构造的 请求持有原生消息,交给不带工具的压缩模型时写成文本。 - 字节预算截断的历史加载在构造请求之前先折叠,再加载一次;仍然截断则请求以 context.budget_unsatisfied 失败。 - 删除覆盖层的 coverage、epoch、claim、source hash、appeared rows 与修复路径, 以及仅为调用方不会传入的状态存在的守卫;保留的行加载不到、调用组无论怎么读 都放不进 fork 时直接失败。 - 未带 fence 的历史删除与 runtime decision 投影删除改为在 run 的 fence 内执行。
edit、retry、删除让会话沿父链回退后,退过的那段 checkpoint 依赖已被移除的行, 再也不会生效,也没有别处引用(fork 复制自己的一份),但一直留到清空历史或删除 会话才删,表随每次回退增长。memoh_rewind_history_checkpoint 现在在同一语句里 删掉保留节点之后的第一个节点,其余经 parent_id 级联删除。 折叠先读 parent、调模型、再提交;期间的回退会删掉它的 parent,插入就会撞外键。 CreateHistoryCheckpoint 改为只在 parent 仍是会话当前 checkpoint 时写入,并像 回退一样锁住会话行,否则返回无行,Commit 报 ErrMoved,与原来 CAS 失败时一样 归为 stale_source。 0170 尚未合并,直接修改其中的函数并同步 0001_init。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fork 事务先以 FOR SHARE 锁住源会话,插入新会话时才因外键对 bot 取 KEY SHARE;删除 bot 则先 FOR UPDATE 锁 bot、再锁它的会话。两者同时发生时 互相等待,PostgreSQL 以 deadlock detected 中止其中一个。fork 现在像其他会话 写入一样先经 LockBotForSessionWrite 锁 bot,遇到被锁的 bot 时在那里等待, 不占着源会话。 新增的集成测试在锁住 bot 时发起 fork,确认删除方仍能锁到源会话;去掉这次修改 时该测试以 40P01 失败。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chen-ran
force-pushed
the
feat/history-checkpoint
branch
from
November 1, 2026 00:17
fc0bfd1 to
5e122ec
Compare
fork 请求里超过 forkGroupCap 的调用组会被截到组上限,截断时把 reasoning 整块 丢掉。run 内折叠最常截断的正是最新一步,也就是 fork 里最后一个工具循环;fork 沿用 run 的推理设置,要求工具循环带回推理的 provider(开启 thinking 的 Anthropic 等)会因最后一条 assistant 缺少 thinking 块直接拒绝请求。 截断现在原样保留 reasoning(带签名的推理只接受原样回传,也无从截短),先从组 上限里扣掉它占的 token,再分给可截断的部分;reasoning 本身就超过上限时,该组 按放不进 fork 处理。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
折叠指令为每个被折叠的工具结果给出 ref=<行>:<part>,get_messages 按存储的原始行 回放后取该 part。读路径补写的关闭结果和从决定读出的结果沿用调用所在的 assistant 行,ref 会指向那一行的第 0 个 part,回读时报“不是工具结果”;读路径拆开的多结果 tool 行,part 序号也与存储的行对不上。 现在结果所在消息渲染自其调用所在的行,或该行被拆成多条消息时,不再给出 ref。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
折叠触发只看静态估算(按固定字节率折算 token)。分词更密的内容,例如大量结构化 JSON,估算远低于 provider 实际计数:实测 provider 报 323K(窗口的 123%),估算 只有约 193K,一次都没有折叠;对严格校验窗口的 provider,这样的请求会被直接拒绝, 而下一轮的估算仍然认为放得下。 armed run 现在用 provider 的计数校准估算:每一步记录对所发请求的估算,provider 回传该请求的 inputTokens 后,以二者之比(下限 1、上限 4)换算之后请求的额度和 折叠事件读取的窗口。估算写进该步 assistant 行的 metadata(input_estimate_tokens), 与 usage.inputTokens 放在一起;会话下一个使用同一模型的 run 从最近这样的一步开始, 第一个请求同样按 provider 的计法衡量。折叠日志在比例大于 1 时记录 density。 端到端测试让假 provider 按估算的约两倍计数:不回传 usage 时同样的 run 一次不折叠, 请求按 provider 计法超出窗口,下一轮第一个请求达到窗口的 128%;回传后每个请求都在 窗口内,下一轮在第一步之前就按读回的比例折叠。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chen-ran
force-pushed
the
feat/history-checkpoint
branch
from
November 1, 2026 00:58
5e122ec to
d4e9b14
Compare
ChrAlpha
added a commit
to ChrAlpha/Memoh
that referenced
this pull request
Nov 1, 2026
Resolve conflicts with felinics#1697 and felinics#1700 (history checkpoint): - tool_closure: keep converting each message once, and adopt main's sources: a closure renders from its call's row, a result read from a decision from the call, and moved results group by source. - continuation: provenance names the repaired outgoing messages. - Chat budget recovery and the compaction overlay fold are gone on main; drop the PR's edits to them and its "fold compacted history" check. Row decoding under the checkpoint also observes the request's end. - Provenance no longer carries a content hash, and records are matched by row ID, so MessageContentHash has no caller left: remove it, the step-version note, and the PostgreSQL test that guarded that version.
2 of 6 tasks
2 of 6 tasks
ChrAlpha
added a commit
to ChrAlpha/Memoh
that referenced
this pull request
Nov 1, 2026
…e-backoff origin/main replaced the compaction overlay with history checkpoints (felinics#1700). The overlay's MeasureUncompactedMessagesBySession and its doCompaction are gone; the new candidate read already counts only when it admits nothing, so the measure change is dropped. The per-session refusal memory now wraps the new Compact event, and its tests run on the checkpoint harness.
ChrAlpha
added a commit
to ChrAlpha/Memoh
that referenced
this pull request
Nov 1, 2026
felinics#1710 now carries origin/main with the history checkpoint redesign (felinics#1700). Conflicts resolve to felinics#1710's side. The continuation's search for the deferred tool call is dropped: the continuation now anchors on the run's turn request IDs from SQL and no longer scans records for it. The entry.go comment fix goes with the deleted file.
ChrAlpha
added a commit
to ChrAlpha/Memoh
that referenced
this pull request
Nov 1, 2026
ListHistoryMessagesAfterWithinBytes 与 ListHistoryMessagesByIDs 返回的 metadata 不再带 context_lifecycle:历史记录在解码后立即丢掉它,读它的 地方都直接读存储行。每个 assistant 行每轮都写一份摘要,旧行带着全部 selection decision,可达数 MB,此前每轮都要整段传输、解码,再随 source payload 哈希重新编码一次。覆盖层的匹配已随 felinics#1700 移除,source 哈希不再 有读者,所以这个 SQL 侧的裁剪不再受限。 集成测试确认两个查询都只返回其余字段,且两者一致。
ChrAlpha
added a commit
to ChrAlpha/Memoh
that referenced
this pull request
Nov 1, 2026
…reads felinics#1700 replaced the compaction overlay with history checkpoints, which supersedes both commits on this branch: MeasureActiveMessagesBySession and the covered turn-response ids no longer exist, and rows behind the checkpoint are never read. The merge takes main's side throughout; what is still warranted on the new design follows as separate commits.
2 of 6 tasks
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Author
Type
Summary
压缩原来以覆盖层(overlay)表达会话上下文:每条摘要记录自己覆盖了哪些行,读路径把有效摘要与未覆盖的行合成上下文,并靠 coverage、epoch、claim、source hash、appeared rows 和修复路径维持一致性。这些机制大多为当前 writer 不会产生的状态而存在,读写两侧都要理解它们。本 PR 改为 history checkpoint,并让轮前与轮内共用一个折叠引擎。
History checkpoint
bot_sessions.history_checkpoint_id指向的 checkpoint 决定:一个位置(turn_position, turn_message_seq),加上按序排列的条目(按引用保留的行与摘要,至少一份摘要)。请求先渲染这些条目,再渲染位置之后的上下文行。每次折叠提交一个新 checkpoint,以父链相连。memoh_rewind_history_checkpoint);run 的所有历史写入(含删除)都在 run 的 fence 内执行。ref=<行>:<part>)都改指 fork 的副本,fork 里的get_messages能按摘要的 ref 读回原始结果,fork 后的第一个请求不会再折叠一次源会话已折叠的内容。折叠引擎
compaction.Service.Fold对 run 即将发送的请求做 cache-safe fork,沿用 run 的 system、工具、cache key、推理与输出设置(run 不发max_tokens就不发),只读窗口在 run 输出额度之外容得下的部分,读不下的留给下一个 pass。provider 按 run 输出设定的参数(如 legacy Anthropic thinking 的budget_tokens)对 fork 同样成立。规划先算最长的读取,其余二分查找。compaction.Service.Compact用同一引擎从行折叠:字节预算截断的历史加载在构造请求前先折叠再重新加载,仍然截断就以context.budget_unsatisfied失败,不会以晚于 checkpoint 的起点静默继续;Discuss 的省略输入与组不下的批次同样走这里。按行读取以整个调用组为单位,调用不会与结果分在 checkpoint 两侧;交给不带工具的压缩模型时,消息写成文本。删除
覆盖层的一致性机制,以及只为调用方不会传入的状态存在的守卫。保留的行加载不到、调用组无论怎么读都放不进 fork,这两种情况直接失败,不再静默降级。
docs/agent-runtime.md、docs/database.md、docs/observability.md、docs/design/context-memory-scheduling.md已按新模型更新。Related Issues
Builds on #1697(stack,base 为 #1697 的分支)。Supersedes #1699 与 #1701:checkpoint 存储与折叠引擎作为一个单元交付;#1699 的按 step 限额被原文预算取代,最终 diff 不含其机制。
Validation
均在 head
7154862c6(或与其行为相同、标注了 head 的提交)上执行。go test ./... -count=1:除internal/container/docker与internal/workspace/shellenv外全部通过;这两个包在 main 上以相同用例失败(本机 Docker API 与 shell 环境)。go test -tags=integration -p 1 ./internal/db/... ./internal/chat/... ./cmd/agent/... ./internal/agent/... ./internal/channel/...,指向一个已迁移的独立 PostgreSQL 库:全部通过。golangci-lint run ./...:通过。go vet -tags=integration除internal/settings(main 上已有的编译错误,本 PR 未触及)外通过。migrate up,0170 用时 47.7ms,每个会话都得到 checkpoint。0170 此后未改动。1db378ebc,与当前 head 行为相同;本地devenv栈,Web 操作全部经浏览器,模型mimo-v2.6-pro,窗口 256k):Bot 用 47 分钟审计 88 个真实源码文件(1.03MB),单次输入峰值 217,113 tokens,同一 run 内自然触发 3 次压缩(199,027→47,662、201,322→59,248、217,113→49,565),每次之后继续原任务;期间两次需求变更、刷新、排队消息、Retry、Edit、Fork 均通过,一次性收据保持不变,交付物包含全部变更后的约束。早期精确数值在三次压缩后未能直接回忆,模型用 5 次只读命令从源码找回。ask_user变为 lost 且无法续跑(无压缩的对照会话同样复现);run 内折叠之后,上下文圆环显示的是 run 开始时记录的组成(本例 104%),而不是折叠后的请求;压缩日志展开后的common.usage未翻译。Screenshots / Recordings
截图由 Agent 通过浏览器自动化在运行中的应用里采集,不代表 Human QA。
Human QA