Repository navigation
feat: add Tau terminal harness - #6493
nikhilachale wants to merge 32 commits into
Conversation
🏆 Review leaderboardOct 2–9, 2026 · UTC
Ranked by distinct external PRs reviewed, then review rounds, then PR comments. Self-activity and bot activity are excluded. Show 5 more reviewers
|
|
Evidence correction from the skill packaging review: the historical Tau 0.4.7 — VPS audit report, 2026-10-10 Functional lifecycle: 17/17 PASS. CI on the audited commit: 24/24 PASS. Overall audit: BLOCKED. This is diagnostic lifecycle coverage; a full strict audit PASS is not claimed. Audited commit: Gate totals: 25 PASS, 2 NOT_RUN, 1 BLOCKED. These are gate counts, separate from the 24 CI checks. Local CLI
AO / AOS11
Cross-check
AO reports credentials as configured, not provider-verified. Real Z.ai calls succeeded, so this metadata gap is not evidence of an expired login. Tau's provider TSV must not be counted as one model ID per row; native model listing/exact catalog comparison remain NOT_RUN. The live audit reproduced an ANSI-split composer marker causing a false startup timeout. The readiness fix strips ANSI before matching. VPS regression, session-manager/Tau race suites and pinned released-TUI conformance passed. Earlier renamed-binary hook-path failures and premature post-restore sends were test setup/runner failures; they are preserved separately. Final attempt 6 passed after using the canonical AO executable and waiting for the native composer. The original strict skill remains unchanged. The separate diagnostic runner continues lifecycle testing only after successful direct native proof and AO installation/freshness checks; it retains the original non-passing auth/model gates. The runner revision used here passed 80/80 fixture tests; subsequent generic improvements passed 91/91. Audited-head backend/aggregate CI · Pinned released-native conformance Cleanup confirmed: the created session was terminated and the isolated daemon/native PTY processes stopped. A final audit-wide process scan found no remaining owned test processes. Credentials and raw native IDs are omitted here. Previous failed attempts and controlled comparisons remain preserved in the audit bundle. Real terminal text was captured; screenshots, Electron UI and native non-Linux harness coverage are not claimed. Evidence identity
|
|
Evidence correction from the skill packaging review: the historical Tau 0.4.7 — failed-attempt report, 2026-10-10 The original failures remain preserved even though the final diagnostic run passed all 17 lifecycle gates. The full strict audit remains BLOCKED: AO reports configured credentials, and native model discovery/comparison are NOT_RUN. See the final gate-by-gate report. All times below are UTC on 2026-10-10. Testing ran on the Ubuntu VPS, with real Z.ai calls; no local Mac checks were run.
Representative original error: The original native marker was Remaining gaps: configured credentials are authorized for this audit but still not classified as provider-verified by AO; this is not an expired-login finding. Local model listing and exact catalog comparison were not executed with a valid contract. Screenshot coverage: the historical attempts have real terminal text/ANSI captures, but no PNG screenshots. No rendered log or recreated error image is claimed as an original screenshot. Evidence paths above are relative to the preserved VPS report directory |
|
The reusable integration/testing skill and consolidated session report are now in #6509.
The report includes two redacted, actual screenshots from the new Open Interpreter early-send reproduction; they are published repository evidence, not historical screenshots of Tau or Neovate. Original captures remain unchanged. The historical instruction-refresh correction remains explicit: the old |
|
Fresh current-runner follow-up, all computation and capture on the authorized VPS. Tau 0.4.7 / Z.ai glm-5.3-flash / TUI / explicit bypass only: tau-002 finished at 2026-10-10T18:30:55.127Z with 25 PASS, 0 FAIL, 1 BLOCKED, 2 NOT_RUN. Functional lifecycle passes; strict overall BLOCKED / diagnostic-only. Auth remains configured, not verified. Native model listing and native/AO comparison remain NOT_RUN. No login, credential changes, or contract relaxation. The fresh restore-only instruction was introduced through the project API after confirmed kill, read back, then consumed in the restored mutation. Same native ID/workspace, initialized empty composer, model/workspace cue, post-restore mutation and history continuity passed. Native digest Preserved tau-001: 21 PASS, 4 FAIL, 1 BLOCKED, 2 NOT_RUN. Provider-authored JSON acquired an extra closing brace on the second mutation; second_message, post_restore_message, history_file_continuity and system_prompt_restore failed. No file repair; fresh retry used identical source/pin/profile/contract. Original gate-time screenshot unavailable (API audit preceded GUI); the first image below is a later retained-session capture with its own timestamp. Historical failures remain unchanged. Historical 17/17 labels did not prove fresh instructions; native mirrors did not prove real AO UI. Tested functional source Actual AO Electron, native bridge confirmed, exact target session selected, screenshots inspected for secrets/relevance. Supervisor reopened the same isolated data with the identical verified binary after stopping finalized runner; Tau-002 native digest and restored generation reconfirmed. Tau-001 records retained-session recovery and audit identity, but no explicit capture-time digest/generation recheck; it is failure-state UI evidence. PNGs are actual scrot/Xvfb captures, not terminal mirrors. Failure capture UTC Success capture UTC Final evidence-head CI verified at |
nikhilachale
left a comment
There was a problem hiding this comment.
[P2] Fence composer readiness to the current Tau launch
PromptReadinessHints uses the visible Ask Tau… marker, but waitForPromptReadiness obtains it from Runtime.GetOutput; the tmux implementation captures the last 80 lines with capture-pane -S, and Runtime.Restart explicitly preserves terminal history. When AO performs a fresh fallback/recovery on an existing Tau handle—for example, a live/exited pane whose native session ID was never captured—the prior Tau composer marker is still in scrollback. The very first poll can therefore report ready before the replacement Tau process has mounted its composer, and deliverAfterStartPrompt can paste the saved task into the initializing process, where it may be dropped.
Please fence readiness to the current runtime launch (for example, require the current-generation session_start signal promised by FirstSignalProvesInputReady in addition to the marker), or baseline/clear old pane output before matching. A focused regression test should return stale Ask Tau… output immediately after restart and prove delivery waits for current-launch readiness.
|
Harness-specific reports, JSON receipts, screenshots and provenance now live on this integration PR branch, rather than the skill PR #6509. Evidence index with embedded screenshots and JSON links. Historical successes, failures and scope limitations remain as recorded. Moving these files does not rerun native/AO testing or upgrade any verdict. This is evidence relocation only. Existing tested source/native hashes and failures are preserved; the documentation commit is not a fresh runtime pass. Applicable checks run in PR CI. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Canonical main conflicts reconciled at Validation is PR CI only; current-head checks are pending until GitHub reports them. This merge does not constitute a new native/AO run or promote historical fresh/restore/provider-role proof. Harness evidence stays on this integration PR; PR6509 remains reusable guidance and linked summaries. |
|
Current Tau audit evidence is on this integration branch at Fresh private/project/user controls, executing Escape, independent AO Kill with live unrelated control, and missing-history guard passed in their exact recorded source/model scope. Restored consumption failed on both an update and separate new-file task despite configured NEW append. Earlier Flash/helper/input/history failures remain visible. Runtime binary is |
Build the real daemon in PR CI for isolated VPS runtime verification without local validation suites or dependency installations. Record source revision, Go build info, and executable hash.
6e788d0 to
1670989
Compare
|
Fresh testing-only retest starting for |
|
Tau bounded current-head retest — BLOCKED, with CI FAIL. No full integration PASS. Observed 2026-10-21T07:12:35.781155+00:00; Linux amd64. Tested/current PR head AO UI attachment evidence: BLOCKED. No actual Electron captures and no approved uploader. The full checkpoint index below is intentionally incomplete. PR-description link to this new evidence comment is pending parent action. Runtime identity and actual attempts:
GOMAXPROCS=1 GOFLAGS=-p=1 GOMEMLIMIT=384MiB go build -o /runtime/bin/ao ./cmd/ao
# Use the isolated native launcher and minimal copied configuration; do not load a denied key route.
TMPDIR=/runtime/tmp PATH=/runtime/bin:$PATH NODE_OPTIONS=--max-old-space-size=384 \
node /pinned-skill/scripts/audit_ao.mjs --agent tau --repo /clean-tested-source \
--ao /runtime/bin/ao --local-contract /runtime/contract-strict.json \
--report /runtime/attempts/strict-001/report --startup-timeout-seconds 60 --timeout-seconds 30Contract original-byte SHA256 {
"schemaVersion": 1,
"agents": {
"tau": {
"binary": "/runtime/bin/tau",
"versionArgs": [
"--version"
],
"integration": {
"args": [
"providers"
],
"expect": "zai-audit"
},
"models": {
"args": [
"__audit-models"
],
"format": "json"
},
"aoModelsPath": "agents/{agent}/models",
"interrupt": {
"input": "\u001b",
"settleTimeoutMs": 5000
}
}
}
}Completion matrix:
|
|
Tau bounded current-head retest — BLOCKED, with CI FAIL. No full integration PASS. Observed 2026-10-21T07:12:35.781155+00:00; Linux amd64. Tested/current PR head AO UI attachment evidence: BLOCKED. No actual Electron captures and no approved uploader. The full checkpoint index below is intentionally incomplete. PR-description link to this new evidence comment is pending parent action. Runtime identity and actual attempts:
GOMAXPROCS=1 GOFLAGS=-p=1 GOMEMLIMIT=384MiB go build -o /runtime/bin/ao ./cmd/ao
# Use the isolated native launcher and minimal copied configuration; do not load a denied key route.
TMPDIR=/runtime/tmp PATH=/runtime/bin:$PATH NODE_OPTIONS=--max-old-space-size=384 \
node /pinned-skill/scripts/audit_ao.mjs --agent tau --repo /clean-tested-source \
--ao /runtime/bin/ao --local-contract /runtime/contract-strict.json \
--report /runtime/attempts/strict-001/report --startup-timeout-seconds 60 --timeout-seconds 30Contract original-byte SHA256 {
"schemaVersion": 1,
"agents": {
"tau": {
"binary": "/runtime/bin/tau",
"versionArgs": [
"--version"
],
"integration": {
"args": [
"providers"
],
"expect": "zai-audit"
},
"models": {
"args": [
"__audit-models"
],
"format": "json"
},
"aoModelsPath": "agents/{agent}/models",
"interrupt": {
"input": "<single ESC byte 0x1b>",
"settleTimeoutMs": 5000
}
}
}
}Completion matrix:
Publication receipt: the earlier new evidence comment is retained intact. Its readback transformed the textual Unicode Escape sequence in the contract snippet. This final comment describes the byte symbolically: construct the actual runtime |
Tau's Account row only offered View documentation. Open tau and inject /login once the composer renders, which opens Tau's native login-method picker (subscription OAuth, API key, or custom provider). Add Tau's official mark (white tau on black, from twotimespi.dev) so session and harness surfaces stop falling back to a lettered tile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tau sessions failed to spawn under AO's default permission mode because the adapter rejected anything but bypass-permissions. Tau has no tool approval policy or permission flags, so handle it like Pi: emit no permission flag for any mode and defer to Tau's own behavior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The adapter accepted exactly tau-ai 0.4.7, so any upgrade, including AO's own managed update to 0.4.8, made every Tau launch fail. Treat 0.4.7 as a minimum in launch, restore and ao doctor through one tau.CheckVersion, and install the latest tau-ai. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
List codewhale and tau in ao spawn --harness help, and declare Tau's terminal version refresh capability (unsupported: restore needs native history and is not yet audited for unattended resume). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Continuation under the new contributor accountThis harness submission has been copied to PR #8210 under @milesatao-svg as part of the contributor's account migration. The replacement draft preserves source head |
|
Closing this original harness submission as superseded by PR #8210 under my new contributor account @milesatao-svg. Copy verification confirmed the same original head The replacement is a draft. Main merge conflicts and fresh CI validation are pending. |






Code changes: +751 -29
Tests: +1415 -6
Others: +5719 -3
Binary assets in the current PR diff: 14 (excluded from line counts).
Adds the pinned Tau terminal harness with native appended private instructions, preserved native/user/project rules, model selection and isolated lifecycle hooks. Chat, reviewers and cross-interface handoff are unsupported; explicit bypass permissions are required.
The actual AO audit exposed and fixed two defects: long multiline messages now reach Tau as one bracketed paste through the final admitted guard write, and exact restore now rejects missing/empty/corrupt/ambiguous/wrong-cwd native history instead of accepting metadata alone. Released-native CI exercises both boundaries. A fake-profile environment test and lint style issues found by CI are corrected for the new full run.
Draft: mixed live result. Current CI artifact
e1ef49b12712b6c5fc17e595f931bc28296dd66fpassed a GLM-4.7 ordinary fresh task with private/project/user actions, cwd, initial once and AGENTS preservation; executing-shell Escape, independent AO Kill/control isolation and missing-history rejection also passed. Exact native identity and new runtime generation restored, with NEW managed instructions confirmed by the supported read-only native SDK, but two ordinary restored tasks left the OLD private token: restored consumption FAIL. Current GLM-5.3-Flash partial fresh failure and earlier failures remain separate. The positive control changes task length/model; it does not identify the sole cause or erase failures.Harness-owned audit report and real AO screenshots. Runtime source, native wheel/118-file fingerprint, serving PID/executable SHA, native-ID digest, generation and cleanup are recorded there. UI source is separately
3e80348c; later fixture/switch-style/evidence commits never relabel the old runtime binary. Controlled native/source payload role issystem; authenticated third-party wire role was NOT_CAPTURED.Validation is GitHub PR CI only; no routine VPS/Mac suites ran. All applicable current-head checks must finish before readiness. No per-harness report/JSON/PNG is added to PR6509.
2026-10-21 update: merged upstream main
7c83bd68c1e60c85b6a90e3ffa1340c859e4a3ffinf2c85a89f7. Resolved the shared harness registrations and launch-environment changes, moved the unmerged Tau migration to timestamp20261021090000000, and regenerated the API artifacts. The daemon build and focused merge/migration regressions pass locally. Full local validation is not green: installed OpenCode discovery times out, the broad session/storage runs exceed their local default timeout, and the broad frontend suite reports multiple failures. Native Executors/New task verification and current-head CI are still in progress. Earlier live limitations above remain applicable.