Skip to content

V03_TEST_BASELINE.md

Latest commit

 

History

History
66 lines (46 loc) · 3.97 KB

V03_TEST_BASELINE.md

File metadata and controls

66 lines (46 loc) · 3.97 KB
Raw

Multi-Agent Collaboration V0.3.1 Test Baseline

Recorded on 2026-08-01 (Asia/Shanghai) on Windows x64 and WSL2 Ubuntu 26.04 x64 with Bun 1.3.14.

Isolation

  • npm registry is process-locally fixed to https://registry.npmjs.org/ through .npmrc.test, NPM_CONFIG_USERCONFIG, and NPM_CONFIG_REGISTRY.
  • Git fsmonitor is disabled only in the child process with GIT_CONFIG_COUNT, GIT_CONFIG_KEY_0=core.fsmonitor, and GIT_CONFIG_VALUE_0=false.
  • Every invocation uses a PID-unique repository-sibling .opencode-v031-test-runtime directory for TEMP, XDG config/data, and Bun cache state. The sibling location prevents discovery of repository .opencode plugins and AGENTS.md; the same drive preserves Windows path semantics.
  • Collaboration V0.2/V0.3, Scanner, Journal, management API, automatic recovery, and fault-injection flags are explicitly disabled for the general baseline.
  • The PID directory is removed after the child exits; cleanup failure makes a successful run fail. No user-level npm or Git configuration is modified.
  • Reusable runner: bun run test:v03-baseline.

Focused V0.3.1 baseline

bun run test:v03-baseline

WSL2 result: 55 passed, 1 platform-specific skip, 0 failed, 228 assertions, 10.02 seconds across 10 files. The selected Worktree.remove fsmonitor fallback case only runs on Windows; all 56 selected tests were accounted for.

Direct collaboration result: 42 passed, 0 failed, 201 assertions, 9.68 seconds across 8 files.

The focused baseline covers V0.1/V0.2 compatibility, V0.3 Scanner, Journal, recovery, lifecycle, management API/UI, real child-process locking, registry isolation, and Windows fsmonitor fallback.

Typecheck and migration baseline

  • OpenCode and Core package-level typechecks passed during focused diagnostics.
  • packages/core: bun run migration --check: passed in 3 seconds in the WSL2 clean bundle clone; no incremental schema changes and the full generated schema matched.
  • Root bun run typecheck: passed all 30 tasks in 99.005 seconds in the WSL2 clean bundle clone. Both tracked declaration links were real symlinks and resolved to ../../ui/src/custom-elements.d.ts.
  • The Windows checkout still materializes those links as 33-byte path text files and produces TS1128. That platform limitation remains documented, but it no longer blocks the release because the required symlink-capable gate passed in WSL2.

Build baseline

The exact all-target bun run build completed successfully in the clean bundle clone after assigning process-local writable runtime directories. It emitted:

  • Linux: arm64, x64, x64-baseline, arm64-musl, x64-musl, x64-baseline-musl.
  • macOS: arm64, x64, x64-baseline.
  • Windows: arm64, x64, x64-baseline.
  • Windows x64 and x64-baseline --version smoke tests passed.

Full repository baseline

bun run test:v03-baseline --timeout 60000

Final result:

  • Total: 3293 tests across 254 files
  • Passed: 3270
  • Failed: 0
  • Skipped: 22
  • Todo: 1
  • Snapshots: 50 passed
  • Assertions: 8831
  • Execution time: 460.73 seconds

The Linux test graph contains 45 fewer platform-selected cases than the Windows graph and 38 fewer skips. The two file-symlink tests that capability-skip on this Windows token executed on Linux. The historical Windows 18-failure analysis remains 9 environment/platform (D) and 9 test-isolation (E), with no V0.3 regression (A); detailed evidence is in docs/collaboration/V0.3.1_BASELINE_REPORT.md.

Repeat and race evidence

  • Four V0.3.1 regressions (Scanner cancel, recovery idempotency, route cache isolation, fault recovery): 50/50 passed, 320 assertions, 31.43 seconds.
  • SQLite/lock real child processes: before fix 13 pass / 7 fail across 10 rounds; after fix 20/20 passed, 180 assertions, 80.81 seconds.
  • Shell progressive metadata: before deterministic synchronization 4/5 passed; file-latch implementation 10/10 passed.
  • Compaction cancellation: five independent Windows measurements were 315–354 ms, supporting a 500 ms Windows-only test threshold while other platforms retain 250 ms.