LocalCut MVP

LocalCut is a local-first video editing Agent prototype. The Agent produces typed, reversible editing operations; the deterministic project core owns the timeline, history, preview state, and FFmpeg export.

The primary product is the Harness plus Project Bundle, not the browser editor. Codex, Claude, or another native Agent supplies the brief and creative judgment. LocalCut matches a workflow, commits versioned edits, verifies the result, and keeps a standard local project that both people and Agents can continue editing. The HTML surface is an optional Inspector and material-level timeline editor.

The canonical timeline supports two modes: a legacy single source with reversible cut ranges, and an ordered multi-material clips sequence. A real rough cut uses the latter: every clip retains its source asset, in/out points, playback speed, crop/free-transform, fit, audio policy, transition, and derived output range. An exported MP4 is a delivery artifact, never a replacement for those edit decisions.

data/
├── projects/<project-id>/
│   ├── bundle.json                 durable Bundle manifest and asset paths
│   ├── project.json                canonical current state
│   ├── operations.jsonl           append-only edit history
│   ├── revisions/                 immutable revision snapshots
│   └── harness/
│       ├── brief.md               original user intent
│       ├── plans/<plan-id>.json   exact revision-bound decision set
│       ├── locks/<plan-id>.json   resolved provider assets and checksums
│       ├── executions/<plan>.json persisted Harness phase and failure state
│       ├── reports/<plan>-rN.json verification and execution report
│       └── deliverables/          revision-bound FCPXML handoffs
└── media/<project-id>/             localized project assets

The complete v1 contract is documented in docs/project-bundle-v1.md.

Run

nvm use
npm install
cp .env.example .env
# Add the keys for the stock providers you want to use.
npm start

Open http://127.0.0.1:4317.

The server binds only to 127.0.0.1. Project data is stored under data/, which is intentionally ignored by Git.

The stock-media panel searches Pexels and Pixabay for video/images and Freesound for licensed audio previews. Create keys from the Pexels API, Pixabay API, and/or Freesound API pages, then expose them as environment variables in .env before starting LocalCut. The runtime loads this ignored local file automatically. Search responses are normalized and cached under data/stock-cache/ for 24 hours. Only media selected for import is downloaded into the local project; API keys and provider download URLs are never sent to the browser or stored in project JSON. The editor discloses that search terms leave the machine before the request is sent.

Native Agent access

The runtime exposes the same project core through an HTTP client, a JSON CLI, and an MCP stdio server. Codex and Claude plan work; LocalCut validates and commits it.

# Machine-readable CLI
node cli/localcut.mjs health
node cli/localcut.mjs list
node cli/localcut.mjs inspect demo-localcut

# Primary Harness flow: prepare, review, then execute the exact rough-cut plan
node cli/localcut.mjs rough-cut-plan demo-localcut \
  --brief "删除废词、生成字幕，每 15-20 秒添加一条 3-6 秒全屏 B-roll" \
  > /tmp/localcut-rough-cut.json
node cli/localcut.mjs rough-cut-apply demo-localcut \
  --plan /tmp/localcut-rough-cut.json --confirm

# Lower-level flow for one narrow edit
node cli/localcut.mjs plan demo-localcut \
  --command "删除废词并加字幕" > /tmp/localcut-plan.json
node cli/localcut.mjs apply demo-localcut \
  --plan /tmp/localcut-plan.json --confirm

The project-level .mcp.json is discovered by Claude Code. Verify it with:

claude mcp get localcut

Register the same local server in Codex with:

codex mcp add localcut \
  --env LOCALCUT_URL=http://127.0.0.1:4317 \
  -- "$(pwd)/scripts/run-mcp.sh"

The launcher selects an installed Node.js 20+ runtime. Set LOCALCUT_NODE when an explicit binary is preferred. The Codex plugin manifest is in .codex-plugin/plugin.json; the canonical workflow is in skills/localcut-native-video/SKILL.md, with a Claude project Skill delegating to it from .claude/skills/.

MCP exposes the complete native editing tool set. prepare_rough_cut and execute_rough_cut are the default Agent-facing workflow; the remaining tools cover health and discovery, project creation and inspection, project-bound Agent sessions, observable Production Runs, media/SRT import, local word-level transcription, caption export, composite review frames, Pexels/Pixabay search and local stock-media import, automatic B-roll planning, placement and replacement, edit planning, reviewed-plan application, expert typed operations, batch undo/redo, named versions, review Markers, canvas presets, verification, full or In/Out range MP4/MP3/WAV/FCPXML delivery, and HTML editor handoff. Stale plans are rejected by revision.

Every project session has one execution preset:

Mode	Behavior
Director	Produces creative and edit proposals without modifying or exporting the project.
Collaborative	Plans first and waits for confirmation before every Agent mutation.
Autopilot	Requires one explicit authorization, then permits a bounded Production Run to execute revision-bound plans.

These are policies in one runtime and one Skill, not three separate plugins. The user can open the same session URL and manually adjust the timeline, captions, Motion Graphics, or any other supported property at any time.

LocalCut is project-first: edit and rough-cut requests commit an editable Bundle by default. FFmpeg rendering is a separate, revision-bound adapter action and runs only when the user explicitly asks to export or deliver a video.

An Autopilot session grants permission but does not start work. A Production Run binds one goal to that session and persists its status, iteration count, steps, verification result, and optional output. The HTML editor uses the deterministic local runner. Codex, Claude, and other MCP clients use the model-independent native-agent runner and report the same inspect-plan-apply-verify-evaluate loop through the Run tools. Queued or running local Runs are recovered automatically after a runtime restart; paused Runs remain paused until the owning session resumes them.

MVP workflow

1. Create or select a local project and import its source video, image, or audio. Local transcription starts automatically when whisper.cpp is configured; SRT import remains available as a fallback.
2. Give the complete rough-cut brief to Codex or Claude. The Agent calls prepare_rough_cut; LocalCut matches the workflow and returns one exact, revision-bound edit and B-roll plan.
3. Review the plan in Collaborative mode, stop at the proposal in Director mode, or execute it through an authorized Autopilot Session.
4. LocalCut applies typed edits, searches and localizes Pexels/Pixabay footage, locks provider IDs and SHA-256 checksums, verifies a staged project, commits the complete operation batch atomically, optionally renders an MP4 preview, and persists the full decision trail in the Project Bundle. Use get_rough_cut_execution to inspect the durable phase or failure reason.
5. Continue by talking to the Agent or open the HTML editor to select, trim, reorder, replace, or remove source clips and adjust captions, MG parameters, B-roll, or audio manually.
6. Use a Production Run only for open-ended, multi-iteration work that needs pause, resume, cancellation, and an observable step loop.

Architecture

Natural-language brief
        |
        v
Skill / workflow matching (replaceable strategy)
        |
        v
Revision-bound RoughCut Plan
        |
        v
Harness execution: typed operations + assets + verification
        |
        v
Canonical Project Bundle
        |
        +--> Codex / Claude edits
        +--> HTML Inspector
        +--> FFmpeg MP4 / MP3 / WAV
        +--> FCPXML 1.10 handoff
        +--> future Jianying adapters

The canonical project supports version checks, idempotency keys, transactional operation batches, revision snapshots, batch undo/redo, and named versions. Generated files never replace the project as the source of truth.

Test

npm test
npm run test:coverage
npm audit --audit-level=high

The test suite includes a real FFmpeg integration test that generates a source video, applies cuts, captions, and a Motion Graphic, renders the result, and verifies its duration with ffprobe.

Run the Talking Head release gate separately with npm run test:phase1. The evidence and manual smoke checklist live in docs/phase-1-acceptance.md.

The Phase 2 release record, including the full FFmpeg gate and the remaining Final Cut Pro application-level import block, lives in docs/phase-2-acceptance.md.

Current boundary

• Included: Harness-first rough-cut planning/execution, standard Project Bundle, project core, local persistence, revision history, three Agent modes, project-bound execution policies, persistent Production Runs with bounded local/native-Agent loops, Agent planning, video/image/audio and folder import, Pexels/Pixabay video/image search, Freesound audio search and local stock-media import, transcript-driven automatic B-roll planning, Agent search-query refinement, explainable multi-factor candidate ranking and BR-track placement, full-screen/ PIP preview and FFmpeg composition, per-clip replacement/removal, multi-material primary sequences with drag insertion, frame-snapped trim, split/duplicate/reorder/remove, 0.25x–4x playback speed, crop/free-transform, directional transitions and crossfades, canonical asset tags with reviewable Agent suggestions, cached local thumbnails, separate reference-mix audio, automatic local word-level transcription with visible retry, SRT import, transcript fact correction, caption display overrides, word highlighting, styled SRT/TXT/ASS delivery, evidence-backed pause cleanup, parameterized MG typography, editable audio roles, loop/fades, automatic ducking and limiting, composite review frames, Timeline/Clip Markers, canonical canvas presets, timeline preview, recoverable project lifecycle, batch undo/redo, named versions, export preflight, full or In/Out range MP4 export, independent MP3/WAV mix delivery, FCPXML 1.10 editable handoff, reversible multi-Timeline editing, optional linked ripple, main-video opacity and fades, free-position PIP and magnetic snapping, typed Motion Graphics with shared Design Style tokens, transparent ProRes 4444 MG export, per-clip LUTs, bins and asset relinking, deep black-frame checks, CJK and aligned bilingual captions, structured Agent references, workflow scenes, local user Skills, background export jobs with progress, cancellation and retry, provider-neutral generation tasks, reversible speech enhancement, phrase-level transcript edits, direct MG positioning, and operation/delivery P50/P95 metrics, JSON CLI, MCP server, Codex plugin metadata, and Codex/Claude Skills.
• Local ASR requires a user-installed whisper.cpp binary and model configured through LOCALCUT_WHISPER_BIN and LOCALCUT_WHISPER_MODEL; the model is not downloaded or uploaded by LocalCut.
• Next adapters: frame-level vision reranking, background B-roll jobs for long-form projects, packaged desktop runtime, and structured Jianying handoff. External image/video/voice/music generation remains a replaceable Agent or BYOK provider layer; generated files re-enter LocalCut as normal local assets instead of becoming a second project format.
• Deferred editing depth: keyframes, explicit main-track gap clips, track-bound zoom, chart/data MG, shader generation, automatic speaker diarization, machine translation, and high-fidelity FCPXML speed/transform mapping. SVG import is also deferred until LocalCut has a trusted local rasterization pipeline; convert SVG files to PNG, JPEG, or WebP first. The current FCPXML is standards-tested, but application import still requires a compatible Final Cut Pro installation.