feat: generate_prd and generate_artifact tools (ENG-968, ENG-969) - #335
Open
StpMax wants to merge 27 commits into
Open
feat: generate_prd and generate_artifact tools (ENG-968, ENG-969)#335StpMax wants to merge 27 commits into
StpMax wants to merge 27 commits into
Conversation
* tool for artifacts generation * make api spec in json, not md * do not save api spec in file * fix prompt * prompts * del `data_refs` from `generate_artifact` * tool update * tests * fix issues and logging * make generate_artifact the primary path for html and fullstack artifacts
* feat: ArtifactStore.update() can change artifact type * feat: raise the shared per-turn question budget from 3 to 8 * feat: generate_prd sub-tool schemas and direct-elicit ask_user dispatch * feat: generate_prd phase 1 gathering loop * feat: generate_prd phase 2 steps (draft_brief, show_and_confirm, classify_feedback, write_prd) * feat: generate_prd orchestrator.run — full two-phase sequence * feat: register generate_prd as a tool * feat: point create_artifact's description at generate_prd for web artifacts * fix: draft_brief/write_prd prompts — lead-in sentence, no process meta-commentary, no technical detail in the short brief Live-testing feedback on ENG-969: the short brief landed on the user with no framing, echoed the tool's own regeneration framing into Goal, listed data sources not used, and leaked CSS/JS implementation detail (clamp(), hex colors) into a document meant for a non-technical reader. * fix: remove Cyrillic from source and docstrings (project convention: code in English) - prompt instruction/label text: 'Тип артефакта' -> 'Artifact type', 'Принять'/'Отменить' button labels -> 'Accept'/'Cancel' - docstrings referencing prd-design.md section titles translated to English * fix: allow multiple lines in the brief's Data model section 'One short line' was too strict for multi-table cases (e.g. a dashboard reading several DB tables) — the actual intent was 'no verbose negatives and no connection details', not a hard one-line cap. Multiple sources now each get their own short line. * feat: default-value support for choice questions — Enter accepts the PRD brief AskRequest gains default_value: the value of an option chosen when the user presses Enter with no input, instead of the channel's usual 'cancelled'. CLIElicitor._ask_choice forwards it to prompt_or_cancel (which already substitutes it on blank Enter and shows it in the prompt suffix) and now also resolves a directly-typed option value, not just its number. generate_prd's show_and_confirm sets default_value="accept" — a bare Enter now continues instead of cancelling, matching what users actually do most of the time. draft_brief's prompt also asks the model to close the brief with a short in-language line making that explicit. ask_user's own JSON schema is untouched — only orchestrator code that builds AskRequest directly (select_path, generate_prd) can set a default; the LLM never picks one on the human's behalf. * feat: compact rendering for self-explanatory choice prompts Live-testing feedback (ENG-969): the PRD brief ends with its own 'continue, or changes?' sentence, then repeats the same choice as a numbered Accept/Cancel list plus a descriptive input caption — pure noise once the sentence already explains the mechanic. AskRequest.compact (opt-in, off by default) tells StreamDisplay.show_question to skip the option list and CLIElicitor._ask_choice to drop the descriptive caption in favor of a bare input point; prompt_or_cancel's own default-value suffix (e.g. '(accept):') still carries the hint. Parsing is unaffected — numbers, typed values, and free text all still resolve the same way. generate_prd's show_and_confirm sets compact=True. Every other ask_user/select_path caller is untouched. * fix: restart the spinner after ask_user before the next direct LLM call elicit() stops the host spinner (phase="interactive") for ask_user/ show_and_confirm but nothing restarts it — the outer agent loop's own reasoning_start signal never fires for generate_prd's direct _llm.plan/code/generate_object calls, since they run outside that loop's tool-round machinery. Made the gap visible: after answering the brief confirmation with free-text feedback, the UI showed nothing for a couple seconds until the revised brief appeared. Add sub_tools.signal_thinking() and call it before every direct LLM call in phase 1's gathering loop and phase 2's draft_brief/ classify_feedback/write_prd. * fix: restart the Live spinner context on reasoning_start, not just update it phase="interactive" tears down the spinner's Live context entirely (_stop_spinner sets _live = None) so the user can type. The preceding fix (signal_thinking, previous commit) emits reasoning_start right after an ask_user/show_and_confirm answer, but that phase's handler only called _update_spinner(), a no-op once _live is None — so the event reached the display and still produced no visible spinner. Elsewhere reasoning_start already finds a running Live (a tool-result line printed in between implicitly restarts it), which is why this gap went unnoticed until generate_prd's direct LLM calls exposed it. * fix: constrain finish_gathering's artifact_type to the closed enum Nothing stopped the model from inventing an artifact_type string outside ArtifactType's closed set (html-app, document, dataset, image, mixed, fullstack-stateless-app, fullstack-stateful-app) — the tool schema had no enum and the gathering system prompt never listed the valid values. An invented type sailed through unvalidated until write_prd's ArtifactStore.update(type=...) call, which raises ValueError for anything outside the enum — crashing the whole generate_prd call with a message the outer agent sometimes misreads as a transient glitch worth silently retrying, producing an apparently-stuck "same brief shown again" loop. Add the enum to FINISH_GATHERING_SCHEMA, list the valid types in the gathering system prompt, and — since a schema enum is a hint, not an enforced constraint — fall back to the originally registered type in engine.py if the model returns one outside it anyway.
StpMax
changed the title
Mirrors generate_artifact/debug_trace.py's GenTrace/NullTrace shape and, deliberately, reads the SAME ANTON_DEBUG_ARTIFACT_GENERATE_TOOL env var: the two tools are the two steps of one end-to-end artifact-generation pipeline (PRD, then code), and both loggers append rather than truncate, so running them back to back writes one combined, chronologically ordered log — viewable as-is in artifact_trace_viewer.html. Instruments phase 1's gathering loop (every LLM call, ask_user, scratchpad/web_search/web_fetch dispatch, and how gathering ended) and phase 2's steps (draft_brief, show_and_confirm, classify_feedback, write_prd). generate_prd.generate() wraps orchestrator.run with run_start/run_result, including on a crash. Off by default (NullTrace) when the env var is unset — no behavior change for existing callers.
The brief's closing line asked the model to spell out that a bare Enter means "continue". That hint is CLI-specific text living in host-agnostic content: the same line is rendered in the terminal, where `prompt_or_cancel` already prints the `(accept)` default, and in a GUI, where the host draws Accept/Cancel buttons and there is no Enter to press. It was also LLM-generated, so its wording, language and very presence drifted between runs — an input affordance belongs to whatever code draws the input. The instruction already forbade inventing accept/cancel option labels for the same reason; the Enter hint just was not covered by it. Keep the closing question itself (added from ENG-969 live-testing feedback, and what makes `compact=True` legible) and forbid describing how to answer.
ENG-970. A fullstack generation runs for minutes and emitted nothing to the UI for the whole time, which reads as a hang. `handle_generate_artifact` becomes an async generator so it can use the streaming-tool protocol from ENG-763: `dispatch_tool_stream` forwards every yielded `ToolProgress` and takes the last non-marker item as the tool result, so all the existing return paths are unchanged apart from `return` becoming `yield`+`return`. That protocol was built and tested but had no production user until now. Yielding rather than calling `session.emit` directly (the way generate_prd's `signal_thinking` does) is what gets the marker its originating `tool_use` id: a handler never sees that id, only `dispatch_tool` does, and it stamps relayed markers with it. Without the id a marker is not the "first per id" the cloud wire never drops, so it falls under the rate limit, and a step UI has nothing to correlate it to. Progress travels up on an `asyncio.Queue` because the FSM cannot yield from the handler: steps start several frames down, including inside the `asyncio.gather` that generates backend and frontend at once. The channel's sentinel is pushed from a `finally`, so a crashed generation closes the drain loop instead of hanging it; the handler cancels the generation task from its own `finally`, so a cancelled turn does not leave the FSM writing files nobody will collect. `GenState.step_started` is deliberately separate from `record`, which fires when a node is already done — the two longest nodes would otherwise report only in hindsight. `progress.py` maps node names to plain-language lines, since graph vocabulary (`is_data_enough`) must never reach a user; an AST test over orchestrator.py fails if a call site has no label.
…rief `generate_prd` wrote prd.md and nothing ever read it. The artifact was built from whatever the calling agent chose to put in `context`, so the document the user actually reviewed and accepted had no effect on the result — the two tools were a pipeline only in the tool descriptions. The generator now reads prd.md from the artifact folder itself. That needs no new schema parameter: the handler already resolves the folder from `slug`, so the calling agent cannot forget to pass the PRD, mis-transcribe it, or paraphrase it. Every failure to load degrades to "no PRD" rather than stopping the run — an agent may legitimately skip the PRD step, artifacts created before ENG-969 have none, and a file that cannot be read must not cost a generation `context` alone can still complete. Which mode ran is recorded, so a wrong-looking artifact can be traced back to the requirements it was really built from. `context` is redefined as a supplement rather than a competing source of truth. Its `## Functional Requirements Specification` section is gone — requirements live in the PRD, and two copies invite disagreement — and `## Data` is narrowed to data that already exists in scratchpad cells, with `### Sample` rows only from cells actually run. Expected-but-unfetched sources are what the PRD describes; repeating them here is a guess competing with an accepted document. All three surfaces that state this contract are updated together, and the tool prompt now names generate_prd before generate_artifact — without that step the model has no reason to believe a PRD exists and every run would take the fallback. Pad matching now searches the PRD alongside the brief: generate_prd must cite the scratchpad and cell behind every source it describes, so on the normal path the PRD is where a pad name appears at all. Cells the PRD points at land in `data_notes`, which lets `is_data_enough` answer from what is already there instead of starting its own fetch loop. PRD_FILENAME is shared by the writer and the reader: a literal in both packages could drift apart silently, and the reader would then simply find nothing and build from a brief the user never confirmed.
`grep generate_prd anton/core/llm/prompts.py` came back empty: every always-on block sent the agent straight from `create_artifact` to `generate_artifact`, while `create_artifact`'s own description said to draft a PRD first. Which instruction won was luck. Since the generator now takes its requirements from the `prd.md` that `generate_prd` leaves in the artifact folder, this is no longer a missing step but a broken one: a run reached that way has no PRD at all and silently falls back to building from `context` — which, with the requirements section removed from it, carries considerably less than it used to. ARTIFACTS_PROMPT's workflow grows a PRD step ahead of generation, and states that the generator reads prd.md itself so the agent does not quote or paraphrase it back into `context`. The two visualization blocks and BACKEND_GENERATION_PROMPT get the same three-call normal path. A test locks the ordering across all four blocks. `generate_prd` has no `ToolDef.prompt` of its own, so these blocks and its tool description are the only places the model can learn the step exists — a new always-on artifact block would otherwise omit it again.
`_reconcile_files` re-derives `files[]` from disk and excluded only the store's own files, so `prd.md`, `spec.md` and `openapi.json` counted as artifact content: they inflated file_count, appeared in the rendered README, and invited the agent to hand the user a specification as the thing it built. They are what the generator built FROM. Publication was not affected, contrary to what the handoff assumed — no publish path reads `files[]`. An html-app publishes its primary file plus the siblings the HTML references, and `_zip_fullstack` bundles an explicit allowlist (backend.py, requirements.txt, static/**), so root-level inputs never entered a bundle. (`_zip_html`'s directory branch would take them, but no artifact path reaches it.) `backend.log` goes in too. The set is documented as mirroring cowork-server's artifacts service, and that service, `publish_access` and the publish bundle all excluded the launched backend's runtime log — the store was the one copy that did not, which is precisely the copy the agent and the UI read their file list from. Generation inputs stay in a set of their own rather than being folded into the housekeeping one: they are authored by the generation tools, not owned by the store, and merging them would quietly falsify the mirror claim. Exclusion is by relative path, so a `static/openapi.json` the artifact genuinely serves remains its own content. The three filenames now live together in `artifacts/internal_files.py` (renamed from prd.py), because spec.md and openapi.json were literals in the orchestrator — the exclusion set and the code writing the files would have drifted apart on the next rename.
ENG-1116. `make_tech_spec` and `make_api_spec` each produce a whole document in one call on the client's default 8192-token budget, and neither checked for truncation. The two then failed very differently: a cut API spec broke `json.loads` and was reported as "not valid JSON", blaming the model's syntax for an output-cap hit, while a cut tech spec was written to spec.md and handed to backend and frontend generation by `_spec_context` — half a spec building half a system with nothing anywhere reporting the loss. That silence is the defect, not the length. Both calls now run with an explicit budget and, when cut off, are re-asked once with more room plus an instruction to write compactly. The re-ask has to change the call: the main loop's own recovery measured three unchanged retries dying identically. If the second attempt is also cut, the node returns an error naming the output limit — nothing partial is written, and the API-spec path reports the cap rather than the JSON parse that follows from it. The budgets are measured, not chosen: against api.mindshub.ai's `opus` alias, 20480 answers normally and 24576 and above return HTTP 500. A 500 is classified as a transient provider error, so an over-large budget does not fail fast — it burns the retry ladder on every generation before dying. 16384/20480 keeps real headroom (reasoning models spend thinking from the same budget) while staying inside what the gateway serves; both constants carry the measurement and must be re-measured before a raise. Truncation is detected with the shared `looks_truncated`, which also honours stop_reason — the gateway has reported it correctly since 2026-08-03, and a cut that stops just under the cap is invisible to a token count alone.
…STATE store Align the generation pipeline with the STATE SDK model merged from staging (#259). Before this, the pipeline still carried the old stateful model (a local sqlite file in the artifact root), and a backend written to the new contract could not pass verification at all. - _STATEFUL_RULES rewritten to the STATE contract: module-level STATE = None slot, store built at point of use via get_store(), Collection-first API, no scan / no secondary indexes, atomic increment/update, no manual retry around mutations, never list anton_state in requirements.txt, and the flat state_manifest.json format (written by the generator sub-agent via write_file; validated by the verifier, so hand-written JSON is safe). - Stateful backend task is now three files; step injections chain backend.py -> state_manifest.json -> requirements.txt. - verify_backend runs its introspection subprocess with build_backend_env (anton_state on PYTHONPATH, same injection the launcher uses) — without it a correct stateful backend failed its own SDK import. The env builder went public in backend_launcher; the underscored name stays as an alias. - evaluate_backend gains artifact_type/state_manifest kwargs and six new contract errors: anton_state listed in requirements (any type); for stateful — missing STATE slot, missing/invalid manifest (validated via anton_state.schema.StateSchema), store built at import time (AST check); for stateless — anton_state imported at all. The requirements parser also drops anton_state before install, mirroring the launcher. - Spec prompts know about durable state: api-spec gets a stateful counterpart to the stateless constraint (key-per-access-pattern, one query per listing), tech-spec pins the STATE store in the fixed stack so spec.md stops inventing sqlite. - Type-selection surfaces rewritten to the new model: create_artifact's fullstack-stateful-app description, the ARTIFACTS_PROMPT paragraph (the staging merge had left it self-contradictory), and generate_prd gathering criteria; the PRD data-model sections now capture app-owned durable state. - Store/publish_access housekeeping sets exclude the local STATE driver's .anton_state.db(-wal/-shm) and the publisher's schema snapshot; state_manifest.json itself stays a visible deliverable. cowork-server's mirror copy still needs the same names. - Contract-lock RULES extended for the six new verifier messages; 19 new tests (pure checks, subprocess integration incl. a PYTHONPATH-injection proof, orchestrator wiring, store exclusions, prompt locks). Known limitation: a live end-to-end run is currently impossible — the api.mindshub.ai gateway WAF returns 403 on any request body containing "<script" (pre-auth), which breaks all artifact generation including the pre-existing html path. See claude_workspace docs/artifact-generation-tools/tracking-stateful.md (S-13). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…4 on long generations api.mindshub.ai sits behind Cloudflare, which kills a proxied connection after ~100s of silence (524). The FSM's spec/code-writing calls used the one-shot plan()/code(), which sends no bytes over the wire until the whole completion is ready — a large tech spec or a full HTML/JS file easily exceeds that window and the run dies with "the model provider returned 524", discarding an already-confirmed PRD. Swap _plan_whole_document and _run_loop onto the existing plan_stream(), plus a new code_stream() (mirrors plan_stream(), previously only code() had no streaming sibling). A small _drain_stream() helper consumes the token-level deltas and returns the terminal StreamComplete's response, so callers keep the exact same LLMResponse shape they had before. Bytes now flow continuously regardless of total generation length, so the proxy never observes silence. generate_prd has the identical plan()/code() call sites and is exposed to the same failure mode; left out of this change to keep it scoped to the reported incident.
…ay (ENG-1986) api.mindshub.ai sits behind a Cloudflare WAF rule that 403s any request body containing the literal `<script` or `</script` (case-insensitive, even with injected whitespace) before authentication. html-app generation necessarily asks the model to emit `<script>...</script>` tags, and once written the content is echoed back into the conversation history on every later round, so the block was unavoidable through prompt wording alone. OpenAIProvider.complete()/stream() now escape `<script`/`</script` to `<_script`/`<_/script` in the outgoing system prompt and messages, and reverse it on the incoming response content and tool-call input, so genuine `<script` never appears in a request body but callers still see real tags. The marker was verified against the live gateway to fall outside the WAF rule's match. Scoped to FLAVOR_MINDS_PASSTHROUGH only — no other provider flavor talks to this gateway.
…iversal !important rule
The `* { ... !important }` check fired on the standard accessibility reset
inside `@media (prefers-reduced-motion: reduce)` — a block models emit
reflexively and one that cannot override host styles, which is what the rule
exists to prevent. Live run 2026-08-27: an otherwise valid 14-slide
presentation was rejected on exactly this false positive.
Universal !important blocks are now allowed inside `prefers-reduced-motion`
and `print` media queries (block spans found by brace counting — nested rule
blocks must not truncate or extend the span). Everywhere else the rule and
its wording are unchanged, so the contract lock stays intact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ry budgets One shared `range(GEN_VERIFY_MAX_RETRIES + 1)` counted a loop failure (round budget, no tool calls) and a verification failure as the same kind of attempt. Live run 2026-08-27: attempt 0 died on the round budget before any verifier ran, attempt 1 failed verification on a one-line CSS fix — and the run was terminal, because the loop failure had already burned the only retry meant for acting on verifier findings. `GEN_LOOP_MAX_RETRIES` and `GEN_VERIFY_MAX_RETRIES` now count independently in both `_gen_verify_frontend` and `_gen_verify_backend` (worst case three attempts instead of two). The terminal message names the budget that actually ran out: "verification failed after N attempt(s)" only when the verifier rejected, "generation failed" otherwise — the old wording claimed a verification retry that, on the loop-failure path, never happened. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eneration loop Four changes to `_run_loop` and its messages, all from the 2026-08-27 live run, where a complete 48 KB page was deleted and regenerated because the loop ran out of rounds counting its own slides: - Round-budget exhaustion with files on disk now returns a normal result dict (`finished: False`) instead of an error, so the caller verifies what was actually written. A missing `finish` call is not evidence the files are bad. With no files written it stays an error. - Every tool-result message now carries a `[N round(s) left]` note, switching to an explicit "wrap up NOW" instruction on the last five rounds. The model had no way to see the budget it kept dying against. - Truncation detection honours `stop_reason` (`length`/`max_tokens`) alongside the token count, and reads the client's public `max_tokens` property instead of the private `_max_tokens` (I-08). The rejection messages name a concrete next-chunk size (4,000 chars) instead of "well under ~6 KB" — the vague form did not stop either live-run attempt from burning a full 8192-token reply on an oversized append. - `read_file` passes the new `full` flag through to the sub-tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e_file warns on oversized chunks `read_file` existed so the model could check what landed, but returning the whole content meant re-reading a 48 KB page into the context (and through the prompt-cache prefix) just to confirm it ends with `</html>` — measured 2026-08-27 at ~19k input tokens per check. It now returns the size plus the last 500 characters; `full=true` (advertised in the schema as expensive) returns everything, and files at or under the tail size come back whole. `write_file` still accepts an oversized chunk — the call that arrived fit under the output cap by definition — but its result now warns that the next one may not: in both live-run attempts the model followed a lucky oversized chunk with a bigger one and lost a full 8192-token reply to truncation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…del verification is code's job Three prompt changes, all measured against the 2026-08-27 live run (929 s, failed): - html-app with a confirmed `prd.md`: `make_tech_spec` is told the generator receives the PRD verbatim next to its document, so it must not restate it — only `## Insights` plus terse implementation notes. The full spec was near-pure duplication (190 s / 13k output tokens restating a 20 KB PRD as 35 KB of spec that then rode into every generation prompt). Fullstack types keep the full spec — it feeds the API design. - `_ROLE_WRITE` states that a deterministic verifier checks the output after `finish` and re-prompts with exact errors on failure: re-reading and re-counting one's own files is what burned nine of twenty rounds. - `_WRITE_DISCIPLINE` names a hard 6,000-character chunk limit that explicitly covers the first `mode="w"` chunk and the first append — the two calls that actually got cut — and `_ROLE_COMMON` forbids creating new scratchpad names (the once-per-turn challenge in `handle_scratchpad` makes the rejection non-deterministic across attempts). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-08-27 live run retold one article three times on the way to the artifact: scratchpad pad → prd.md (20 KB) → spec.md (35 KB) → index.html. Only the last copy is necessary. `_WRITE_PRD_INSTRUCTION` now forbids copying long-form source content that already lives in scratchpad cells — the PRD describes the structure (e.g. a slide outline, one line per slide) and cites the pad and cell; the generator reads those cells directly. Short samples stay allowed, and the existing Data-model requirements (connection code, sample rows) are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iants The WAF escape is visible to the model: it reads `<_script`-style forms in its system prompt and in its own echoed chunks, and reproduces the underscore in positions the strict reverse pattern does not cover. Live run 2026-08-27: a generator closed its only script block with `</_script>` (underscore after the slash), the unescape left it untouched, and the mangled tag reached the file on disk — the page's JavaScript never ran, and the calling agent spent most of its token bill discovering and hand-fixing it. The unescape now normalises every underscore variant (`<_script`, `<_/script`, `</_script`, `<__script`, `<_/_script`) — none of these is meaningful anywhere else, so being wider than the escape is safe. The escape side is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tags Two new frontend rules, both from the 2026-08-27 live run, where a page whose only script block was closed with `</_script>` passed verification and shipped with all of its JavaScript disabled: - any underscore variant of a script tag (`<_script`, `<_/script`, `</_script`) is an error — residue of the ENG-1986 WAF escape that the provider-level unescape may not have caught; - an opening `<script` with no closing `</script` anywhere in the document is an error — the general "JS never runs" class. Deliberately narrow (absence of ANY closer, not a count mismatch): a literal `"<script>"` inside an inline-JS string legitimately unbalances the raw counts. Both rules are announced in the HARD OUTPUT CONTRACT prompt block and added to the contract-lock table — a conscious rule addition, not a table refresh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ccessful generation After a successful run the calling agent re-verified the pipeline's work by hand: the 2026-08-27 run spent 11 planning-model calls (each resending a 42k- char system prompt plus the whole conversation) re-reading and re-parsing an artifact the generator had already verified — the bulk of the run's token bill. The success payload now carries an explicit instruction: the files were statically verified, do not re-read or re-verify them, act only on problems the user actually reports. Same nudge one level down: `read_file`'s schema now says `full=true` is never for verifying finished work — the inner loop pulled the entire 40 KB page back into its context right after the size+tail check said it was complete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One textual conflict, in tool_handlers.py: both branches inserted code right after `_artifact_store` — this branch the `_prd_generation_failed` + `handle_generate_prd` block, staging the artifact turn-tracking helpers (`_track_artifact` and friends, #399). Kept both. Plus one semantic gap git could not see: staging's turn tracking covers create/update/open_artifact and scratchpad-detected edits, but not this branch's `handle_generate_prd` / `handle_generate_artifact`, which write into the artifact folder directly. A PRD or regeneration happening in a later turn than `create_artifact` would leave that turn unattributed. Both handlers now call `_track_artifact` when they actually wrote (generate_prd: statuses that produced prd.md; generate_artifact: success and FSM failure alike — only the crash path skips). Tests after merge: 3076 passed, 30 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
verify_frontend rejected any absolute `href`/`src` outside a `<script>` tag. Two consecutive live runs failed on it, both wrongly: a dashboard built from a web article links back to its source and shows that source's images, and the PRD the user had already accepted asked for exactly those. The retry then "fixed" it by moving the same URLs into JS strings rendered through innerHTML — identical DOM, one full regeneration burned (~50% of that run's tokens), and the model now knows the workaround. A check that a correct artifact fails and an incorrect one passes is worse than no check. Only fetch() keeps the rule: a hardcoded host there breaks the artifact the moment it is published, because the backend it names is the local one. No warning replaces the removed rule — of the remaining cases, a `<link>` to a font degrades appearance without the network, and an `<img>` from the source article IS the content. The contract changes, so the Rule is removed from RULES in the contract lock deliberately, and the bullet is removed from _VISUAL_RULES: that block is declared to the model as "a static verifier checks each of these", and leaving a rule there that no longer exists is a shipped contradiction. Restoring this needs a check that sees the rendered DOM, not the HTML text. Co-Authored-By: Claude <noreply@anthropic.com>
… dropped streams
The generation loop ran every round on the client default of 8192 output
tokens, a quarter of what the gateway accepts. A 41 570-character artifact
(16 063 tokens) therefore had to be written in ~8 chunks, and each chunk is
re-sent on every later round — 23% of the node's context was chunk bodies
being paid for again.
Measured 2026-08-28 against api.mindshub.ai:
- 20480 is accepted on BOTH aliases. The note claiming 16384 for the coding
model was wrong; its ceiling is the same 20480.
- One write_file call at that budget delivered 50 402 characters of Russian
HTML in 15 754 tokens.
Round 0 deliberately does NOT get the raised budget. It runs on the planning
model, ~2.3x slower per token, and at 20480 a write there runs long enough to
lose its connection — 4 failures out of 4 at 131-143s. At 8192 the same call is
merely truncated, which the loop recovers from.
Truncation is now judged against the budget the round actually ran on. Against
the client default instead, every reply over 8192 tokens would be called
truncated and its last tool call rejected — the exact failure the raised budget
exists to remove.
Dropped streams are retried once, halving the budget. A large tool-call
argument is NOT streamed incrementally: the connection carries nothing for the
whole generation and everything arrives in one burst (112s of silence for a
59 000-character argument). That profile is identical when talking straight to
api.anthropic.com, so it is not the gateway's doing and cannot be fixed here.
Whether such a call survives is a race against the proxy's idle timeout, so a
retry on the same budget would mostly reproduce the failure; half the budget is
half the silence. Nothing has executed when a drop happens, so the retry cannot
double-apply a write.
The chunk limit moves 6 000 -> 16 000 characters and is now derived from
duration, not from the token budget: ~170 tok/s and ~2.59 characters per token
on Cyrillic prose gives ~37s of silence, a ~3x margin against the shortest
observed drop. The number was written out in four places the model reads; all
four now read the one constant, as does the halved recovery size.
test_write_discipline_names_a_hard_chunk_limit was passing vacuously:
"6,000 characters" is a substring of "16,000 characters", so it would have
stayed green through this change while checking nothing. It now asserts against
the constant, and a new test pins the limit across all four surfaces.
httpx becomes a declared dependency: already installed as a hard requirement of
both SDKs, but now imported directly to catch transport errors by type.
Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Combines two artifact-generation tools into the full two-step pipeline (PRD → code):
generate_artifact(ENG-968) — a deterministic FSM orchestrator that generates a finishedhtml-app/fullstack-stateless-app/fullstack-stateful-appartifact: data-sufficiency check → technical spec → (fullstack) API spec → backend & frontend generation with verification, running in parallel → app launch & health check. Backend/frontend verification checks routing under/api/*, health endpoint presence, secret handling, dependency manifests, and CSS/URL policy for the frontend. Replaces hand-written scratchpad generation as the primary path for these three artifact types.generate_prd(ENG-969) — a bounded two-phase tool that drafts a PRD and gets explicit user confirmation before any code is written: phase 1 gathers/verifies data and asks clarifying questions; phase 2 drafts a short brief, shows it for accept/cancel/revise, and on acceptance writes the fullprd.md.create_artifact's description now routes a web artifact through both tools in order: register →generate_prd(draft + confirm requirements) →generate_artifact(write and verify the code against that PRD). Non-web artifact types (document,dataset,image,mixed) skip both steps, as before.Resolves
Test plan
pytest— full project test suite: 2213 passed, 30 skipped (pre-existing), no failuresgenerate_artifact: 7 dedicated test files (FSM graph/prompts,_run_loop, state/models, orchestrator data loop and backend/frontend retries, backend/frontend verification, tool-handler error wrapping)generate_prd: 9 dedicated test files (state, prompts, sub-tools, gathering loop, phase-2 steps, fullrun()sequence, tool registration/handler,create_artifactdescription)generate_prd's confirm/revise flow (spinner behavior, artifact-type validation, brief formatting) — see individual fix commits on this branch🤖 Generated with Claude Code