Skip to content

framework-dev.md

Latest commit

 

History

History
91 lines (50 loc) · 14 KB

framework-dev.md

File metadata and controls

91 lines (50 loc) · 14 KB
Raw

Framework development (editing WebJs itself)

Read this only when editing the WebJs monorepo (this repo), not a scaffolded app. The repo is buildless: packages/ is plain .js with JSDoc (never add .ts there); TypeScript is fine in examples/, docs/, website/. Each in-repo app (website/, docs/, examples/blog/, packages/ui/packages/website/) is run from its OWN dir via npm run dev / npm start; as of #550 a bare webjs dev / webjs start is equivalent (each app's per-environment orchestration, the Tailwind watcher, webjs db migrate, the registry copy, moved into its webjs.dev / webjs.start tasks config, which webjs dev/start run). The sections below cover the repo-health git config, the changelog flow, and the dev error overlay.

Deploying the in-repo apps (Docker image + readiness gate)

The four in-repo apps (website, docs, examples/blog, packages/ui/packages/website) deploy from ONE image built by the root Dockerfile, each run as a separate service with its own PORT (compose sets it locally, the platform injects it in prod). compose.yaml is local parity for that setup; the platform never reads it.

The readiness gate is the same /__webjs/ready endpoint the framework ships and documents (503 until fully warm, then 200, see the deployment docs page). Two seams carry it, because no single file configures every platform:

  • Docker / compose / most Docker-based hosts: the root Dockerfile HEALTHCHECK (PORT-driven, dependency-free node -e fetch) makes the image self-gate. This mirrors packages/cli/templates/Dockerfile, the pattern the scaffold ships to users.
  • Railway: it IGNORES the Docker HEALTHCHECK and only honours its own healthcheckPath. railway.json declares healthcheckPath: /__webjs/ready, but a service only applies it if it is wired to read railway.json (config-as-code) AND built via the Dockerfile builder. A service left on the RAILPACK builder with no config-file path ignores railway.json entirely, so its healthcheckPath is null and deploys serve a cold-start window. Wire each service to railway.json rather than setting healthcheckPath by hand in the dashboard (dashboard values drift from the repo).

Net: edit the HEALTHCHECK for the Docker contract, keep railway.json for the Railway contract, and never hand-set deploy config in a platform dashboard.

Repo health: worktree-safe git config (core.bare / hooksPath)

This repo uses git worktrees (the review subagents spawn throwaway ones under .claude/worktrees/). Git's worktree machinery can leave core.bare=true in the shared .git/config, which is lethal to the main checkout: every git operation that needs a work tree then fails with fatal: this operation must be run in a work tree. The shared value is harmless only while the main worktree carries a per-worktree override (extensions.worktreeConfig=true plus a .git/config.worktree pinning core.bare=false).

scripts/git-worktree-safe.mjs establishes that override and pins an absolute core.hooksPath to .hooks on the main worktree, where both survive a shared-config reset (which is what otherwise silently disables the framework .hooks/pre-commit). It runs from the root prepare script, so every npm install self-heals. Two manual entry points:

  • npm run fix:git heals the config on demand (run it if a git command reports the work-tree error).
  • npm run check:git asserts the invariant (core.bare resolves false, the framework hook is active) and exits non-zero otherwise. The regression test is test/repo-health/git-worktree-safe.test.mjs.

Because the pin lives in the main worktree's config.worktree, git worktree add copies it into each linked worktree, so a commit made inside a throwaway review worktree also runs the framework .hooks/pre-commit. That is harmless (the hook only blocks main and auto-generates a changelog on a version bump), and review subagents are read-only so they do not commit; the inheritance is noted here only so the behavior is not surprising.

Merged worktrees are auto-removed (cleanup-merged-worktree.sh)

Per-task worktrees pile up when a session merges its PR but never runs git worktree remove (a skipped step, or a crash mid-task). The .claude/hooks/cleanup-merged-worktree.sh PostToolUse hook (matcher Bash, wired in .claude/settings.json) closes that gap: after any gh pr merge, it sweeps every linked worktree and removes the ones that are safe to drop, so cleanup is deterministic rather than a thing an agent has to remember.

It is conservative. A worktree is removed ONLY when it is a linked (non-primary) checkout, on a non-main/master branch, whose branch is MERGED (an ancestor of origin/main, OR a merged GitHub PR for that head branch, which is how squash-merges are detected via gh) AND whose working tree is clean apart from untracked node_modules / .webjs. It KEEPS anything dirty, unmerged, the primary checkout, or the worktree the merge was run from (you cannot remove your current directory; cd out and remove it manually), reporting each kept/removed worktree back to the model via hookSpecificOutput. It never blocks the tool (always exits 0). Escape hatch: WEBJS_NO_WORKTREE_CLEANUP=1. Regression test: test/hooks/cleanup-merged-worktree.test.mjs.

The fix only repairs the LOCAL checkout. Commits and branches are always safe on GitHub regardless.

Scaffold teaching-coverage gate (gallery-coverage.test.js)

The scaffold is webjs's primary teaching surface for AI agents, so a new framework feature must ship a runnable gallery demo, not just a doc bullet. Enforcement is two tiers, mirroring how tests are enforced:

  • Tier 1 (commit floor): .claude/hooks/require-scaffold-with-src.sh blocks a commit that stages packages/(core|server|cli)/src with no scaffold surface. It only proves you touched a scaffold file, so a documented-but-undemoed feature can still pass (this is exactly how #848 shipped forbidden() / unauthorized() with app-tree bullets and no demo).

  • Tier 2 (CI gate): test/scaffolds/gallery-coverage.test.js reconciles the LIVE framework surface against test/scaffolds/gallery-coverage.json and FAILS when something new is neither demoed nor exempted. It gates three surfaces: @webjsdev/core exports (a { demo } gallery-file pointer), @webjsdev/server exports ({ demoed: true }, verified by a generated app importing it), and routing convention files (the stems DERIVED from packages/server/src/router.js, each demonstrated by a file in a generated app). It runs under npm test, so a local --no-verify cannot skip it: a new export or convention turns CI red until classified. The reconcile() / reconcileSet() cores are pure and their failure modes (new name, stale key, missing/over-claimed demo, empty reason) are proven with synthetic inputs alongside the real-surface assertions. The deferred backlog is tracked in #859.

When you add or rename a @webjsdev/core or @webjsdev/server export, or add a routing convention file the router parses, update the manifest the same way you write a test: add a demo pointer / { demoed: true }, or an honest exemption. All three surfaces are gated (the convention stems are derived from packages/server/src/router.js, so a new stem === '...' branch auto-appears and must be classified).

The scaffold gate is one of a FAMILY of tier-2 coverage gates that keep the framework's agent-facing surfaces from rotting behind a new feature. The others:

  • Knowledge coverage (test/knowledge/knowledge-coverage.test.js): reconciles the live webjs check RULES against the troubleshooting page + gotcha docs (a new rule must be explained in a symptom-keyed surface or exempted in knowledge-coverage.json), and asserts the AGENTS.md headings the MCP init primer sources (DERIVED from the sectionByHeading(agents, /.../) calls in packages/mcp/src/mcp-docs.js) still exist, so a heading rename cannot silently empty the primer.
  • API docs + test coverage (test/api-coverage/api-coverage.test.js): every agent-facing @webjsdev/core + @webjsdev/server export (NON-internal per the scaffold manifest, the single source of truth) must be referenced in the docs corpus (AGENTS.md + the skill + docs site) AND in a test. A new public export that ships undocumented or untested turns CI red.
  • Types (test/types/dts-export-coverage.test.mjs, server-types.test.mjs), elision (packages/server/test/elision/lifecycle-coverage.test.js), and llms.txt (test/docs/llms.test.mjs) round out the family. Each reads its live surface dynamically so it cannot go stale.

Changelog: per-package, per-version, auto-generated

WebJs ships per-package per-version changelogs under changelog/<pkg>/<version>.md. The model: a version bump is the trigger. When any commit on main changes the version field in packages/<pkg>/package.json, the scripts/backfill-changelog.js generator emits a new changelog/<pkg>/<version>.md summarising every conventional-commit (feat: / fix: / breaking: / perf:) that landed in that package since the prior bump. The website renders the union of all packages' files at /changelog.

How it works for AI agents and humans:

  1. Bump the version field in a packages/<pkg>/package.json and stage the change.
  2. Run git commit as usual. The .hooks/pre-commit hook detects the staged bump, runs node scripts/backfill-changelog.js automatically, stages the resulting changelog/<pkg>/<version>.md, and lets the commit proceed. The bump and its release notes land in the same commit.
  3. Optionally review and edit the generated file before pushing. The script's body excerpts are the first lines of each commit message; for breaking entries especially, add migration notes by hand. Re-runs are idempotent (existing files are never overwritten), so hand-edits survive.
  4. Never edit changelog/<pkg>/<version>.md for a version that has already been published. Bump the version and edit changelog/<pkg>/<next>.md instead.

If the package has zero feat: / fix: / breaking: / perf: commits in the range (a release-only bump with no user-facing changes), the script writes nothing and the hook fails the commit. Either add a hand-written entry, downgrade the bump if it was unintentional, or git commit --no-verify to bypass.

The whole flow is tool-agnostic: the universal pre-commit hook fires for every git commit, regardless of who or what is running it. AI agents using Claude Code, Cursor, Copilot, Aider, etc. all get the same behavior, as do human contributors.

npm publishes AND GitHub Releases are auto-created from the same files. The .github/workflows/release.yml workflow watches for new changelog/**.md files added in a push to main. For each new file:

  1. scripts/publish-npm.js parses the frontmatter, checks npm view @webjsdev/<pkg>@<version>; if the version is not yet on the registry, it runs npm publish --workspace=@webjsdev/<pkg> --access=public. Idempotent: already-published versions are skipped.
  2. scripts/publish-release.js composes a tag <pkg>@<version> (e.g. core@0.6.0), title @webjsdev/<pkg> <version>, body (the markdown after frontmatter), then runs gh release create. Idempotent: existing release tags are skipped.

npm runs first; if it fails (auth, network, transient registry error), the GitHub Release step is skipped and the workflow fails. After fixing, a re-run picks up where it left off: the npm-side check makes the completed package a no-op and only the missing release lands.

The workflow uses NPM_TOKEN (repo secret) and the auto-provisioned GITHUB_TOKEN. Free for public repos.

Update the global CLI after the publish lands. The maintainer scaffolds and dogfoods with the globally installed webjs CLI, which lags a release until refreshed. So once release.yml has published (verify npm view @webjsdev/cli version matches the released version), refresh the global CLI on every manager: npm update -g webjsdev, bun add -g webjsdev, and mise use -g npm:webjsdev@latest. Run them AFTER the publish, never at merge time (they pull the LATEST PUBLISHED version). The mise use line is the one that actually moves a mise-shimmed webjs (a shim on PATH ahead of the npm/bun globals); verify with mise which webjs. This is reminded automatically by the .claude/hooks/release-global-update.sh PostToolUse hook, which fires when a chore/release-* PR merges (escape hatch WEBJS_NO_RELEASE_GLOBAL_UPDATE=1, regression test test/hooks/release-global-update.test.mjs).

Dev error overlay: rich, pushed live over SSE (dev-only) (#264)

In development, three error sources push a structured error frame to the open tab over the existing live-reload SSE channel (a distinct webjs-error event, NOT EventSource's native error), and a small dev-only client renders a plain-DOM overlay without a manual reload: an SSR render crash (a page / layout throws, or the no-browser-globals walker trips), a non-erasable-TypeScript strip failure (which breaks only the CLIENT module fetch, so the page still SSRs but hydration is silently dead, the exact gap this closes), and a failed rebuild (previously only logged server-side). The overlay carries the message, the parsed file:line:column, a source code frame of the offending line with context, and for a TS strip the no-non-erasable hint surfaced in the UI rather than buried in a JS comment. A successful rebuild clears it (the reload also dismisses any on-screen overlay), and the current frame is replayed to a tab connecting after the breaking edit.

The overlay client uses textContent throughout (never innerHTML), so the error content cannot inject markup. It is strictly dev-only: reportDevError early-returns when !dev, /__webjs/reload.js 404s in prod, and the prod 500 stays terse (only error.message, never the stack or a file path), so no source leaks. An embedding host can observe the same frames via the onDevError option on createRequestHandler / startServer. Mechanism: buildDevErrorFrame in packages/server/src/dev-error.js, reportDevError + the SSE push in packages/server/src/dev.js, the SSR-catch hook in packages/server/src/ssr.js.