Skip to content

Release 0.25.0: current local models, local first, comparable proof - #34

Merged
mattbaconz merged 16 commits into
mainfrom
mbcz/release-0.25.0
Oct 17, 2026
Merged

mattbaconz merged 16 commits into
mainfrom
mbcz/release-0.25.0

Conversation

@mattbaconz

Copy link
Copy Markdown
Contributor

Release 0.25.0: current local models, local first, comparable proof.

  • phonton models catalog suggests 2026 local models by total memory (qwen3.5:4b, qwen3.5:9b, gpt-oss:20b, qwen3.6:35b-a3b; gemma4:12b, qwen3.6:27b as alternatives). The qwen2.5-coder entries are gone.
  • With no provider key, Phonton defaults to a local model instead of Anthropic; doctor and models name phonton models setup MODEL first.
  • Fit estimates read the KV cache size from model metadata, keep a fixed 1 GiB runtime reserve, and admit partly offloaded models.
  • Local goal runs verify jest projects (offline npm ci, JSON report, V8 coverage of edited sources). Lockfile cap raised to 2,048 entries.
  • phonton review / why-tokens list the files each prompt actually inlined, with per-prompt token estimates.
  • phonton models setup --help (and --help after any models verb) prints usage instead of taking --help as a model name and downloading the runtime.
  • benchmarks/polyglot: Aider Polyglot slice runner (Phonton, bare model, OpenCode) reporting pass rate and false-pass rate, plus Kaggle and Modal GPU jobs.

Release notes: release-notes/v0.25.0.md. Changelog: ## 0.25.0.

Local checks (Windows): cargo fmt --all -- --check, release build, phonton version = 0.25.0, models setup --help exits 0 without downloading, npm run test:npm-wrapper, npm pack --dry-run (0.25.0, 9 files). The Polyglot runner completed a smoke run on Windows (RTX 4050, qwen3.5:4b) for all three arms and languages; the full slice is running and its results will be published separately.

- Model catalog: hardware-tiered picks by total memory (largest GPU or half
  of RAM): qwen3.5:4b under 6 GB, qwen3.5:9b from 6 GB, gpt-oss:20b from
  16 GB, qwen3.6:35b-a3b from 24 GB; gemma4:12b and qwen3.6:27b as
  alternatives. Replaces the qwen2.5-coder defaults and hints.
- No provider key: default to a local model; doctor and models name the
  setup command for this machine before provider keys.
- review / why-tokens: report the files each prompt inlined (whole file or
  excerpts) with a per-prompt token estimate, not only retrieved snippets.
- benchmarks/polyglot: Aider Polyglot slice runner (Python, JavaScript,
  Rust) scoring Phonton, the bare model and OpenCode on the same model and
  prompt for pass rate, false-pass rate, tokens and wall time, plus a
  reference-model scorer check.
Builds Phonton, pins Ollama 0.34.2, Node and OpenCode, calibrates the model
at 16384 tokens and runs one runner process per language within the 12 h
session, keeping artifacts and the built binary for reuse.
A candidate copy leaves node_modules out, so jest could never run there and
every JavaScript Polyglot run ended unverified. Keep the reviewed offline
npm ci preparation (registry lockfile with integrity hashes, no lifecycle
scripts, recorded as preparation, never as verification) and let a direct
jest check use it:

- node node_modules/jest/bin/jest.js --json, with an allowlist of flags, is
  a recognised verifier. Exit 0 passes only when stdout is jest's JSON report
  with no failure or interruption, every test file inside the candidate and
  at least one passing test; V8 coverage must show each edited source loaded.
  Any other jest form or a skipped-only run stays Not run.
- The lockfile entry cap rises from 512 to 2048 inside the existing 2 MiB
  bound; jest + babel locks 580.
- babel.config.* and .babelrc* are protected from model edits.
- fixtures/jest-sum and an ignored real-jest test cover fail, pass and
  skipped-only runs.
- The Polyglot runner commits the shared lockfile into JavaScript
  workspaces and gives Phonton the jest check; the JavaScript no-check
  exception and its README limit are gone.
- Size the KV cache from Ollama's reported attention shape (per-layer KV
  heads, so qwen3.5's linear-attention layers cost nothing: 32 KiB/token for
  4b and 9b) instead of 1 GiB per 4096 tokens plus 25% of the weights. Missing
  shape metadata keeps a conservative 256 KiB/token fallback. Add a fixed
  1 GiB runtime reserve (measured: qwen3.5:4b at 16384 tokens holds 4224 MiB
  of VRAM = download + KV + 479 MiB).
- Accept a model that fits the largest GPU plus available RAM (less the 2 GiB
  host reserve) as partly offloaded, so the 6 GB tier's qwen3.5:9b is admitted
  once about 4 GiB of RAM is free. GPUs are still never summed.
- A copy of the same model (name and digest) loaded at another context counts
  as free memory in calibration and goal admission; automatic calibration
  keeps a larger context that copy already holds in VRAM.

Live on RTX 4050 6 GB / Ollama 0.34.2: `phonton models calibrate qwen3.5:4b
16384` is admitted cold and with the model warm at 4096 (was refused).
A root "test": "jest" script used to be withheld as a diagnostic-only npm
test, so ordinary jest projects needed an explicit --check. When jest is a
declared dependency, there are no pretest/posttest hooks and the script has
only literal candidate test paths, the plan now proposes
node node_modules/jest/bin/jest.js --ci --json with those paths, and the
reviewed offline npm ci preparation follows. Flags, globs and shell
operators keep the script withheld.
gpu_job.py (was kaggle_job.py) splits an idempotent setup from the run and
takes its paths, model and budget from the environment. modal_job.py bakes
the setup into a Modal image from a clean checkout, turns --max-dollars into
the runner's time budget at list prices and kills the job shortly after.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T07:16:21.664707Z 884b4f0 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 884b4f087c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread phonton-cli/src/models_cli.rs Resolved
Comment thread phonton-local/src/lib.rs Resolved
@mattbaconz
mattbaconz merged commit 138ea63 into main Oct 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant