Skip to content

README.md

Latest commit

 

History

History
98 lines (81 loc) · 16.9 KB

README.md

File metadata and controls

98 lines (81 loc) · 16.9 KB
Raw

Documentation Index

Architecture and Design

  • architecture.md -- Component descriptions, dependency graph, project structure
  • design/2026-02-21-incremental-compilation.md -- Historical design record of the salsa pipeline: symbolic bytecode, per-variable tracking, LTM integration (the pipeline as it stands is in src/simlin-engine/CLAUDE.md and the compiler-unification plan)
  • design/conveyors.md -- XMILE conveyor support: complete specification of syntax, per-DT simulation semantics, leakage/initialization formulas, spread inputs, arrays, and engine integration
  • design/diagram-scene.md -- The diagram scene display list (version 1): the engine's resolution-independent drawing of one stock-and-flow view for native renderers -- elements, shapes, paints, labels, sparklines, and the stability rules that keep it the SVG renderer's twin
  • design/engine-performance.md -- Engine compile/simulate profile (C-LEARN), how to measure a change (the instruction, LTM, artifact, sweep and dump channels), implemented optimizations, and remaining proposals
  • design/layout-quality.md -- Diagram layout quality: the rate-based metric over the drawn scene (terms, weights, defect overlays, calibration against taste checks and visual judgments), the on-demand eval harness (graded corpus, production timing, taste battery, run comparison), and the improvement loop
  • design/ltm--loops-that-matter.md -- LTM implementation design: data structures, synthetic variables, module handling, array/element-level support, and post-simulation loop discovery (candidate generation, retention against the loop universe, ranking and the coverage-aware cap)
  • design/ltm-always-on.md -- Proposed always-on LTM constraints: faithful counterfactuals, immutable run analysis, completeness and numerical validity, cheaper instrumentation and storage, bounded discovery, and acceptance evidence
  • design/mdl-parser.md -- Vensim MDL parser design history and implementation notes
  • design/queues.md -- XMILE queue support: complete specification of queue stocks, FIFO discipline, conveyor/queue coupling, overflow flows, and engine integration
  • design/vdf.md -- VDF binary format specification and parser design

Development Standards

  • dev/commands.md -- Build, test, lint commands for all languages
  • dev/deploy.md -- Deploying the web app to Google App Engine: runbook, env vars, smoke test, rollback
  • dev/benchmarks.md -- Running and profiling benchmarks (criterion, valgrind, perf, gperftools)
  • dev/rust.md -- Rust development standards
  • dev/typescript.md -- TypeScript/React development standards
  • dev/python.md -- Python (pysimlin) development standards
  • dev/workflow.md -- Problem-solving philosophy and TDD workflow
  • dev/design.md -- Product design context: users, brand, design principles, tokens, accessibility
  • dev/harness.md -- Improving the harness: how guidance, checks, and agent tooling change and prove their worth
  • dev/frontend-audit-2026-06.md -- June 2026 frontend bug audit of src/app and src/diagram: findings, fix status, tracked follow-ups
  • dev/flow-editing-audit-2026-06.md -- June 2026 deep audit of interactive flow editing: rename/stale-view corruption chain, name-editor UX, routing robustness, group-movement dedup

Project Management

  • tech-debt.md -- Known technical debt items with measurement commands
  • design-plans/ -- Design plans (architecture and phasing for major efforts)
    • design-plans/2026-08-25-compiler-unification.md -- Engine compiler de-duplication, landed: one fragment compiler (lower_fragment/DepShape), one BuiltinFn::signature table, one temp allocator and materialization pass, one axis matcher (match_axes), structured DepRefs, one Variable, one Diagnostic, a (variable, project)-keyed parse with AST-carried captures and implicit modules, per-variable lowered memos, Loops That Matter as a consumer of each; every pinned semantic divergence and a per-commit measured ledger
    • design-plans/2026-09-04-link-scores-from-fragments.md -- LTM link scores synthesized from the target's compiled fragment instead of generated equation text: the rewrite, its per-link keying, the formula families as typed builders, one LinkScore opcode, a native post-pass; the roofline for "LTM always on"
    • design-plans/2026-09-07-ltm-single-mode.md -- Single-mode LTM: score every causal edge, compute every loop score after the run from the recorded series (one pipeline for every surface), Johnson only for the structural preview and ids, pins injected into the pipeline, a reducer as a real loop node, static polarity and the shortest-path fallback deleted, sub-model pathway scores emitted only for instances that can lie on a loop; 8 phases
    • design-plans/2026-09-10-diagram-editing-core.md -- Diagram editing core: one flow-geometry owner (terminals, route/routeEnd, offsetSegment, heal), one pure gesture planner whose frame at pointer-up is the commit, one controller executor over committed + pending[] edits with rollback; the G1-G8/M1-M3/E1-E6 invariants the tests assert; 7 phases
    • design-plans/2026-09-20-agent-tool-surface.md -- The agent tool surface in the engine: a tool catalog with JSON Schemas and a session per model (revision, evidence ids, named runs) that every host mounts -- libsimlin, the MCP servers, pysimlin; the read tools, experiments, loops, the validation battery, planned edits behind a gate, and verified findings; 7 phases
    • design-plans/2026-09-12-editing-core-in-rust.md -- The editing core moves into the engine for native hosts and tool edits: simlin_engine::editing (flow geometry, tiered hit testing, taps, a gesture session per drag, frame previews, delete and rename planning), the EditView patch op that derives model ops at apply time, the libsimlin gesture surface, how a host drives it, what stays with each host, and the touch-first differences from the TypeScript planner; 4 phases
    • design-plans/2026-08-26-compiler-unification-phase7-investigation.md -- Companion to the compiler unification plan: every parse-time decision that reads model state, where each moves under a (variable, project) parse key, every text/name identity site of synthesized helpers, the salsa keys, the capture runlist contract
    • design-plans/2026-08-17-pysimlin-widget.md -- pysimlin file-backed models + anywidget in-notebook editor (file on disk as sync authority)
    • design-plans/2026-04-05-server-rewrite.md -- Local-first simlin-serve binary: filesystem-backed editor + in-process MCP
    • design-plans/2026-04-25-ltm-per-ref-elem-graph.md -- Per-reference element causal graph: classify each AST reference by access shape, emit truthful per-reference element edges (includes a post-Phases-1-5 measurement postscript)
    • design-plans/2026-05-06-ltm-482-variable-level-loop-enumeration.md -- Tiered LTM loop enumeration: variable-level Johnson first, expand only the cross-element subgraph
    • design-plans/2026-05-09-ltm-503-cross-element-agg.md -- Cross-element LTM scoring: per-element arrayed-target partials, element-level cross-element loops, array reducers as aggregate nodes
    • design-plans/2026-05-11-ltm-arrays-hardening.md -- Arrayed/cross-element LTM hardening: unify the reference-site walkers behind one classification IR (#520), then layer eight fixes (#487, #511, #510, #514, #515, #483, #502, #492)
    • design-plans/2026-05-13-macros.md -- Vensim macro support: macros as a data-driven generalization of the stdlib module mechanism, persisted via a MacroSpec marker on Model; 7 implementation phases
    • design-plans/2026-05-19-clearn-residual.md -- Close C-LEARN's residual (#590/#591) as general Vensim import/simulation primitives: arrayed inline graphical functions, import-time macro shadowing, user-macro INITIAL recurrence, residual attribution; 5 phases
    • design-plans/2026-05-20-wasm-backend.md -- WebAssembly code-generation backend: compile a model to one self-contained wasm module as an alternative to the bytecode VM (for fast interactive re-simulation), validated to full VM parity; 8 phases
    • design-plans/2026-05-22-engine-wasm-sim.md -- Integrate the wasm backend into @simlin/engine as a selectable engine (Model.simulate({engine:'wasm'})): vm-vs-wasm demux below the Sim facade in DirectBackend, a resumable blob run ABI for runTo, and a node VM-vs-wasm benchmark; 4 phases
    • design-plans/2026-05-22-layout-quality-eval.md -- Layout quality evaluation + hill-climbing harness: a pure geometry-accurate LayoutMetrics (overlap/sprawl/accurate-arc crossings) and benchstat-style seed-distribution stats, an on-demand corpus sweep that renders and scores layouts against human references, and Rung 0 (rank seeds by weighted_cost); 5 phases
    • design-plans/2026-05-26-wasm-ltm.md -- LTM on the wasm backend: thread enableLtm into simlin_model_compile_to_wasm, add Results-from-slab + from-wasm analyze FFI, and wire Sim.getLinks / Run.links to run the shared analytic core over a wasm sim so its links match the VM (no protocol change); 6 phases
    • design-plans/2026-06-01-ltm-653-pinned-loop-dimensions.md -- Pinned-loop dimension classification (GH #653): route pinned cycles through the same classify_cycle machinery as enumerated loops, add slot-aware (Equation::Arrayed) loop-score generation shared by both the pin path and the enumerator's A2A-collapse, C-LEARN end-to-end validation; 6 phases
    • design-plans/2026-06-04-time-invariant-hoisting.md -- Time-invariant variable hoisting (GH #712): run-invariance classification, flow-runlist partition, split metadata (B1); B2 execution sketch
    • design-plans/2026-06-11-ltm-shape-expressiveness.md -- LTM shape expressiveness (epic #488 Phase 1): one per-axis access model (AxisRead with subset-Reduced, per-source AggNode slices, RefShape::PerElement) consumed by classification/expansion/emission/naming through one row derivation; resolves #765/#766/#767/#764/#771/#751/#757/#525/#526/#769 in 7 tasks
    • design-plans/2026-08-10-ltm-discovery-union-enumeration.md -- Union-graph circuit enumeration as discovery's primary candidate generator: enumerate the ever-simultaneously-active cycles of the recorded link scores once (activity-bitset pruning), streaming retention with full-universe denominators, and enumeration_complete observability. Superseded on the fallback and observability points by the 2026-08-17 plan below
    • design-plans/2026-08-17-ltm-discovery-exact.md -- Exact union-graph enumeration as discovery's sole primary candidate generator (no singleton cycles, edge-row emission, single-pass retention, universe denominators), shortest-path (-log|link score|) fallback replacing the per-step DFS, coverage-aware cap, enumeration_complete surfaced end to end; 6 phases
  • notebooks/ -- Notebook GENERATORS. Each build_*.py constructs an analysis notebook cell-by-cell and executes it against the current engine, with a verify_*.py that fails on cell errors, silent cells, missing figures, and missing claim markers; the executed .ipynb is a gitignored artifact, so an audit is always re-derived rather than read. build_ltm_discovery_audit.py (--model clearn|cross_agg|wrld3) audits LTM discovery: it dumps the engine's exact link-score edge set, re-enumerates the loop universe in pure Python, and diffs universe size, scores, ranking, and per-step dominance against what the engine reported
  • plans/ -- Implementation plans (active and completed)
  • test-plans/ -- Human verification plans for completed features
    • test-plans/2026-05-22-engine-wasm-sim.md -- Manual verification for the @simlin/engine selectable wasm engine (Model.simulate({engine:'wasm'})): re-running the automated gates, driving the gated/#[ignore]d heavy tests, and the human-judged extras (interactive scrubbing feel, VM-vs-wasm benchmark numbers); all 25 ACs already have automated coverage
    • test-plans/2026-05-22-layout-quality-eval.md -- Manual verification for the layout-quality eval: running the on-demand corpus sweep and inspecting its target/layout-eval/ artifacts (metrics.json, the worst-first contact-sheet), plus the human-judgment calibration gate (best/median/worst ordering, reference-vs-auto scoring, weight magnitudes)
    • test-plans/2026-05-26-wasm-ltm.md -- Manual verification for LTM on the wasm backend: re-running the automated gates (simulate_ltm_wasm, the libsimlin from-wasm analyze FFIs, wasm-ltm.test.ts, worker-wasm.test.ts), driving the ignored heavy discovery twins (C-LEARN, World3), and the human-judged end-to-end Node-REPL scenarios (Model.simulate({engine:'wasm', enableLtm:true}), sim.getLinks()/Run.links parity, the WorkerBackend round-trip, and the no-silent-fallback contract on an Unsupported LTM model); all 16 ACs already have automated coverage
  • implementation-plans/ -- Detailed phase-by-phase implementation plans, created during plan execution

Security

  • threat-model.md -- simlin-serve V1 threat model: trust boundary, defenses (loopback bind, bearer token, DNS-rebinding mitigation, cross-origin defense, supply chain), and out-of-scope items

Domain Knowledge

  • reference/xmile-v1.0.html -- XMILE interchange format specification
  • reference/vensim-macros.md -- Vensim macros (:MACRO:): definition/call syntax, semantics (per-invocation stock state, locality, recursion), XMILE <macro> representation, the mapping xmutil (the converter that produced the .xmile fixtures) gives them, and implementation implications
  • reference/ltm--loops-that-matter.md -- Loops That Matter technique: link scores, loop scores, algorithm reference
  • array-design.md -- Array/subscript design notes

Research Papers

Each paper has a detailed markdown summary -- consult these first for paper-specific details before the PDFs.

Schemas

Other