Skip to content

Latest commit

 

History

History
262 lines (212 loc) · 31.1 KB

CONFIGURATION.md

File metadata and controls

262 lines (212 loc) · 31.1 KB

Configuring D1A

Every setting D1A reads, in one place. It is written for people and for coding agents: each entry gives the default, what the setting changes, and when to change it. tests/test_configuration_doc.py fails when the code reads a D1A_* variable or a server flag that this page does not list.

D1A is configured three ways:

  • command-line flags on python -m d1a.serving.serve and the other entry points;
  • environment variables (D1A_*), read when the process starts;
  • files the server re-reads while it runs: the outcome calibrator (D1A_OUTCOME_CALIBRATOR).

Nothing else changes D1A's behaviour. The defaults are the measured, released configuration. Leave a setting at its default unless this page names a reason to change it.

Check what is in force

GET /v1/models on a running server returns the effective configuration:

  • the checkpoint (run), device, backend and dtype;
  • the temperature and per-use-case temperatures, and whether it is calibrated;
  • learning: whether decisions are logged, the calibrator file, and the questions it corrects;
  • the prefix cache's size and limits, and recent batch latency.
curl -s 127.0.0.1:8009/v1/models | python -m json.tool

Before starting a server, python -m d1a.serving.serve --show-config prints every D1A_* setting with its value, whether it was set or is the default, and the module that reads it (secrets masked), plus where the self-learning settings come from. The list lives in d1a/core/settings.py; every module reads its settings through it.

I want to ...

Goal Setting
serve a released model --run JohnP1/d1a-e4b-mlx-q8@v0.6.1 (Apple Silicon) or --run JohnP1/d1a-e4b@v0.6.1 (CUDA, CPU); pin the tag
reach the server from another machine or a container --host 0.0.0.0, and set D1A_API_KEY
require a key D1A_API_KEY=<key>; clients send Authorization: Bearer <key>
call the server from a web page on another origin D1A_CORS_ORIGINS=https://app.example (comma-separated)
free memory when idle --idle-unload 600
answer about photos, voice and video install the media extra and serve an MLX export that carries media/
learn from outcomes D1A_FEEDBACK_LOG, POST /v1/feedback, python -m d1a.learning.feedback promote, D1A_OUTCOME_CALIBRATOR (see Self-learning)
keep test or demo traffic out of the learning log request header x-d1a-decision-log: off
prove which weights, temperature and cache produced an answer request header x-d1a-audit: on (Audit an answer)
reproduce reported numbers exactly D1A_DTYPE=fp32 (and D1A_BACKEND=torch on Apple Silicon)
give agents D1A as tools python -m d1a.agents.mcp_server with D1A_URL or D1A_RUN
score a remote D1A endpoint python -m d1a.eval.benchmark --remote <url> with D1A_REMOTE_API_KEY

The server: python -m d1a.serving.serve

Install with uv sync --extra serve. Add --extra mlx on Apple Silicon, --extra media for photos, voice and video, and --extra mcp for the MCP server.

Flag Default What it does
--run runs/d1a The checkpoint: a Hub id with an optional @tag (JohnP1/d1a-e4b-mlx-q8@v0.6.1), a local training run (holds head.pt) or an MLX export folder. Pin a tag in production.
--fallback runs/smoke Served when --run is a local path that does not exist.
--host 127.0.0.1 The interface to bind. 0.0.0.0 serves beyond this machine; set D1A_API_KEY when you do.
--port 8008 The port. The docs and the playground use 8009.
--device auto auto tries cuda, then mps, then cpu, and skips a device whose test kernel fails. Name a device to insist on it.
--idle-unload 0 Seconds without a request before the model is dropped from memory; the next request reloads it. 0 keeps it loaded.
--show-config off Prints the settings in force (above) and exits without loading a model.
Variable Default What it does When to change it
D1A_API_KEY unset (open) Requires Authorization: Bearer <key> on /v1/* and /metrics. Always when the server is reachable beyond localhost. Never commit the key.
D1A_CORS_ORIGINS unset (none) The web origins a browser may call the server from, comma-separated, each scheme://host[:port] with no path or trailing slash (null for a page opened from a file); * allows any. Unset, a page on another site cannot call it; programs and same-origin proxies (the playground's) are not affected. d1a.serving.media reads it too. When a web page on another origin calls the server directly. Avoid *: any site open in a browser on a machine that reaches the server could then call it, POST /v1/feedback included.
D1A_PREFIX_CACHE 4 States (documents) kept in the prefix cache, so repeated questions on one document skip its prefill. 0 turns it off. Raise it for many clients asking about different documents; 0 for an exact-memory budget.
D1A_PREFIX_MAX_TOKENS 65536 State tokens the cache holds in total, least recently used evicted first. A longer state is not cached. Lower it on small-memory machines.
D1A_PREFIX_MIN_TOKENS the model's (0 for MLX and hybrid backbones, 384 for attention-only torch models) States shorter than this are not cached. Rarely.
D1A_DATE_FACTS 0 1 adds computed date facts to a state that holds two or more dates (d1a.core.api.with_date_facts). When questions compare dates; measure first, as the reported numbers are without it.
D1A_FEEDBACK_LOG unset See Self-learning.
D1A_OUTCOME_CALIBRATOR unset See Self-learning.
D1A_CHAT_CACHE_MB 1024 Memory for the chat prompt cache: the KV cache of up to 16 recent chat prompts, so a chat that continues one (an agent's next step, a new session with the same system prompt) prefills only its new tokens. With D1A-E4B-it, 1024 holds three or four 20k-token prompts. 0 turns it off. Raise it when many agents or long prompts share a big machine; lower it on a small one. 0 for an exact-memory budget, or when clients must not share a server's cache: cached_tokens tells a client that the server has seen its prompt's start before.
D1A_CHAT_CONTEXT unset (the model's context: 131,072 for D1A-E4B-it) The longest prompt plus max_tokens a chat may ask for. A longer request gets an immediate 400 (context_length_exceeded), which clients such as Hermes answer by compressing. Set it below the model's context when several clients share the server: one chat holds the model while it reads its prompt (~230 tokens/s on an M4, so 32k tokens ~2 minutes, 131k ~9 minutes).
D1A_CHAT_LOG 1 Prints a line per chat: prompt tokens, how many came from the prompt cache, completion tokens and time (Logs). 0 turns it off. 0 to keep the output to request lines only.
D1A_CHAT_NAME d1a-chat The model id /v1/chat/completions answers as and /v1/models lists. Any id a client sends is answered. When an OpenAI client insists on a particular id.
D1A_EMBED_GGUF unset (off) A GGUF embedding model /v1/embeddings serves through llama.cpp (the embed extra), e.g. nomic-embed-text-v1.5.Q4_K_M.gguf. To serve embeddings, e.g. for mem0 (Chat with D1A).
D1A_EMBED_NAME text-embedding-nomic-embed-text-v1.5 The embedding model id /v1/models lists (LM Studio's id for nomic, so clients set up for it need no change). With a different embedder.

Fixed in code, not settings: up to 64 requests per batch (MAX_BATCH), a 75 s keep-alive, and the model list (/v1/models lists d1a-latest, but any model name in a request is answered).

Logs

The server writes its log to standard output and standard error. It keeps no log file of its own: whatever starts it decides where the output goes. On the serving host, the LaunchAgents write it to ~/Library/Logs/d1a-model.log (below).

Line When Holds
startup lines once, at start the run, device, backend and dtype, the load time, warnings (an uncalibrated checkpoint, a device that failed its test); --show-config prints the settings instead of serving
INFO: <client> - "POST /v1/systemone HTTP/1.1" 200 OK each request (uvicorn) the client address, method, path and status; never the body
<date> <time> chat: N prompt tokens (M cached), K completion tokens, T s[, tools][, empty answer after <roles> | , cut off by max_tokens before any text] each chat, unless D1A_CHAT_LOG=0 counts and time only; never the prompt or the answer. M is the prompt tokens taken from the prompt cache. An answer with no text and no tool call ends with the roles of the last three messages (for example user > assistant(tool_calls) > tool; a role outside the OpenAI roles is logged as other), never their text; one cut off by max_tokens before any text says so instead
a traceback an error the code path that failed

None of these hold request or answer text, so they are safe to keep and to share in a bug report. About 90 bytes a request: a busy day of 10,000 requests adds under 1 MB. Rotation is left to whatever keeps the log.

The decision log (D1A_FEEDBACK_LOG, Self-learning) is separate and off by default. When it is on, it does hold each request's text and answers, because outcomes are matched to them.

Audit an answer

Send the request header x-d1a-audit: on to /v1/systemone or /v1/systemone/media, and the response gains an audit block that says what produced the answer, for a port, a bug report or a support ticket. Without the header the response is unchanged. It holds hashes and settings, never the request's or the answer's text:

Field What it is
d1a, run, backend, dtype the package version, the checkpoint as served (--run), and how it runs
weights_sha256 the adapter's sha256, or for an MLX export one sha256 over its shards (the first audited request computes it, seconds for an MLX export)
state_sha256 the sha256 of the state text the model read (after D1A_DATE_FACTS, when on); media_sha256: the sha256 of the photo, clip or video
temperature, checkpoint_temperature, temperature_source the temperature the answer was read at, the checkpoint's own, and checkpoint or use_case:<name>
outcome_calibrator the questions the outcome calibrator recalibrated (empty when none)
rules, flags per question, the decode rules that fired and the flags served
passes per model pass (one, or one per ask_if level): tokens, prefix_cache_hit, cached_tokens (the state tokens read from the prefix cache)
logged whether the decision went into the decision log

The Python and JS clients send no custom headers; use any HTTP client (curl, requests, fetch).

Model loading (D1A_* load options)

d1a.backends.checkpoint.LoadOptions.from_env reads these for the command-line tools (the server, the benchmark). Library code passes LoadOptions(...) instead. Without a setting, the server picks the defaults shown.

Variable Default What it does When to change it
D1A_BACKEND auto in the server torch, mlx or auto. auto uses MLX on Apple Silicon for Gemma 4 and Qwen3.5 bases when mlx-lm is installed and fp32 was not asked for. An MLX export always runs on MLX. torch on a Mac to reproduce the reported (torch) numbers.
D1A_DTYPE bf16 on cuda and mps, fp32 on cpu bf16, fp16 or fp32. fp32 is the exact path every reported number uses. bf16 halves memory and is 2–4.5× faster, with probabilities within ~0.01. Ignored on MLX. fp32 to reproduce numbers.
D1A_ATTN sdpa on mps and cuda, eager on cpu The attention kernel: sdpa or eager. Ignored on MLX. Rarely.
D1A_MERGE 1 0 keeps the LoRA adapter unmerged. On MLX it is then attached as LoRA layers that the server can switch off per request, giving back the base model (#274); on PyTorch the PEFT adapter stays separate, so a LoRA run on an instruction-tuned base chats too. Debugging, or an instruction-tuned base that also chats (an export made with export_mlx --unmerged does this by itself).
D1A_LORA_SCALE 1 Interpolates between the base model (0) and the fine-tuned weights (1). Experiments only.
D1A_TEMPERATURE the checkpoint's Serves every request, use cases included, at this temperature. 1.0 gives raw logits. Experiments only: the checkpoint's own temperature is fitted and released with it.
D1A_CUDA_GRAPHS on for cuda in the server 0 declines CUDA-graph replay of the hybrid backbone's passes. Debugging on CUDA.
D1A_FUSED on for cuda when available 0 declines and 1 insists on the fused Qwen3.5 Triton kernels (needs flash-linear-attention at the pinned version). Debugging on CUDA.
D1A_PLE_FLASH on MLX only: reads Gemma 4's per-layer embeddings from the weight files per request instead of holding them, about 1.3–1.5 GB less memory with identical probabilities. 0 keeps them in memory. Rarely.
D1A_SHAPE_BUCKET 64 Torch on MPS: pads sequences to a multiple of this, so compiled kernels are reused. 1 disables it. Rarely.

Self-learning

The loop: log each decision → report its real outcome → fit a calibrator on the outcomes → promote it only through a statistical gate. Details are in SELF_LEARNING.md and in the d1a.learning.feedback docstring.

Setting Default What it does
D1A_FEEDBACK_LOG=<path> unset (off) Logs every decision to this append-only JSONL file, and gives each answer a decision_id. Enables POST /v1/feedback. To start a new log, move the file aside (mv) rather than copy over it: the server reads a log from its start again when the file is a new one.
header x-d1a-decision-log: off logged Answers the request but keeps it out of the log. Use it for demos, smoke tests and anything no outcome will follow.
POST /v1/feedback — {"decision_id", "labels": {question_id: answer}, "src": "human" or "reviewer", "group": "<e.g. owner/repo#123>"}. When sources disagree on a question, the most trusted one's label wins. Returns 404 when logging is off, and 422 for a decision_id the log does not hold. Anyone who can reach the server can report outcomes, so set D1A_API_KEY when it is reachable beyond localhost.
D1A_LOG_REWRITE unset (the log keeps each request's text as sent) A private JSON list of {"pattern": regex, "replace": text} rules, applied (case ignored) to the state, questions and answers of every decision the log records, so names the log must not keep are written in general wording. Decision ids, the run, outcome groups and meta stay as they are, because outcomes are matched by them. Refused at start: a replacement with a backslash, a replacement its pattern matches again (a replayed state must reword to itself), or, with D1A_DATA_BLOCKLIST set, a listed entry no rule rewrites. Reword an existing log with python -m d1a.learning.feedback reword <log> --out <new> and move the new file into place (mv; the server reads a new file from its start), and rebuild the served model's trained manifest with trained ... --reworded so the replay screen hashes states the same way.
D1A_OUTCOME_CALIBRATOR=<path> unset (off) Applies this calibrator to yes/no and choice answers. The server reloads it when the file changes, so promotion needs no restart.
D1A_LEARNING=<path> unset (today's behaviour) The self-learning settings file (below), read by promote, replay and tick on every run, so a change needs no restart. The server reads only its flags and rules, re-read when the file changes; decision logging stays D1A_FEEDBACK_LOG, because the outcome poster and /review depend on that log.
header x-d1a-replay-of: <decision id> — Sent by replay only. The server logs the answer as a decision of the model it serves now, marked as a replay of that earlier decision, so the replay takes its outcomes. /v1/systemone only.

The python -m d1a.learning.feedback commands each take the log as their first argument, and also --src (only outcomes from one source) and --run (only decisions one model made):

Command What it does
status <log> Counts resolved and pending decisions, per question.
void <log> --ids <id,...> --src <source> --reason <text> Withdraws one source's outcomes on those decisions that arrived before it (outcomes logged in error, e.g. a PR author's own labels taken for a person's correction). The log is only appended to; a later outcome from that source counts again.
records <log> --out <file> Turns resolved decisions into training records for d1a.training.train --data.
calibrate <log> --out <file> [--min-outcomes 20] Fits a calibrator on every outcome, with no gate.
promote <log> --calibrator <file> [--min-outcomes 20] [--held-out 0.3] --run <served run> Fits on part of the outcomes, split by group. It gates each question on the rest and rewrites the calibrator only for the questions that pass: log loss better, with a bootstrap interval clear of zero. Flags not given come from the settings file. Each question's report gives mde, the smallest log-loss gain its held-out set can detect.
config init|show|set key=value ...|validate [--file <path>] Creates, shows (with where each value came from and the last changes), changes or checks the settings file ($D1A_LEARNING). set validates the whole file first and writes nothing when a value is out of range.
trained --mix <mix.jsonl> [--mix ...] --run <repo@tag> --out <file> [--log <log>] The trained manifest of a model: every PR id and state hash in the mixes of its lineage (give every stage's mix). A mix that is not found makes it incomplete, and replay refuses it. --log gives records exported from a feedback log their PR id.
replay <log> --server <url> --trained <file> [--cap 50] [--pause 1] Re-sends the decisions an earlier model made, that have outcomes, to the live server, one at a time and only while no request waits, so the served model starts with every earlier outcome. It skips a decision whose PR or state is in the manifest, or that has no PR id (by default), and refuses a missing, incomplete or other model's manifest. The idle check can race with one live request, which then waits one replay. With D1A_DATE_FACTS=1 a replayed state gets today's date facts, not the original day's.
tick <log> --calibrator <file> --server <url> --trained-dir <dir> One run of the loop, meant every few minutes: replay after a model change, then promote when it is due (schedule), a report line in learning-status.json beside the log, and a webhook message on a promotion or when a question first has enough outcomes.

The settings file (D1A_LEARNING; config show prints it). Unset, or a key left out, means the default:

Setting Default Allowed What it does
outcomes.enabled true true, false Whether outcome collectors (the playground's poster) run.
outcomes.sources null (all) a list, e.g. ["reviewer", "human"], or per question, e.g. {"sev": ["human"], "*": ["reviewer", "human"]} ("*": every other question) The outcome sources that count for learning. Per question, a source can teach one question and not another: a review bot whose severity scale differs from your labellers' teaches change type but not severity. Outcomes that do not count are still logged.
promotion.auto true true, false false: the gate still runs and reports, and never writes the calibrator.
promotion.questions null (all) a list of question ids The questions that learn.
promotion.min_outcomes 20 ≥ 10 Fit outcomes a question needs before it is fitted at all.
promotion.held_out 0.3 0.1 to 0.5 The share of groups the gate holds out.
promotion.min_held_out 30 ≥ 10 Held-out groups the gate needs before it may promote. With fewer, a candidate fitted on a few dozen outcomes passes the bootstrap by chance far more often than its interval says (#364).
promotion.ci_level 0.95 0.9, 0.95, 0.99 The gate's bootstrap interval.
promotion.tolerance, promotion.bootstrap 0.01, 2000 0 to 0.1; 200 to 20,000 Frozen-suite tolerance; bootstrap resamples.
promotion.form "temperature" "temperature", "vector" How a choice question is recalibrated: one inverse temperature, or that plus a per-option bias, which also corrects a label mix the served traffic does not share with training.
promotion.split "random" "random", "time" How the gate holds out groups: by hash, or the newest ones (it then asks whether what older outcomes taught helps on newer ones, as a promoted calibrator is used).
promotion.serve_choice false true, false false: a recalibrated choice keeps the model's chosen option and only its probabilities move. true: the recalibrated top option is served.
promotion.hold {} {question: [options]} With serve_choice, the model's own choice is kept when it is one of these, e.g. {"sev": ["P0", "P1"]}, so a correction toward common classes never demotes a rare, costly one.
schedule.daily_at "04:00" HH:MM local time, or null tick runs the gate once a day after this time, once schedule.growth allows it.
schedule.after_new_outcomes 10 ≥ 1, or null tick also runs the gate after this many new outcomes, once schedule.growth allows it.
schedule.growth 0.5 0 to 4 Either trigger runs the gate only once the outcomes have grown by this share since its last run: after a run at 200, the next waits for 300. Every run can promote, so a run every 10 outcomes promoted noise in 36.5% and 56.5% of 200 null replays by 1,000 outcomes; at 0.5, with promotion.min_held_out 30, in 7% and 9.5% (#364). A changed promotion setting (form, split, questions) also waits for that growth; run promote to apply it sooner, or set 0: every trigger runs the gate.
replay.enabled, replay.on_model_change true, true true, false Replay earlier outcomes when the served model changes.
replay.exclude_no_pr_id true true, false Skip decisions with no PR id (they cannot be checked against training).
replay.per_tick_cap, replay.pause_s 50, 1.0 1 to 500; 0 to 60 s Replays per tick, and the pause between them.
reports.webhook_file null a path A file holding a webhook URL (Discord-compatible); the URL itself never goes in the settings.
reports.notify ["promotion", "min_reached"] from promotion, min_reached, every_gate, flag_alert What tick posts to the webhook. flag_alert: a tracked flag's alert, when it starts.
flags.<name> none {question, options, t, space: "raw", run, budget, fitted_on} A learned decision flag (#237): raised when the answer's top option is in options or their summed probability reaches t, read on the model's own probabilities and served only for run (repo@tag, or the absolute path of the local folder the server runs, as for an unmerged chat build). The server adds it inside that question's answer as flags.<name> (on, p, t, space) and keeps it in the log's meta; the answer itself never changes. Re-read when the file changes. Fit and gate one with python -m d1a.learning.flags (below), on labelled kits only, never on the live log.
flags.<name>.mode "serve" "serve", "shadow" "shadow" computes the flag and keeps it in the log's meta (with "shadow": true) but leaves it out of the response, which stays byte-identical. Labels collected then are ones the flag never influenced (#296).
flags.<name>.track none {window, every, clamp: [low, high], alert_rate} t follows the flag rate without labels (#296). tick refits it every every new logged decisions, as the 1 − budget quantile of the options' summed probability over the last window decisions of the flag's run, clamped to clamp. It counts no replays and one decision per distinct state. Below window decisions, the offline t stays. tick reports an alert when the last 250 decisions' flag rate is above alert_rate, or when t is at a clamp.
rules.<name> none {question, option, t, from, space: "raw", run, use_case, fitted_on: {kit, n, tested_on}} A served decode rule (#252): for a request that sends use_case, answered by run, when the model's choice is in from and its probability of option reaches t, the served choice becomes option and the answer carries rule (name, from, p, t). The probabilities, the confidence and the log's answers stay the model's; the log keeps the served answers in meta.served and the rule in meta.rules, and status reports the served choice beside the top one. A client that recomputes the top option from the probabilities disagrees with choice exactly where a rule fired. One rule per question and use case; fitted and tested offline on labelled rows, never on the live log. t is compared with the serving machine's own probability, and Apple chips differ slightly (v0.6 on an M1 Max and an M4: up to 0.012 apart), so an answer within about 0.01 of t can land on either side on another machine; check a rule's flips on the machine that serves it. Re-read when the file changes.

A file with an error is never used: jobs keep the last valid one (<file>.last-valid.json) and report the error, and every accepted change is appended to <file>.audit.jsonl.

python -m d1a.learning.flags fits and checks a flag offline on d1a.eval.benchmark rows (question, keys, label, p):

Command What it does
fit <rows.json> --name <n> --question <q> --options A,B --budget 0.15 --run <repo@tag> --kit <name> --created-max <ISO time> Sets t at the flag-rate budget (the 1 − budget quantile of the options' summed probability) and prints the flag to put in flags.<name>, with its recall, precision and flag rate.
gate <rows.json> --flag '<that JSON>' On a later, held-out kit: serve it only if it catches more positives than the top answer (95% bootstrap over groups clear of zero) within budget + 0.03.

How to set it up on your own deployment:

  1. Serve with D1A_FEEDBACK_LOG=runs/feedback/decisions.jsonl and D1A_OUTCOME_CALIBRATOR=runs/feedback/calibrator.json.
  2. Send the outcome back to POST /v1/feedback whenever you learn it, with a group (the PR, ticket or document it belongs to) and a src.
  3. export D1A_LEARNING=runs/feedback/learning.json and run python -m d1a.learning.feedback config init; change what you need with config set.
  4. Write the served model's trained manifest (trained --mix ... --run <served run>) into a folder, then schedule tick runs/feedback/decisions.jsonl --calibrator runs/feedback/calibrator.json --server http://127.0.0.1:8009 --trained-dir <folder> every few minutes. It replays after a model change and gates when schedule says so; a calibrator fitted on another model's answers is never applied to this one's.
  5. Read runs/feedback/learning-status.json (each question's outcomes and how many it still needs, and the last gate reports), and GET /v1/models → learning.calibrated_questions after a promotion.

A question needs --min-outcomes (20) outcomes before it is fitted, and the gate needs enough held-out groups to decide. With a few outcomes a day, expect weeks rather than days.

Photos, voice and video

Install the media extra. Serve an MLX export that carries media/ (E4B: v0.3 to v0.6.1; E2B: v0.2, v0.6 and v0.6.1, not v0.2.1-2epoch-calibrated), or a torch checkpoint (the encoders come from its base). POST /v1/systemone/media takes a System One request plus {"media": {"type": "image" | "audio" | "video", "data": <base64>}}. A video is read as 16 frames; its sound is not used. There are no further settings.

Agents (MCP)

python -m d1a.agents.mcp_server (the mcp extra) exposes D1A's agent decisions as MCP tools over stdio. The tools' and the gate hook's decisions go into the server's decision log like any other (D1A_FEEDBACK_LOG), so outcomes can be learned from; python -m d1a.agents.demo keeps its decisions out of it.

Variable Default What it does
D1A_URL http://127.0.0.1:8009 The D1A server the tools ask.
D1A_RUN unset Loads this checkpoint in the MCP process instead of asking a server.
D1A_API_KEY unset The bearer key the tools, the gate hook and the demo send to D1A_URL: set it to the server's D1A_API_KEY when the server requires one.
D1A_GATE_CARD unset The Claude Code gate hook (d1a.agents.claude_code_gate): the task the agent is on, as the gate reads it (unset: "work in <the session's directory>").
D1A_GATE_ALLOW 0 1: the gate hook passes D1A's "allow" on, so Claude Code skips its prompt (your own deny and ask rules still apply). Only for agents you trust D1A's gate with. 0: the hook only adds "ask" and "deny", and Claude Code's own rules decide the rest.
D1A_GATE_TIMEOUT 10 Seconds the gate hook waits for D1A before asking a person. Keep the hook's own timeout in settings.json above it.

Evaluation

python -m d1a.eval.benchmark scores a checkpoint (--run, using the load options above) or a running server (--remote <url> --remote-model <name>).

Variable Default What it does
D1A_REMOTE_API_KEY local The bearer key sent to --remote; match the server's D1A_API_KEY.

Training and evaluation data

Variable Default What it does When to change it
D1A_DATA_BLOCKLIST unset (no check) A private file of names D1A's data must never hold: an organisation, its products, people. One entry per line; # starts a comment; case is ignored. An entry's ASCII letter or digit ends match only at a word or identifier edge, so a short name does not hit inside a longer word (<name>y, or PATTERN for an entry pat) but does inside a code identifier (<Name>Tests, <name>_ios, IOS<Name>, <Name>2) and with a plural s/es after it. When set, python -m d1a.eval.suites freeze refuses to pin a partition that holds an entry; loading a suite partition (training, scoring, calibration, the mix tools) refuses it; and d1a.training.train refuses --data or any training record (replay included) that holds one, before training starts. A refusal names the file and line numbers, never the entry. training_config.json records how many entries were checked. Set it on every machine that builds, pins or trains on D1A data, pointing at a file kept out of the repository. Rewrite a refused line in general wording ("the agent kit", "the original author"), never with a made-up name.

The always-on playground deployment

The playground (jonpol01/d1a-playground) runs D1A as launchd services through mini.sh, configured by .demo/mini.env (MODEL_RUN, FEEDBACK_LOG, OUTCOME_CALIBRATOR, LABEL_OUTCOMES, ...). The playground's README documents those keys. ./mini.sh update applies a change. Each service logs to ~/Library/Logs/<service>.log: d1a-model.log (the model server, Logs), d1a-web.log, d1a-openai.log (the :1234 proxy), d1a-outcomes.log and d1a-promote.log (the learning loop).