Skip to content

feat: the mutating half — chart automation, connections connect/disconnect, dialog/strategy/playbackctl/log, layout - #5

Open
silentsudo-io wants to merge 19 commits into
eman007:mainfrom
silentsudo-io:feat/chart-cell-automation
Open

feat: the mutating half — chart automation, connections connect/disconnect, dialog/strategy/playbackctl/log, layout#5
silentsudo-io wants to merge 19 commits into
eman007:mainfrom
silentsudo-io:feat/chart-cell-automation

Conversation

@silentsudo-io

Copy link
Copy Markdown
Contributor

Thanks for merging #2 and #4 — this builds on both.

This is the mutating half of the bridge, plus the chart automation it enables. It is big, so the
sections below are independent enough to review (or reject) one at a time, and I'm happy to split it
into separate PRs if you'd prefer
— I kept it as one because this is the exact tree that has been
compiled and driven against a live NinjaTrader. Splitting it would mean sending you configurations I
have not actually run.

Everything here was verified by driving it, not by the call returning. That distinction is the
theme of the whole diff.


1. chart — list / add / remove / close, and --api discovery

--api is read-only and dumps what this build actually exposes (BarsProperties, Bars,
BarsPeriod, ChartBars methods, the Connection API). It earns its place immediately — it refuted
two assumptions before they became bugs:

  • Bars.IsResetOnNewTradingDay is read-only.
  • "Break at EOD" is settable nowhere: absent from BarsProperties, absent from BarsPeriod
    (unfiltered dump), read-only on Bars, absent from every chart template. A setter written against
    the obvious name would have resolved, changed nothing, and reported success.

2. chart --apply-template — indicators that actually run

The existing addIndicator path cannot bring an indicator to life: Indicators.Add +
SetState(Active) + RefreshIndicators leaves it at Configure, and the best advice it can give is
"re-add it from the UI." This hands NT the XML it wrote via TemplateLoadIndicators, so NT's own
loader runs the state machine.

Result on a bare chart: 0 → 14 indicators, all 14 at Realtime.

Three things it measures rather than assumes:

  • Which XElement? TemplateLoadIndicators(<Indicators>) works; the same method given the document
    root "resolved but count stayed 0". Candidates are tried and the winner is reported as via, so a
    silent no-op cannot read as success.
  • RefreshAllBars REGRESSES indicators to 0 running. Found by laddering each step separately:
    ApplyNinjaScripts → 2 running, RefreshIndicators(true,true) → 3, then RefreshAllBars0.
    It is excluded here. Heads-up: addIndicator still calls it as a fallback, so that path is
    suspect for the same reason
    — I have not changed it in this PR.
  • Activation is asynchronous. Indicators climb SetDefaults → Configure → DataLoaded → Historical → Realtime as bars load. Counting once, immediately, read 3 of 14 and would have reported a
    healthy chart as broken. It now settle-polls, the same way playbackctl --seek does.

3. chart --data-window

RangeType / DaysBack / BarsBack / MonthsBack / From / To — all [RW] (measured on two
builds). ChartBars exposes no reload of its own, so the apply is Chart.OnDataSeriesChanged, then it
settles on IsBarsLoading and reads every field back.

  • Only fields the caller named are sent, and only those move — no silent side effects.
  • A rejected field fails the whole call (succeeded=false, rejectedFields=N, CLI exit 2). The
    first version returned success while silently dropping an unparseable date, which a script would
    never have noticed. That bug is in the history on purpose.
  • Dates parse culture-invariantly: a window that means one thing on a US box and another on an EU
    box is a silently different backtest.

4. connections --connect / --disconnect

The read side has existed since 1.5.0 but nothing could raise a connection. Uses
Connection.Connect(ConnectOptions).

Verified: Connected → (none) → Connected, plus three refusals — no --confirm, an ambiguous name
(named all four candidates and changed nothing), and an unknown name (listed what is configured).

  • An ambiguous match is refused, never resolved to the first hit.
  • connect requires --confirm because it can arm an order-capable surface; disconnect does not,
    because the safe direction should never be the harder one to reach.
  • The verdict comes from polling Status to a settled value, and succeeded is reported
    separately from changed, so "already connected" cannot masquerade as work done.

5. The rest of 1.7.0 — dialog, strategy, playbackctl, log, selfcheck

Answer modal and non-modal dialogs; start/stop chart strategies; settle-polled seek + speed +
range; grep a log NT holds open (filtered in-process, since the payload ceiling is small); and prove a
Python tree against its manifest. Each ships with tests.

A few hard-won notes baked into these:

  • A seek can succeed and land where there is no data. Writing the clock validates nothing, so an
    out-of-range seek now fails closed instead of truthfully reporting succeeded: true, offset 0.
  • A stale object reference watches a corpse — identity is the (type, chart) pair, never the
    pointer; collections are re-queried.
  • MethodInfo.Invoke hides the real error behind the content-free "Exception has been thrown by
    the target of an invocation"
    ; InnerException is unwrapped.
  • GetMethod binds public-only by default and much of the chart API is internal — members are
    enumerated instead.
  • Zero is a value. --speed 0 was rejected as "missing" by a falsy test, which happened to be the
    one value needed to park a running replay.

6. layout — window placement as a hashable file

Capture and re-apply where NT's windows sit, in fractions rather than pixels, keyed by identity
rather than HWND, so it survives a different monitor layout and a restart.


Notes for a headless user

restart correctly refuses without --task/--exe, but there's no default task and, from a
session-0 shell (SSH), a bare --exe cannot produce a usable GUI. The recipe that works:

schtasks /Create /TN NtLaunch /TR "powershell.exe -NoProfile -ExecutionPolicy Bypass -File <launcher>" \
         /SC ONCE /ST 00:00 /SD 01/01/2099 /RU <user> /IT /F
nt8bridge restart --task NtLaunch --wait 200

/IT is the whole trick — it reaches the interactive session. Happy to add this to the README if
useful.

Also: NinjaTrader is multi-UI-threaded. Reading Window.Title off the poller thread throws, and
swallowing that turns a --chart filter into a silent mismatch — so titles are read on the owning
dispatcher throughout, with bounded waits.

🤖 Generated with Claude Code

silentsudo-io and others added 7 commits August 4, 2026 00:35
The AddOn enumerates and moves an HWND it is told to move; every judgement
(identity matching, fractions, monitor mapping) lives in Python where it is
unit-testable without a running NinjaTrader.

Fractions not pixels, so one file describes the same arrangement on a 2560x1440
desktop and a 1920x1080 VM. Identity not HWND, so it survives a restart — which
required stripping WPF's per-launch GUID out of the window class.

Shares the sentinel.berth.layout/1 schema with the desktop tool. 28 new tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ds + layout

Not for upstream. The fleet must run one tree; deploying whichever PR branch was
touched last is how nodes drift. Upstream takes eman007#2 (1.4.0) and eman007#4 (1.5.0) separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
NinjaTrader parents its windows to a hidden owner, so every real NT window
reports owned=True. Skipping owned windows as 'dialogs' turned 27 live windows
into 1 — the whole application — while reporting success. Regression test added.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The AddOn computes 'monitor' from the on-screen rect, which for a minimized
window is the park coordinate (~-32000). Dividing the restored geometry by that
monitor's work area produced fractions like x = -3.1.

Found by cross-checking this module's capture against SentinelBerth's on the same
desktop: two independent producers of one schema agreed on three windows and
disagreed on the two minimized ones. Neither tool alone would have shown it —
the numbers were wrong, not absent. Now 5 agree / 0 differ.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Commits the 1.7.0 mutating half, which until now existed only as
uncommitted files on a single disk (chart, dialog, strategy, playbackctl,
logread, selfcheck + their tests), and adds four verbs proven against a
live NinjaTrader.

chart --api: dump BarsProperties / Bars / BarsPeriod / ChartBars methods /
Connection API. Read-only. It immediately refuted two assumptions:
Bars.IsResetOnNewTradingDay is READ-ONLY, and "Break at EOD" is settable
nowhere at all (absent from BarsProperties, from BarsPeriod unfiltered, and
from every chart template). A setter written against the obvious name would
have resolved, changed nothing, and reported success.

connections --connect/--disconnect: the read side has existed since 1.5.0
but nothing could raise a connection. Ambiguity is refused rather than
resolved to the first match; connect needs --confirm and disconnect does
not; the verdict comes from polling Status to a settled value, and
succeeded is reported separately from changed.

chart --apply-template: hands NT the XML it wrote, via
TemplateLoadIndicators, instead of poking the Indicators collection --
which leaves indicators stuck at Configure and could only advise "re-add it
from the UI". Proven: 0 -> 14 indicators, all 14 at Realtime.
  - The element matters: <Indicators> works, the document root "resolved
    but count stayed 0". Candidates are tried and the winner reported.
  - RefreshAllBars REGRESSES indicators to 0 running (laddered: 2, then 3,
    then 0). Excluded. addIndicator still calls it as a fallback and is
    suspect for the same reason.
  - Activation is ASYNCHRONOUS; counting once read 3 of 14. Now settle-polled.

chart --data-window: RangeType/DaysBack/BarsBack/MonthsBack/From/To, applied
via OnDataSeriesChanged, settled on IsBarsLoading, every field read back.
Only fields the caller named are sent, and only those move. A REJECTED field
now fails the whole call (exit 2) -- the first version returned success while
silently dropping an unparseable date.

Dates parse culture-invariantly: a window meaning one thing on a US box and
another on an EU box is a silently different backtest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ation

# Conflicts:
#	CHANGELOG.md
#	pyproject.toml
…rror

addIndicator counted and read State immediately after RefreshIndicators,
which tears the collection down and rebuilds it. Adding an indicator to a
2-indicator chart therefore reported "2 -> 1", state Configure, verdict
"treat this as NOT applied" -- while the chart a moment later held all
three, every one at Realtime.

A false negative here is worse than a false positive: a caller that
believes "not applied" retries, and the chart ends up with duplicates.

Now settle-polled, the same way applyTemplate and playbackctl --seek are:
poll until the count reaches its target and the new indicator is live, and
give up only once it stops improving. Verified on a live chart -- the same
operation now reports "indicators 2 -> 3 and it is Realtime".

Also: the RefreshIndicators failure note used ex.Message, so it printed the
content-free "Exception has been thrown by the target of an invocation."
It now uses Explain(), which unwraps InnerException.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@silentsudo-io

Copy link
Copy Markdown
Contributor Author

Follow-up: I said in the description that addIndicator's RefreshAllBars fallback looked suspect. I went and measured it rather than leaving you a vague warning, and it turned out to be a false negative, which is the more dangerous direction.

Adding an indicator to a 2-indicator chart reported:

indicatorsBefore 2  ->  indicatorsAfter 1
stateAfter Configure    succeeded false
verdict: THE CALL RESOLVED BUT THE INDICATOR COUNT WENT 2 -> 1 — treat this as NOT applied

…while the chart a moment later held all three, every one at Realtime. RefreshIndicators tears the collection down and rebuilds it, so a count taken during the rebuild sees a transient that was never a real state. A caller that believes "not applied" retries, and you get duplicate indicators on the chart.

Pushed e526db6: it now settle-polls, the same way applyTemplate and playbackctl --seek do. Same operation now reports indicators 2 -> 3 and it is Realtime.

Also in that commit: the RefreshIndicators failure note used ex.Message, so it printed the content-free "Exception has been thrown by the target of an invocation." It now unwraps InnerException.

365 tests green. Still happy to split this PR up if that's easier to review.

🤖 Generated with Claude Code

…as not a sandbox

Both found by driving the tool against a live NT on 2026-08-09, not by
reading it. Both could have quietly corrupted a data corpus.

1. A REPLY IS NOT AN ARTIFACT.
   The AddOn answers `exists`, which is equally true for a file that was
   already on disk and never fetched, so run_histget counted every
   pre-existing date as `downloaded`. Measured: two dates already present
   came back `downloaded / count: 1` with nothing written anywhere.
   Each date is now stat'd (size, mtime_ns) in the REAL write directory
   before and after the call. Only a file that appeared or changed counts.
   A file that is absent afterwards is a FAILURE naming the path. A forced
   re-download that produced byte-identical output is `unchanged`, not
   `downloaded` -- we cannot prove a fetch we cannot see, and under-claiming
   is the safe direction to be wrong in.

2. --replay-dir NEVER CHOSE WHERE DOWNLOADS LAND.
   It only built the path for the skip-existing check; download_one sends
   the AddOn an instrument and a date and nothing more, and NT8 writes into
   its own db\replay. The old help text said "db/replay dir", which reads as
   a destination. Measured decisively: a date the corpus did not hold,
   requested with --replay-dir pointed at an empty scratch directory, landed
   in the LIVE corpus while the scratch stayed empty.
   The honest flag is now --check-dir. --replay-dir is refused (exit 2)
   unless it names the directory writes truly go to, and the refusal states
   that path. Every result now carries write_dir and check_dir so no caller
   has to guess.

Also: .nrd mtimes are SOURCE-derived, not download time -- a file fetched
minutes ago carried a two-day-old stamp. Do not date an acquisition from an
.nrd mtime; the reliable discriminator is a date the corpus does not hold.

TESTS -- 10 in test_histget.py, and the new ones are proven able to fail:
reverted to the old "trust the reply" classification, the two load-bearing
controls FAIL and the rest still pass.
  * bridge says ok + nothing written  -> FAILURE naming the path
  * unchanged file                    -> `unchanged`, never `downloaded`
  * changed file                      -> IS a download (the guard must not
                                         turn real fetches into non-events)
  * CLI refuses a foreign --replay-dir (exit 2, message names --check-dir)
  * CLI still ACCEPTS the real one -- a refusal that blocks the legitimate
    call is worse than the bug it replaces
  * result states write_dir / check_dir

AND THE EXISTING TEST WAS VACUOUS. test_run_histget_skips_saturdays_and_existing
asserted `count == len(downloaded)`, which holds at zero; its fake download
wrote no file, so under real verification all five dates fail and the test
still passed. It now writes, and asserts something was downloaded and
nothing failed.

Full suite: 372 passed, 6 skipped.
@silentsudo-io

Copy link
Copy Markdown
Contributor Author

Pushed one more commit to this branch: 98a80c0 — two histget defects found by driving it against a live NT today. Both could quietly damage a replay corpus, so I'd rather they were on the record here than sitting in a fork.

1. It reported downloads it never made. The AddOn answers exists, which is equally true for a file that was already on disk and never fetched, so every pre-existing date came back as downloaded. Two dates already present returned downloaded / count: 1 with nothing written anywhere. Each date is now stat'd (size, mtime_ns) in the real write directory before and after the call; only a file that appeared or changed counts, an absent file is a failure naming the path, and a re-download that produced identical bytes reports unchanged rather than claiming a fetch we can't see.

2. --replay-dir never chose where downloads land. It only built the path for the skip-existing check — download_one sends an instrument and a date and nothing more, and NT8 writes into its own db\replay. The old help text ("db/replay dir") reads as a destination. Requesting a date the corpus didn't hold, with --replay-dir pointed at an empty scratch directory, put the file in the live corpus while the scratch stayed empty. The honest flag is now --check-dir; --replay-dir is refused (exit 2) unless it names the real directory, and the refusal prints that path. Results carry write_dir/check_dir.

Also worth knowing if you touch this area: .nrd mtimes are source-derived, not download time — a file fetched minutes ago carried a two-day-old stamp, which is how I initially mis-concluded I hadn't written anything.

Tests: 10 in test_histget.py, and the new ones are proven able to fail (reverting to the old "trust the reply" classification makes the two load-bearing controls fail). One existing test turned out vacuous — it asserted count == len(downloaded), which holds at zero, while its fake wrote no file; it now writes and asserts a real outcome. Full suite 372 passed, 6 skipped. Verified live: forced re-download of a held date now reports unchanged/count 0, and the foreign --replay-dir exits 2.

Happy to split this into its own PR if you'd prefer it separate from the chart-cell work.

silentsudo-io and others added 2 commits August 9, 2026 22:50
…only, refusal-first

The highest-risk verb family here: it puts real orders on real accounts from a
headless shell. Built refusal-first and driven end to end on Sim101 before commit.

Discovered from the running platform via `order --api`, not from memory:
  Account.Provider -> Provider.Simulator marks a sim account (note: [RW])
  CreateOrder(Instrument, OrderAction, OrderType, TimeInForce, int, double, double,
              string oco, string name, CustomOrder) -> Order
  Submit/Change/Cancel(IEnumerable<Order>), CancelAllOrders(Instrument)
  16 OrderStates; terminal = Filled | Cancelled | Rejected

Gates, in order, before anything is constructed: confirm -> account named -> account
found -> account is SIMULATED -> instrument named -> instrument resolves -> side ->
type -> quantity -> a type's required price present. Nothing is inferred at any step.

The sim gate is an ALLOWLIST (== Provider.Simulator), not a denylist. Proven against
two real broker accounts, which report Provider31 -- an unnamed enum value. A
"!= Live" check would have passed them.

Settle-poll, never judge by the call. This codebase has burned four bugs on reading a
state mid-transition. A still-moving order returns settled:false with its state, NOT
an error -- a false negative makes a caller retry, and a retried placement is a
duplicate order.

Driven on Sim101: place -> Working; change 1000 -> 1200 with quantity preserved;
status; cancel -> Cancelled/terminal. Two orders placed, both cancelled, zero fills,
nothing left behind (the NQ short on that account is pre-existing -- different
instrument, no executions under the order name).

Bug found by driving, not reading: `change --limit-price` alone came back BADQTY
because the CLI defaulted --quantity to 0, so "absent" arrived as an explicit zero.
change treats absent as leave-alone; for a price the same bug would have repriced a
resting order to 0. Defaults are None now, with a regression test.

Deliberately NOT included: any escalation flag. Reaching a live account requires
editing and recompiling the AddOn.

34 new tests, every one asserting a refusal. Suite: 406 passed, 6 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
They used to be nested under `if bars_type:`, so a call that named no type sent no
values -- and there was therefore no way to set Value/Value2 on a bars type that was
already selected.

That matters because switching to a CUSTOM bars type makes NT re-apply that type's OWN
defaults and discard the values on the incoming BarsPeriod: asking for SentinelTBars
6/24 produced 212201_0_2_... twice in a row. Stock types (Renko 11) are unaffected,
which is why it stayed hidden.

The fix opens the two-pass path: switch the type, then call again WITHOUT --bars-type
to stamp the values on. It does not defeat NT's override on its own -- that is still
open -- but the client is no longer the thing standing in the way.

Written during the replay-cell work on 2026-08-08/09 and left uncommitted on one disk
until now. The same tree already recorded that failure once, when the whole 1.7.0
mutating half turned out to exist in exactly one place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@silentsudo-io

Copy link
Copy Markdown
Contributor Author

Two commits added since the initial review request — flagging them rather than letting them appear silently.

da68705 — a new order verb family (place / cancel / change / status / list / api).

This is the highest-risk thing in the AddOn, so it is built refusal-first:

  • Simulation accounts only. The gate is an allowlistAccount.Provider == Provider.Simulator — not a denylist. That turned out to matter: real broker accounts on my box report Provider31, an unnamed enum value, which a != Live check would have passed straight through. Proven against two funded accounts.
  • No escalation flag exists. Reaching a live account requires editing and recompiling the AddOn. That is deliberate, not an oversight.
  • confirm=true is mandatory for every mutating action, and account / instrument / side / type are never inferred.
  • Outcomes are settle-polled, never judged by the call — consistent with the apply-template and addIndicator fixes elsewhere in this PR. A still-moving order returns settled:false with its state rather than an error, because a false negative makes a caller retry, and a retried placement is a duplicate order.
  • order --api dumps the live Account/Order surface (unfiltered) so the contract is read off the platform rather than from docs.

Driven end to end on Sim101: place → Working, change → reprice with quantity preserved, status, cancel → Cancelled. 34 new tests, every one asserting a refusal.

3e14423chartseries sends Value/Value2 independently of --bars-type.

They were nested under if bars_type:, so a call naming no type sent no values, and there was no way to set values on an already-selected bars type. This opens the two-pass path (switch type, then stamp values). It does not by itself defeat NT re-applying a custom bars type's own defaults — that remains open — but the client is no longer the blocker.

Suite: 406 passed, 6 skipped. Happy to split either commit into its own PR if that is easier to review.

silentsudo-io and others added 9 commits August 11, 2026 15:19
Staging one bake across three sentries cost most of an afternoon, and every
minute of it was orchestration rather than capability. applyTemplate already
existed in the AddOn and worked; what did not exist was everything around a
verb -- fanning it across the fleet, noticing a replay had stalled, counting
what a bake produced, and knowing which box ran stale code.

All four are Python-only on purpose. No AddOn change means no compile, no
reload, no restart, and nothing to break on a box that is mid-bake.

  fleet     run one verb on every sentry and tabulate; UNREACHABLE is its own
            outcome, never folded into "fine"
  corpus    how much corpus each box holds AND how fresh -- a large count
            with a stale timestamp is a bake that died, which looks identical
            to a healthy one if you only count files
  versions  bridge drift, deployed vs source. Born from an hour lost today
            debugging --apply-template as if it were missing. It was not
            missing, it was undeployed, and nothing would say so. First run
            found 5 of 6 sentries stale, 4 with no applyTemplate at all, and
            sentry-4 on a third intermediate build.
  runrange  drive a replay across a range unattended. A replay halts at the
            end of each day's .nrd and simply sits there; nothing errors. A
            7-day bake was 7 silent stops each needing a human. A stall is
            treated as a GAP, not an error: step the clock and resume, and
            report stalls/steps rather than hiding them.

Two bugs found and fixed while testing, both from this repo's own written
lessons: PowerShell 5.1 has no `if` as an expression (first cut returned
"The term 'if' is not recognized" from every host), and the corpus probe now
uses forward slashes to kill an entire class of backslash escaping between
Python, the shell and ssh.
The dispatch was an if/elif chain, so `playbackctl --seek X --speed 5` ran the
seek, printed a successful seek verdict, and never set the speed. `playback`
then read speed 0 and nothing replayed. A call that looks like it worked is
worse than one that errors; this one cost a session of "why is nothing moving".

Both actions now run, and the ORDER was decided by driving it, not by argument.
Speed-first was tried first: the seek's settle poll then spent its whole 60s
timeout chasing a clock the speed write had already set walking - "still 420s
from target, the clock was still moving when we stopped watching". A seek can
only be judged against a parked transport, so it is seek-then-speed. A seek that
does not land aborts before the speed write: starting a replay from a position
we never confirmed is the same silent half-success.

Proven on sentry-2, readback included:
  completed: ['seek','speed']  allSucceeded: True
  landed within a minute of target after 250ms - speed is now 7
  READBACK -> clock 2025-12-30T13:00:14  speed 7  moving True

Also: --range-type accepts custom/CustomRange/"custom range" and sends NT the
member it actually parses. NT's enum is CustomRange; this tool's own --help said
"custom", so the operator was told to write a value the tool then reported as
rejected. Help now also states that From/To are ignored unless RangeType is
CustomRange, which is why a correct-looking window silently did nothing.
…ched

Measured three times on three occasions (sentry-1 twice, sentry-2 once): NT
restores its workspace after a restart or reboot and the chart comes back with
SentinelExcursionRecorder at Realtime and Playback CONNECTED - one --speed away
from writing REPLAYED rows into the Council corpus, where nothing downstream can
separate them from live ones. A fleet sweep tonight found four more armed on
sentry-3 and sentry-6. It was caught by eye every time, and being written down
did not prevent the third occurrence, so it is a refusal now.

The recorder set is part of the bake spec, not of the machine's leftover state:
declare it with --expect-recorder (repeatable), and anything else attached stops
the run, names it, and prints the removal command. A refused run exits 2 - a
wrapper that only reads the exit code must not read "I did nothing" as "it ran".
Every run reports `recorders` either way, so "nothing was recording" is a
recorded measurement instead of an assumption.

Proven able to fire, and able to pass:
  - detector: finds ExcursionRecorder, ignores BarDump
  - a version bump (_v2_0_0 -> _v9_9_9) does NOT un-guard it (substring match)
  - control: a chart with no recorder returns none -> the bake proceeds
  - declared vs not, same live state -> RUN vs REFUSE
  - driven against sentry-3's real armed recorders: refused, exit 2, both named,
    transport untouched
MEASURED on sentry-2: `playbackctl --seek 2025-12-30T12:00 --speed 5` reported
"landed within a minute of target", `playback` read the clock walking at 12:07,
and the bar transcript wrote ZERO new bars - still 946, newest still
2025-12-31T21:59Z. Earlier the same session the clock read 12-30T23:08 while
every bar produced was stamped 12-31T21:15Z. NowEst is a DISPLAYED clock, not
the data cursor: the settle poll confirms the value it just wrote to a property
nothing reads back.

That is not cosmetic. runrange steps over gaps BY SEEKING, so it counted steps
and reported success while the tape never moved, and no replay-built corpus row
can assert its window from the seek that preceded it.

seekwitness.py judges a seek by the FIRST live bar written after it:
  REPOSITIONED        (exit 0) bars arrive near the target
  DID NOT REPOSITION  (exit 2) bars arrive somewhere else, and it says where
  UNVERIFIED          (exit 3) no witness - parked tape, data gap, or nothing
                      recording. Three outcomes, three exit codes, because
                      "unverified is not passed" only means something if a
                      script can act on it.

Two bugs its own controls caught, both worth keeping:
  - it first took max(bar time), which is blind to a BACKWARD seek: earlier
    stamps never move the maximum, so a real reposition read as no-witness. The
    baseline is a byte offset now - what we want is what was APPENDED, which is
    a position in the file, not a value.
  - reading from that offset then skipped a line to "handle a partial line",
    eating the first new bar - the exact row it exists to return.

Also: a seek combined with a speed now PARKS first. Driven against a tape left
at 5x, the settle poll spent its full 60s reporting "still 296s from target, the
clock was still moving when we stopped watching" - a verdict about the previous
command's speed, not this seek. Parking makes the sequence deterministic
whatever state the box was left in, which is what an unattended bake needs.
Measured on sentry-2 across three consecutive drives:
  1. --seek moves a DISPLAYED clock and never the tape. Seek to 12-30T23:00 and
     the bars that arrived were stamped 12-30T04:59:50Z - the start of the
     loaded data, 23 hours away. Repeated after rewriting the range: identical
     bar, identical verdict.
  2. --set-start/--set-end no-op while Playback is CONNECTED (that is the
     2099-12-01 read-back) and stick while it is disconnected.
  3. Even a correct range did not bound the feed: with two day files staged the
     tape fed from the earlier one regardless. Removing 20251230.nrd collapsed
     coverage to 12-30T23:00 -> 12-31T16:00 and the run finally began where it
     was asked to.

So the day file is the control surface, the range is a filter on top of it, and
the seek is decoration. runrange was written to step over gaps BY SEEKING, which
means every multi-day bake this fleet has run was positioned by a mechanism that
does nothing. This makes the real mechanism one call with before/after coverage
printed as evidence.

Reversible by construction: parked days go to db/_replay_parked/<inst>/, never a
delete, OUTSIDE the replay tree because a parking spot NT can still scan is not
a parking spot. --list is read-only. Staging nothing is REFUSED rather than
silently emptying the folder, and a day that exists on neither side is REFUSED
rather than producing an empty run that reads as a short one.

Driven end to end on sentry-5 (23 day files, no active bake):
  - nonexistent day            -> REFUSED, named
  - stage one day              -> 23 staged, 22 parked, stagedAfter == request
  - restore                    -> 22 restored, 23 staged, nothing parked
  - independent check ON THE BOX: 23 files present, parked dir empty
versions watches the bridge AddOn. Nothing watched the suite itself, and it
mattered: sentry-2 was writing bar transcripts stamped dumpVer 1.0.0 / bars.1 /
resetOnNewTradingDay: null while this tree has been v1.1.0 / bars.2 since 08-09.
bars.2 exists so "Break at EOD" is a COMPARED field instead of an assumed
precondition, so a box on bars.1 cannot serve the bar-type parity gate - and its
rows do not say so, they just omit the field.

TWO WRONG VERSIONS OF THIS CHECK BEFORE THE RIGHT ONE, both caught by driving it:
  1. Hashing NinjaTrader.Custom.dll reported all six sentries DRIFTED with six
     DIFFERENT hashes - which is what it would print if they were perfectly in
     sync. Every box compiles its own DLL and a .NET build is not reproducible.
     A check that can only ever say DRIFTED is not a check.
  2. The source probe's Where-Object regex was mangled by escaping across
     python/bash/ssh/cmd/PowerShell, so it filtered EVERYTHING, hashed an empty
     set, and reported the sha256 of the empty string as a confident verdict on
     all six boxes. The tell was `files 0`, which is why the file count is
     reported next to the digest and not hidden behind it.

It now digests the .cs SOURCE (name+size, sorted) and judges the fleet AGAINST
ITSELF: sentries run a 398-file deployed subset while this tree has 988, so
comparing to main would print DRIFTED forever. Main's digest is reported as
context, never as the verdict. Outlier = exit 2, unreachable = exit 3.

What it found, and it inverts the naive reading: 4 boxes agreed and sentry-4
looked like the odd one - but sentry-4 was the ONLY box carrying the current
bars.2 BarDump (26,600); the four-box majority all ran the old bars.1 (20,959).
Consensus finds the odd box, not the right one. sentry-4 and sentry-6 were also
running a SentinelCandidateRecorder six KB older than the fleet's - the tool
that writes the corpus.

Main's canonical BarDump and CandidateRecorder are now deployed to all six and
verified byte-for-byte (26600 / 50753 on every box). sentry-4 remains a reported
outlier on files main does not carry (ChartDataSeriesSwitcher, a v0_2_0 copier).

STILL OWED: source deployed is not source COMPILED. Each box needs a recompile;
sentry-2 must wait until its Test C bake finishes.
…ells

A typed MCP seam over nt8bridge - not a reimplementation. The bridge already has 50 verbs, a
validated in-process AddOn and refusals earned by measurement; this exposes the useful subset as
schema-checked tools.

WHY. One session on 2026-08-14 lost real time to four quoting failures, none of them about
NinjaTrader: a db path containing "NinjaTrader 8", a dialog title "Auto Rollover Notification", and
two chained commands - each dying somewhere in bash -> ssh -> cmd.exe -> PowerShell. The CLI was
never wrong. subprocess.run([...], shell=False) has no shell in it, so the whole class of bug goes
away, and test_server.py asserts that with the exact path that broke it.

THREE THINGS MEASURED WHILE BUILDING IT, each of which would otherwise have shipped as a silent bug:

1. --host is NOT global. Exactly five verbs accept it (stage, fleet, corpus, versions, builds). The
   first cut passed it to everything and nt_status host=sentry-1 returned an argparse usage dump.
   Other verbs now run ON the box over ssh with each argument quoted for cmd.exe - double quotes,
   because a single quote is an ordinary character there, which is precisely what split that
   "NinjaTrader 8" path.
2. A nonzero exit code does not mean failure. ntstatus returns 2 while emitting good JSON; selfcheck
   returns 0. Exit codes are verb-specific, so the PAYLOAD decides. Same lesson as a background job
   whose "exit 0" was the shell's and not the program's.
3. A typo'd argument key is refused rather than ignored - a dropped "acount=" is how a caller
   believes it passed a limit price it never passed.

SAFETY IS IN CODE, NOT IN GOOD INTENTIONS. Mutating tools require confirm=true, and any named account
is checked against an allowlist that DEFAULTS TO REFUSE - the operating contract's first ALWAYS-STOP
is live/funded accounts. nt_strategy_disable is deliberately NOT gated: the safe direction must never
be the harder one to reach.

test_server.py drives real stdio and provokes every refusal rather than assuming it - mutation without
confirm, a live-sounding account even WITH confirm, a typo'd key, a missing required argument - plus a
positive control so the harness cannot pass by testing nothing. 19 checks, offline.

Proven end to end: nt_status local (pid 46408) and on sentry-1 (pid 6244) over ssh, and
nt_strategies host=sentry-1 reading back "ScalpProbe on Chart - NQ 06-26 = Realtime".
MEASURED 2026-08-17: a Sentinel risk container auto-flattened SimBURN-1 at its
daily loss stop and locked the account out. A market order placed through THIS
verb then FILLED on it -- 1 lot, status ok, no refusal, no error.

The container is a PARTICIPANT, not a PERIMETER. SentinelCore's own header says
"that acts must consult it before acting"; there is no interception layer. So
every order path that does not ASK walks past a halted account -- this verb, the
DOM, a chart trade button, a hand-placed ticket.

OrderGate now consults, via REFLECTION so this AddOn still compiles standalone
where no Sentinel suite exists:
  1. SentinelCore.DrawdownAllowsEntry(acct, out reason)  -- the trailing floor
  2. SentinelCore.GetGovernorState(name).Status          -- the daily-loss stop
Both, because they are INDEPENDENT states on different clocks: v1 wired only the
governor and the re-probe still filled, since the governor had rolled with the
trading day while the DD floor was still breaching.

⛔ TWO SILENT FAILURES I WROTE INTO THE FIX FOR A SILENT FAILURE:
  * the type lookup used ...AddOns.SentinelCore; the real namespace is
    ...AddOns.Sentinel.SentinelCore, so it returned null and BOTH consults were
    skipped while the method reported "permitted" with no trace. Three probes
    filled and each looked like a policy decision.
  * nothing logged what was read. A guard that cannot say "I did not run" is
    indistinguishable from a guard that ran and permitted.
⇒ Correct name (several candidates tried), absence logged once and loudly, and
  the consult now logs the values it read on EVERY call, not only on refusal.

Proven to RUN, from bridge.log:
  risk consult SimBURN-1: DrawdownAllowsEntry=True reason=-
  risk consult SimBURN-1: governorStatus=Trading
⚠ NOT yet proven to REFUSE -- every probe was on an account Sentinel considers
permitted. The refusal path needs an account genuinely in Breach/DayHalted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MEASURED 2026-08-17, and it is the root cause of a four-hour chase:

  01:25:45  RISK writes:  GOV TRACE SimBURN-1 ... -> DayHalted
  01:27:07  BRIDGE reads: governorStatus=Trading
  (GOV TRACE logs on CHANGE; no change back was ever logged)

A value cannot be both. The writer and the reader were looking at DIFFERENT
static stores. The new counter says how many:

  RISK CONSULT AMBIGUOUS: 14 loaded assemblies expose SentinelCore.

Every `reload` leaves the previous NinjaTrader.Custom resident -- assemblies
cannot be unloaded -- so six reloads left FOURTEEN generations in the AppDomain,
each with its own static _gov dictionary. The risk service writes to its
generation's store; this consult resolved whichever assembly GetAssemblies()
returned first and read an orphan.

⇒ There was never a status flip, never a forgiven baseline, never a lying seam.
There were fourteen stores. Three separate "root causes" I published tonight
were all artefacts of reading the wrong one.

⇒ The implication is far wider than this verb: EVERY cross-component seam read
is generation-sensitive after a reload -- the Council reading sensors, the
Copier reading GetGovernorState, the Cockpit. It is the same phenomenon behind
this project's "F5 decouples bar-type seams" rule and SentinelCore v1.40.0's
generation beacon, now with a count attached.

This commit does not try to pick the right generation -- that is guesswork
dressed as a fix. It makes the ambiguity LOUD, and says plainly that a
"permitted" answer under multiple generations is not authoritative.

⚠ The consult therefore remains NOT PROVEN in its refusal direction. A fair
test needs ONE generation, i.e. a freshly restarted NT.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@eman007

eman007 commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Heads-up that main moved under this branch: aeca908, released as 1.5.1. It fixes #6 (reported
by @Quantrosoft) — configure resolved its write targets once before the key loop, so after
Strategy swapped the StrategyTemplate every later key wrote to a detached object, reported
"status": "set", echoed its value back, and left the tab on its old data series. Same shape as
several things in this PR: the call resolved, and that was mistaken for the work having happened.

I tested the merge locally so you are not guessing at it:

addon/NT8BridgeServer.cs   Auto-merging ✓  — no conflict
README.md                  Auto-merging ✓  — no conflict
CHANGELOG.md               CONFLICT
pyproject.toml             CONFLICT

The C# merges clean, +4,387 lines and all — you did not touch RunConfigure, so the two diffs
do not overlap. Both conflicts are bookkeeping:

  • pyproject.toml — keep your 1.7.0, drop the 1.5.1.
  • CHANGELOG.md — the new ## [1.5.1] section slots in between your [1.6.0] and [1.5.0].

Two notes in case they save you a step. configure now applies Strategy first regardless of
map order, so anything in this branch that configures a tab no longer depends on key ordering. And
each set carries a new nowReads field, read off a freshly resolved chain rather than off the
object just written to — a read-back from the object you wrote to passes cleanly in exactly that
bug. It is a value and never a verdict, since setters legitimately transform (BarsPeriod "77077:120:1" reads back as "Wave 120").

No need to rebase on my account unless you would rather keep the branch current — the review is not
blocked on it.

@eman007 eman007 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the whole diff — AddOn C#, every new Python module, and the MCP server. The AddOn compiles
clean offline ([SUCCESS], only the expected CS0436 duplicate-type warnings against the deployed
Custom.dll), so nothing below is about build health.

The evidence discipline in here is the best thing about the PR, and it is not a small thing: the
RefreshAllBars regression found by laddering each step, the addIndicator false negative caught
because a count taken mid-rebuild is not a state, the Provider31 discovery that turned the order
gate from a denylist into an allowlist. Several of the findings below are cases where that same
standard was not applied to the new code itself.

I am requesting changes on three of them. The rest are notes.


Blocking

1. The risk gate blocks cancel and change — and its own text says it doesn't

NT8BridgeServer.cs:5537

SentinelGovernorRefusal lives in OrderGate, and OrderGate is called by all three mutating
verbs: place (5712), cancel (5778), change (5825).

The refusal it emits says:

Flatten and exit paths are unaffected; this refusal applies to NEW positions only.

That is false for two of the three call sites. Concretely: an account trips its daily-loss halt
while a stop-limit or an unfilled entry is still working. order --action cancel --confirm now
returns RISKLOCKED and the order stays live — at exactly the moment you most want it gone.
change (pulling a stop in) is blocked the same way.

The reasoning in the comment block above it is right — the container is a participant, not a
perimeter — but the conclusion only applies to taking on new exposure. Cancelling a resting order
is risk-reducing. The consult belongs in RunOrderPlace, not in the shared gate.

2. run_range's corpus-recorder refusal fails OPEN

fleet.py:388

for chart in (call("chart") or {}).get("charts", []) or []:

call() discards the return code and parses stdout. On timeout the CLI prints
{"command":"chart","status":"timeout","ok":false} — a truthy dict with no charts key — so
live == [], undeclared == [], and the bake starts. The unparseable-output fallback
({"_raw", "_rc"}) does the same. Every failure mode of the probe reads as "nothing attached."

So: NT restores a workspace with a corpus recorder at Realtime, the probe happens to time out, and
the replay writes replayed rows into the live corpus — the precise outcome this refusal exists to
prevent. The result payload then reports "recorders": [] as though it were measured.

fleet.py's own header states the rule this breaks:

a box that could not be reached is UNKNOWN, never "fine"

A probe that did not answer needs to be its own refusal, distinct from a probe that answered "none".

3. --verify-utc-offset defaults to 0.0, so the witness fails every correct seek

cli.py:1401

seekwitness.verify computes abs((got - timedelta(hours=utc_offset_h)) - tgt), where got is a
UTC bar stamp and tgt is the transport's Est target. With the default offset of 0 the
drift is a constant ~300 min against tolerance_min=90.

The witness is on by default (--no-verify-bars is the opt-out, cli.py:847/887), so a plain
playbackctl --seek '2025-12-30T12:00' --speed 5 that landed perfectly reports
verdict: DID NOT REPOSITION, prints "⛔ THE CLOCK MOVED AND THE TAPE DID NOT", and exits 2.

That is the same class of misverdict seekwitness was written to eliminate, arriving by the
opposite sign. The default should be the box's own Est→UTC offset, not 0.


Should fix

  • sentinel_mcp/tools.py:169/181/241 — three MCP tools send arguments the CLI does not accept.
    Verified by running the parser: flatten --confirmunrecognized arguments: --confirm (exit 2);
    chart --indicatorsunrecognized arguments: --indicators; fleet with no --verbthe following arguments are required: --verb, and verb is not in nt_fleet's schema, so a caller
    cannot supply it. nt_flatten and nt_fleet can never succeed. Given the README's argument that
    shell=False removes a class of quoting bug, these are worth a test that asserts each tool's argv
    against the real parser — the bug class moved rather than disappeared.

  • NT8BridgeServer.cs:2610dataWindow reports succeeded: true while its own verdict says
    the values did not stick.
    bool wOk = wAfter != null && !wRejected; ignores wChanged, so a
    write NT accepted by reflection but silently discarded returns succeeded: true next to
    "verdict": "…the values did not stick", and the CLI exits 0. A human reading summary sees
    failure, a script gating on the exit code sees success. Same principle as the section-3 rule in
    your description ("a rejected field fails the whole call") — this is the unchanged case rather
    than the rejected one.

  • cli.py:835 — the park-first fix only applies when --speed is also given.
    actions.append("park") is nested inside if args.speed is not None:, so a bare --seek against
    a transport left at 5x skips the park and hits the documented failure: the settle poll burns the
    full timeout and reports the clock was still moving. The docstring's justification — deterministic
    regardless of the state the box was left in — argues for making it unconditional for seek.

  • cli.py:1605runrange/fleet/corpus/versions exit 0 unless explicitly refused.
    A runrange that logged "no clock reading; abandoning" after zero progress exits 0; so does a
    fleet where every host was unreachable and a versions where every box read STALE. builds
    (~1576) already does this correctly with 2/3.

  • staging.py:52NT8 is hardcoded to C:/Users/Administrator/…, including on the local
    path.
    On a box where the user is not Administrator — the normal case for this repo — survey
    returns empty and stage refuses with "requested day(s) exist on neither side" while the file is
    sitting in the real db\replay. The rest of the package goes through ntio.nt8_root(), which is
    NT8_DIR-overridable. The fleet.py constants have the same issue but are remote-only.

Smaller

  • NT8BridgeServer.cs:5914order --action status passes settleSeconds = 0, so the settle loop
    never executes and settled is always false, including on a filled order. That's the field a
    caller polls instead of re-submitting.
  • fleet.py:60retired=false in fleet.conf excludes the host: values are raw strings and
    callers test truthiness, so false/no/0 all read as retired. The omission is invisible,
    because the report only lists hosts it saw.
  • fleet.py:335 — the refusal branch returns from/to and the success branch returns
    start/end, so a wrapper reading out["start"] raises KeyError on exactly the refusal it
    handles. Also, the opening call("playbackctl --seek …") is unchecked — a bake whose first seek
    was refused still runs to completion and reports refused: false, started: true.
  • fleet.py:225builds --reference documents a DLL path, but local_digest returns None
    unless it is a directory, so the documented value makes referenceDigest silently null.
  • s3.json at repo root looks like a stray — a captured layout output from your box.

On scope

Taking you up on the offer to split, which you have made three times.

The chart work — --api, --apply-template, --data-window — is the strongest part of this and is
close to mergeable on its own. connections --connect/--disconnect and the
dialog/strategy/playbackctl/log/layout set are a reasonable second piece. order wants
finding 1 fixed first, and I'd like it to land by itself so it gets reviewed on its own terms.

fleet/corpus/versions/runrange/stage/seekwitness and sentinel_mcp/ are a different
question, and it is not a code-quality one. They are orchestration for a specific multi-box Sentinel
deployment, and sentinel_mcp/ is an MCP surface for a suite that is not part of this project.
I'd rather not take those into this repo — not because they are bad (the seekwitness finding that
NowEst is a displayed clock rather than a data cursor is genuinely valuable and I want that
written down somewhere), but because this repo's scope is "drive one NinjaTrader from a CLI," and a
fleet orchestrator with a hardcoded Administrator path and an MCP server are both a step outside it.
They would fit better as a separate repo that depends on nt8bridge. Happy to link it from the
README.

On that note: the AddOn's file header still says

⚠ NO SENTINEL REFERENCES IN THIS FILE, BY DESIGN. It ships in an open-source repo.

while the file now names Sentinel 35 times. The reflection-based consult itself is done carefully —
no hard reference, compiles and no-ops where the suite is absent, logs RISK CONSULT INERT once —
so the mechanism is fine. The standing rule the header states is what needs updating, or restoring.

Also worth knowing: main has moved to 1.5.1 (see my earlier note) — NT8BridgeServer.cs still
auto-merges, only CHANGELOG.md and pyproject.toml conflict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants