Charts in the data store
Problem
A chart stores history rather than a value: many messages build the series, bounded by a time window (default one hour) or a point cap (default none).
Once the window is full, each arriving point ages one out, so trimming runs on every message from then on. A chart is free for its first hour and pays on every message after, which means it looks fine in any test shorter than the window.
Trimming the legacy datastore is cheap because it only reshuffles pointers:
data[node.id] = currentData.filter(filterFn)
The surviving messages are the same objects. Nothing is copied.
The store cannot do that. Its contract is that stored values are insulated from the flows that wrote them, so writing a series has to project the points, deep-clone them, and wrap each one in a Proxy — the same logical operation at 50x the cost.
| per message, 3,600-point series |
|
append (push) |
0.001ms, 1 event |
| trim — legacy datastore |
0.034ms |
| trim — into the store |
1.73ms |
3,600 points is the default config at 1 msg/sec.
Mutating in place rather than rebuilding is not a way out: shift() reindexes a proxied array, so every element hits the set trap — 1.66ms and 1,001 events.
Options
Either trim less often and let the series briefly exceed its window, or give the store explicit operations for series.
1. Batch the trim. Rebuild every N messages rather than every message. At 1-in-60 that is 0.03ms/msg, level with the legacy path, and needs no new API. Each trim still sends the full series over the socket.
2. Explicit series ops. append / trim / replace on the store, each emitting one structured event ("k removed from the front"). Fixes the socket payload as well as the CPU. Adds internal store API.
Neither is a breaking change. datastore.append / save / filter keep their signatures and only their internals change, so chart nodes, third-party widgets and user flows are untouched.
Charts in the data store
Problem
A chart stores history rather than a value: many messages build the series, bounded by a time window (default one hour) or a point cap (default none).
Once the window is full, each arriving point ages one out, so trimming runs on every message from then on. A chart is free for its first hour and pays on every message after, which means it looks fine in any test shorter than the window.
Trimming the legacy datastore is cheap because it only reshuffles pointers:
The surviving messages are the same objects. Nothing is copied.
The store cannot do that. Its contract is that stored values are insulated from the flows that wrote them, so writing a series has to project the points, deep-clone them, and wrap each one in a Proxy — the same logical operation at 50x the cost.
push)3,600 points is the default config at 1 msg/sec.
Mutating in place rather than rebuilding is not a way out:
shift()reindexes a proxied array, so every element hits the set trap — 1.66ms and 1,001 events.Options
Either trim less often and let the series briefly exceed its window, or give the store explicit operations for series.
1. Batch the trim. Rebuild every N messages rather than every message. At 1-in-60 that is 0.03ms/msg, level with the legacy path, and needs no new API. Each trim still sends the full series over the socket.
2. Explicit series ops.
append/trim/replaceon the store, each emitting one structured event ("k removed from the front"). Fixes the socket payload as well as the CPU. Adds internal store API.Neither is a breaking change.
datastore.append/save/filterkeep their signatures and only their internals change, so chart nodes, third-party widgets and user flows are untouched.