You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0262822
Browse filesBrowse the repository at this point in the historyBrowse files
fix(mcp): shorten instead of replacing when trimming tool responses, ordered reductions, trim notes (#3554)
Refs #3542
Follows the proposal in the maintainer comment on #3542 (the three points, in order). Kept to `trimToolResponse`; the table tool's `having` / `limit` will go to a separate issue and PR as asked.
## What
1. **Shorten instead of replace** (`packages/api/src/utils/trimToolResponse.ts`). A string over its budget is cut and gets a `…(+N chars)` suffix (e.g. `…(+6,900 chars)`), instead of becoming `{"__hdx_trimmed":true,"originalSize":N}`. This applies to every tool through the fallback. In arrays, items within an equal share stay whole and the large items split what's left, so shortened items add up to the array's budget (with replacement markers that didn't matter; with shortened strings it does).
2. **Trim notes.** `TrimResult` gains `notes: string[]`: the size before/after and limit, each reduction that changed something (e.g. `patterns[].samples: kept the first 1 (60 elements removed from 30 items)`), arrays cut by path (`patterns: cut from 30 to 10 items`), and how many values were shortened. Every caller this PR changes that already adds a `note` when trimmed now also returns them as `trimNotes`; `clickstack_describe_table` is not changed here.
3. **Ordered reductions.** `trimToolResponse(data, maxSize, { reductions })` with `{ path, keep }`, `{ path, maxChars }` and `{ path, drop: true }`, using the path syntax from the proposal (`patterns[].samples`, `patterns[].samples[]`). It works as described:
- a response that fits is returned unchanged (same object);
- otherwise the reductions run in order on a copy, each on every match, and the whole response is measured after each step; it stops as soon as it fits;
- only then the existing per-key split and array cuts run.
`clickstack_event_patterns` uses the list from the proposal (samples keep 3, keep 1, samples[] maxChars 2000, pattern maxChars 2000, trend drop), plus a last step: `patterns[].whereSnippet` is dropped, never shortened, because a cut drill-down predicate is invalid SQL. Since it's dropped before the fallback runs, the fallback (the only part that shortens strings) never sees one; the `note` says when snippets were removed. `clickstack_emerging_signals` shortens `emerging[].pattern` / `disappeared[].pattern` to 2,000 chars first.
4. **Smaller fixes from review.** `shortenString` never makes a value longer and doesn't split surrogate pairs; array item budgets count the separators; array cuts are noted once per path and size markers are noted; note paths match the response (`result.…`); `search` with `denoise` keeps the first trim's notes; the `clickstack_sql` batch appends its row cut to the per-query notes.
## Effect
An `event_patterns`-shaped result with 30 patterns, the top 3 being ~4.5 KB stack traces with 5 samples each (101,450 chars, limit 50,000):
| | Patterns returned | Top pattern | Size |
|---|---|---|---|
| `main` | 10 | `{"__hdx_trimmed":true,"originalSize":4736}` | 3,848 |
| Fallback only (this PR, no reductions) | 10 | shortened, with `…(+N chars)` | 8,282 |
| With the event_patterns reductions | **30** | complete | 41,082 (stopped after "samples keep 1") |
## Tests
`packages/api/src/utils/__tests__/trimToolResponse.test.ts` (12 new):
- suffix format; shortened (not replaced) strings in the fallback within the budget; notes for size, shortened values and array cuts; no notes when nothing is trimmed;
- each reduction kind on every match, with its note; stop-early (later steps don't run once it fits); unchanged when it fits; input not mutated; fallback after reductions stays within the budget;
- the 30-pattern case above: all 30 kept, no `__hdx_trimmed`.
- JSON-escape expansion (quotes, control chars, non-ASCII, `…`) stays within the budget; separators-only arrays; no-growth and surrogate cases; grouped notes; `drop` on array elements.
`packages/api/src/mcp/__tests__/queryTool.int.test.ts`: the real `event_patterns` tool with 30 large patterns keeps all 30, drops (never cuts) `whereSnippet`, and returns `trimNotes`. `sqlToolBatch.test.ts`: batch cuts replace `trimNotes`.
`npx jest src/mcp src/utils --testPathIgnorePatterns '\.int\.test\.ts$'`: 932 passed; `queryTool` + `emergingSignalsTool` int tests: 117 passed. tsc, eslint (no new warnings) and prettier clean on the changed files.
## Not included
- `having` / `limit` / default order for `clickstack_table`: separate issue and PR (#3555, #3558), as asked.
fix: MCP tool responses that are over the size budget now shorten long strings with a `…(+N chars)` suffix instead of replacing them, say what was trimmed in `trimNotes`, and `clickstack_event_patterns` / `clickstack_emerging_signals` drop samples and shorten long patterns before cutting whole patterns, so the requested `topN` survives when it can
note: 'Result was trimmed for context size. Narrow the time range, add filters, or reduce topN to reduce data.',
584
+
note:
585
+
'Result was trimmed for context size. Narrow the time range, add filters, or reduce topN to reduce data.'+
586
+
(snippetsDropped
587
+
? ' whereSnippet was removed from the patterns to fit, so the usage steps for it do not apply; narrow the time range or filters, or lower topN, to get it back.'
0 commit comments