To recognize abuse, start by observing legitimate tool use.
MCP Detect recorded an AI agent choosing and calling MCP tools, froze that
behavior as a labeled benign corpus, and tested every detection against what
the agent actually did when nothing was wrong.
Reproduce · Method · Architecture · Coverage · Evidence · Limitations · Contributing
Caution
MCP Detect is research software, not a production monitor or security guarantee. Its included traffic is synthetic and self-authored. A clean run means only that these bounded checks found no matching indicators.
Python 3.11 or newer. No credentials, Docker, Wazuh installation, or private files required.
git clone https://github.com/RasheedFarhat/mcp-detect.git
cd mcp-detect
make measureThe command checks the committed sample corpus against frozen Wazuh verdicts:
# Offline reproduction -- tier: sample
Aggregate benign FP: 0/4727
...
PASSED -- reproduces the committed sample-level numbers
(data/attack_corpus_sample_v1.summary.md) from public files alone,
no Docker/Ollama/Wazuh needed.
Run the complete published corpus with:
make measure-fullThe reproduction note defines exactly what this proves and what still requires a live engine.
An MCP tool call compresses several trust decisions into one ordinary-looking JSON-RPC exchange: which identity initiated it, which arguments crossed the boundary, which handler accepted them, and which downstream resource was reached.
MCP Detect makes the observable part of that path inspectable. A detection is published with its threat model, signal, implementation, synthetic fixtures, expected result, known misses, and reproduction path. The limitations ship with the rule.
To catch an agent crossing a line, first learn how it uses the same tools when the task is legitimate.
MCP Detect gave a local qwen3:1.7b model benign tasks across filesystem,
Git, memory, time, and fetch servers. The model—not a scripted client—decided
which tools to call, which arguments to send, and when it had enough
information to stop. A transparent proxy recorded every JSON-RPC exchange and
labeled it with its session, task, and scenario.
benign task ──▶ local AI agent ──▶ autonomous MCP tool calls ──▶ result
│
▼
transparent proxy
│
▼
frozen, labeled benign corpus
│
detections must stay quiet
The frozen v2 corpus contains 541 sessions, 1,011 model-chosen tool calls, 4,727 total records, and 20 distinct tools across six server configurations. It deliberately preserves the model's wrong paths, malformed calls, retries, tool substitutions, and improvised workarounds. Those rough edges are useful: they make the false-positive test resemble agent behavior rather than a tidy set of invented happy paths.
The tasks and environment are synthetic; the tool choices and mistakes were produced by the model. The published capture is immutable and checksummed because a fresh LLM run can be behaviorally similar without being byte-for-byte identical.
Read the corpus summary, inspect the agent loop, or review the task set.
authenticated identity
│
▼
MCP client ──JSON-RPC──▶ transparent proxy ──▶ MCP handler ──▶ resource
│ │
│ └─ authorization
│ must happen here
▼
telemetry.jsonl
╱ │ ╲
Wazuh rules stateful evidence +
drift coverage tools
Who can call it? What can it reach?
The proxy forwards protocol bytes unchanged and records one JSONL event per JSON-RPC message. That telemetry can show what identity and arguments reached a handler. It cannot prove the handler enforced tenant ownership, object-level authorization, or downstream access policy. That distinction is a boundary of the architecture, not a rule still waiting to be written.
These are the results reproduced by make measure-full against the complete
committed corpus.
| Detection | Engine | Full-corpus result | Deliberate boundary |
|---|---|---|---|
| Tool poisoning | Wazuh, structural | 12/12 | Keyword-avoiding, non-comment, homoglyph, and padded instructions can evade |
| Sensitive absolute-path read | Wazuh, structural | 6/8 direct-rule matches | Renamed tools, unlisted paths, encoding, and symlink indirection remain outside the signal |
| Path traversal | Wazuh, structural | 6/8 direct-rule matches | The other two cases intentionally resolve to the credential-read rule; encoded and tool-name-spoofed traversal can evade |
| Rug-pull drift | Stateful baseline → Wazuh | 3/3 drift shapes | Behavior-only changes are invisible; legitimate updates also require review |
| Credential exfiltration | Wazuh, two-hop | 11/11 read + 11/11 exfil | Unknown argument keys, unlisted secret shapes, encoding, and renamed paths can evade individual hops |
The same replay reports 0 alerts across 4,727 project-authored benign records. These are regression results over synthetic traffic—not an independent false-positive rate, field benchmark, or validated accuracy claim.
The detailed ledger, including untested and structurally invisible cases, lives in State of the Project.
MCP Detect does not ask you to trust a screenshot or a summary table.
checksummed JSONL ──▶ frozen Wazuh verdicts ──▶ golden rule matches
│ │ │
exact input pinned rule hash expected metrics
└─────────────────────┴────────────────────────┘
offline replay
- Corpus checksums identify the exact records being measured.
- Golden-match files preserve the final Wazuh rule ID captured for each line.
- Every golden file pins the SHA-256 of
local_rules.xml. - The replay refuses stale evidence when the current rules no longer match that hash.
- The optional live path recaptures verdicts from Wazuh and checks semantic parity with the frozen evidence.
Start with Reproduction Verification, then inspect the full corpus or the measurement implementation.
Note
Synthetic reference—not customer work. All tenants, invoices, code, and evidence in this example are fabricated for the repository.
The cross-tenant authorization reference
shows a valid-looking MCP invoice request that crosses a tenant boundary because
the handler trusts a caller-controlled tenant_id. Normal protocol telemetry
records a legitimate tool call; only the source and trust path reveal that the
authenticated tenant was ignored.
The reference includes the complete chain:
- Intentionally vulnerable handler
- Exact source-level fix
- Denied cross-tenant retest
- Checksummed evidence manifest
Run it independently:
python3 -m unittest discover \
-s examples/reference-mcp-review/tests -p 'test_*.py' -v| Path | Command | What it exercises |
|---|---|---|
| Fast offline replay | make measure |
Public sample, frozen verdicts, published sample numbers |
| Full offline replay | make measure-full |
Complete synthetic corpus and published full measurements |
| Offline release gate | make verify |
Both corpus tiers, framework regressions, proxy, baseline, redaction, manifests, and authorization retest |
| Live engine parity | make lab-up, then follow docs/REPRO.md |
Local Docker/Wazuh pipeline and recaptured engine verdicts |
The Docker stack uses development credentials and loopback assumptions. Do not expose it to an untrusted network or treat it as a production deployment.
Repository map
| Path | Purpose |
|---|---|
lab/ |
Agent, proxy, schema, corpus generation, attack harnesses, baseline, analysis, and redaction |
examples/ |
Synthetic NorthwindPay environment and cross-tenant authorization reference |
detections/ · wazuh/ |
Detection definitions and Wazuh rules |
framework/ |
Compiler, registry, coverage, inventory, and verification tools |
data/ |
Sample, benign, evasion, and complete attack corpora |
docs/ |
Design decisions, engine findings, reports, and research history |
- Every bundled malicious and benign record is synthetic and self-authored.
- Structural rules match observable shapes, not semantic intent.
- Encoded payloads, alternate field names, homoglyphs, renamed tools, and behavior-only compromise can cross the boundary without matching a rule.
- Trust-on-first-use drift detection needs analyst context to separate an approved change from a rug pull.
- Offline replay verifies committed calculations from frozen engine verdicts; it does not substitute for current live-engine parity.
- Handler authorization and downstream resource policy require source, configuration, identity, and denied-path testing beyond telemetry.
Contributions should make the evidence stronger, not merely increase a coverage count. Read CONTRIBUTING.md before proposing a rule, fixture, or framework change.
Report vulnerabilities privately through GitHub's
security advisory flow,
following SECURITY.md. For research use, citation metadata is in
CITATION.cff. Release history is available in
CHANGELOG.md and on the
releases page.
MCP Detect was created and is maintained by Rasheed Farhat.
MIT © 2026 Rasheed Farhat. The license covers the original code, schemas, rules, documentation, fixtures, and complete synthetic corpora in this repository.