Skip to content

feat: propagate MCP request trace context - #869

Merged
inanna-apollo merged 5 commits into
mainfrom
mcp-request-trace-context
Oct 2, 2026
Merged

inanna-apollo merged 5 commits into
mainfrom
mcp-request-trace-context

Conversation

@inanna-apollo

@inanna-apollo inanna-apollo commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

MCP callers can supply W3C trace context in request _meta, but handler spans previously only inherited HTTP context. This connects every implemented request handler to valid _meta.traceparent context over stdio and Streamable HTTP and propagates the active GraphQL client span downstream.

Valid metadata trace context directly parents the handler span; absent or invalid context falls back to the HTTP span or a new stdio trace root. Metadata baggage replaces HTTP baggage when string-valued, with existing HTTP-injection sanitization. Handler spans retain HTTP tracing ancestry for formatted logs while the selected OTel context is set before entry, when the OTel layer starts spans.

The selected context is attached during each future poll, so propagation and Rhai trace-ID correlation also work when logging filters disable handler spans. If the HTTP span is filtered too, the extracted HTTP-header context supplies the fallback. Context attachment is restored on suspension and cancellation, keeping concurrent requests isolated.

Explicit header configuration keeps its existing behavior: automatic injection regenerates traceparent/tracestate, while explicitly supplied baggage remains when the active context has no baggage. Includes documentation and a patch changeset. rmcp 3.5.0 already supplies the required metadata accessors.

Supports both protocol versions 2025-11-25 and 2026-07-28, retaining main's initialize/discovery lifecycle.

Validation

  • cargo test --offline --workspace — passed (1,332 tests).
  • cargo test -p apollo-mcp-server mcp_trace_context --offline — all 50 regression tests passed across both protocol versions.
  • cargo clippy --offline --all-targets -- --deny warnings — passed.
  • cargo fmt --all -- --check and git diff --check — passed.
  • Real rmcp HTTP/stdio lifecycles with the OTel in-memory exporter verify exact remote parent IDs, downstream client-span headers, carrier precedence, invalid metadata, safe baggage, filtered-span fallback, Rhai trace IDs, and concurrent/sequential isolation. A focused poll/drop test verifies context restoration on suspension and cancellation.
  • Fresh-context independent and informed adversarial reviews found no actionable issues. Transport coverage includes every implemented request handler; subscriptions exercise discovery, acknowledgement, readiness, and cancellation. Formatted logs preserve HTTP path scope and accepted-session attribution while metadata still selects the OTel parent.

@inanna-apollo
inanna-apollo requested review from a team as code owners October 1, 2026 21:27
@apollo-librarian

apollo-librarian Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Docs preview ready

The preview is ready to be viewed. View the preview

File Changes

0 new, 2 changed, 0 removed
* (developer-tools)/apollo-mcp-server/(latest)/config-file.mdx
* (developer-tools)/apollo-mcp-server/(latest)/telemetry.mdx

Build ID: 263199513faf1a3bc48ac991
Build Logs: View logs

URL: https://www.apollographql.com/docs/deploy-preview/263199513faf1a3bc48ac991


⚠️ AI Style Review — 7 Issues Found

Summary

The documentation was updated to align with the style guide across several key sections. In text-formatting, code font is now applied to symbols like trace_id and stdio. Significant updates to word-and-symbol-usage include adopting dictionary-valid contractions such as "doesn't," replacing device-specific terms like "selected" with "chosen," avoiding the use of "while" as a synonym for "although," and replacing semicolons with periods. Framing and voice improvements include using imperative verbs for direct instructions, adopting an authoritative tone to prescribe best paths, and simplifying language by avoiding "parents" as a verb. Additionally, list items now use sentence case for labels, and content was updated to use present tense for consistent verb-tense-and-voice.

Duration: 3360ms
Review Log: View detailed log

This review is AI-generated. Please use common sense when accepting these suggestions, as they may not always be accurate or appropriate for your specific context.

@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: fb9eaf7a-2f40-4230-9377-de8e54a063fa

📥 Commits

Reviewing files that changed from the base of the PR and between 8f176dc and be2a989.

📒 Files selected for processing (7)
  • .changeset/mcp_request_trace_context.md
  • crates/apollo-mcp-server/src/server/states/mcp_trace_context_tests.rs
  • crates/apollo-mcp-server/src/server/states/running.rs
  • crates/apollo-mcp-server/src/server/states/running/test_support.rs
  • crates/apollo-mcp-server/src/server/states/server_span_tests.rs
  • crates/apollo-mcp-server/src/server/states/telemetry.rs
  • docs/source/telemetry.mdx

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.


📝 Summary

Summary by CodeRabbit

  • Features
    • MCP request trace context and baggage now propagate to downstream GraphQL requests. Valid trace metadata is honored; malformed metadata does not fail requests.
    • HTTP trace context is used when metadata does not provide a valid trace parent, and stdio requests can start their own trace.
    • Trace context remains available when request spans are filtered out.
  • Documentation
    • Updated the telemetry guide with trace-parent selection, baggage handling, and propagation details.

Walkthrough

MCP request metadata now sets trace context for handler spans and downstream GraphQL requests. The changes add context propagation across handlers, tests for HTTP and stdio behavior, and telemetry documentation.

Changes

MCP trace-context propagation

Layer / File(s) Summary
Build request trace context
crates/apollo-mcp-server/src/server/states/telemetry.rs, crates/apollo-mcp-server/src/server/states/mcp_trace_context_tests.rs
The context builder validates trace metadata, selects a parent context, and applies baggage rules. Tests cover metadata precedence, invalid values, filtered spans, and context restoration across future polls.
Apply context to MCP handlers
crates/apollo-mcp-server/src/server/states/running.rs, crates/apollo-mcp-server/src/server/states/running/test_support.rs, crates/apollo-mcp-server/src/server/states/server_span_tests.rs, crates/apollo-mcp-server/src/server/states/mcp_trace_context_tests.rs, docs/source/telemetry.mdx, .changeset/mcp_request_trace_context.md
Request and subscription handlers instrument their work with request context. Tests cover propagation through HTTP and stdio handlers, subscriptions, and logs. The telemetry guide and changeset describe the behavior.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant MCPClient
  participant MCPHandler
  participant with_request_context
  participant GraphQLClient
  MCPClient->>MCPHandler: Send request with _meta
  MCPHandler->>with_request_context: Build handler span and request context
  with_request_context-->>MCPHandler: Return span and context
  MCPHandler->>GraphQLClient: Send request with active trace context
Loading

Suggested reviewers: daleseo

Merge Risk: ⚪ Minimal · up to be2a9

No actionable issue remains identified for this trace-context change; it is mergeable after normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to be2a9

The reviewed path carries caller-provided diagnostic context, not authentication identity. Metadata validation, header sanitization and request-scoped attachment limit the apparent risk. No new access-control bypass or cross-request disclosure was substantiated, but downstream policies and failure-path coverage remain incomplete.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — A caller able to submit an MCP request can influence its handler trace ancestry, Rhai trace-ID correlation and propagated tracing headers on resulting GraphQL requests. The documented scope includes endpoints selected by execution hooks. Effective exposure beyond those endpoints depends on downstream propagation and consumer policies that were not established.

Security Findings and Attack Paths

  • inferred — The reviewed metadata-to-GraphQL path did not substantiate an authorization bypass, arbitrary non-tracing header injection or cross-request context disclosure. Metadata selection is separate from authorization construction, and concurrent-request assertions preserve distinct trace and baggage associations. This conclusion is bounded to the inspected path, not a complete security assessment of downstream consumers.

Trust Boundaries and Controls

  • observed — Caller metadata crosses into diagnostic context through the W3C propagator. Tracestate must be HTTP-header text, and baggage members with metadata that cannot form an HTTP header are removed before propagation. These are syntax controls, not a confidentiality allowlist or authorization policy; the documentation explicitly treats baggage as caller-controlled context.

Resilience and Maintainability Implications

  • inferred — The common future wrapper and request-owned contexts provide strong isolation evidence for normal concurrency, suspension, repetition and drop cancellation. Error-returning handlers use the same wrapper, but restoration after a production-handler error or panic unwind remains less directly evidenced than suspension and drop restoration.

Hardening Proposals

  • proposed — Where GraphQL destinations cross a trust boundary, apply a deployment-specific baggage allowlist and ensure downstream services and execution hooks never use trace IDs or baggage as authorization identity. This is hardening guidance, not an observed vulnerability.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.46% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 41 functions across 5 files. (2 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: propagation of MCP request trace context.
Description check ✅ Passed The description directly explains MCP trace-context propagation, fallback behavior, baggage handling, downstream GraphQL propagation, testing, and supported protocol versions.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 41.46% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 41 functions across 5 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

✅ Changeset file added - thank you!

Comment thread crates/apollo-mcp-server/src/server/states/running.rs Outdated
/// Prepare the handler span and retain its parent context for poll-scoped
/// attachment even when the span is filtered. Set the parent before entry:
/// the OTel layer starts spans on entry, after which parentage cannot change.
pub(super) fn with_request_context(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Consider] Every handler now repeats the same three steps: call with_request_context(...), wrap the body in async, then .instrument(span).with_context(parent_context).await. That is nine copies, and it forced a full re-indent of listen and call_tool. If a future handler forgets either .instrument or .with_context, it silently loses the parent or the filtered-span fallback.

A helper that takes the future would keep the ordering invariant (context outside, span inside) in one place and make the call sites flat:

pub(super) fn in_request_context<F: Future>(
    span: tracing::Span,
    context: &RequestContext<RoleServer>,
    fut: F,
) -> impl Future<Output = F::Output> {
    let parent = /* current body of with_request_context */;
    fut.instrument(span).with_context(parent)
}

Single-expression handlers such as list_tools, list_prompts and get_prompt could then drop their async wrapper blocks. (Ch. 1.6: prefer structure over repeated patterns.)

@claude

claude Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Review Summary

Overall: This PR has every MCP request handler read W3C trace context from _meta, over both stdio and Streamable HTTP. It parents the handler span before entry and attaches the selected context on each poll, so downstream propagation and Rhai trace_id still work when spans are filtered. The fallback order is clear: _meta first, then the HTTP span, then the extracted HTTP headers. Baggage is sanitized.

Update (be2a989): This commit fixes the tracestate panic that @gocamille found. TraceState only rejects ,, = and over-long values, and the propagator writes the value back unchanged. That meant a control character or non-ASCII byte in _meta.tracestate reached reqwest-tracing, whose HeaderValue::from_str(...).expect(...) panicked. is_http_header_text now drops any tracestate that HeaderValue rejects or that is not visible ASCII, which is the same set HeaderExtractor accepts over HTTP. The valid traceparent is still used. The new control_char_state and non_ascii_state cases cover both inputs. The fix is minimal and correct, and it introduces no new issues.

Earlier findings:

  • ✅ Handler spans keep the http_request tracing parent, so fmt log scope is preserved (ee91177).
  • ✅ Every handler has _meta parentage coverage (ee91177).
  • [Consider, still open, non-blocking] The with_request_context → .instrument(span).with_context(ctx) sequence is still repeated in each handler. A helper would keep the ordering in one place. The per-handler tests now catch a handler that drops the wiring.

Test coverage: Strong. Tests cover precedence, invalid, non-string and non-header-safe metadata, baggage clearing and sanitization, filtered handler and HTTP spans, and concurrent stdio isolation. Every handler is covered across both protocol versions.

Recommendation: Approve.


Reviewed by Claude Code Opus 5.5

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it reworks trace-context propagation across all MCP handler methods and parses external W3C trace-context metadata from request _meta, a human look would still be worthwhile.

What was reviewed: the with_request_context/request_parent_context extraction and fallback logic in telemetry.rs (invalid traceparent never grafts tracestate from a different trace; baggage precedence between metadata and HTTP), the mechanical running.rs refactor from #[tracing::instrument(parent = ...)] to explicit info_span + .instrument().with_context() across all nine ServerHandler methods, and the poll/cancellation context-restoration test. The flagged baggage-size concern was ruled out as pre-existing (same normalization path already used for HTTP headers).

Extended reasoning...

The change replaces instrument-macro parenting with manual OTel context propagation across every ServerHandler method (initialize, listen, call_tool, list_tools, list_resources, read_resource, list_prompts, get_prompt, set_level) in running.rs, and adds metadata-based trace-context extraction with fallback-to-HTTP logic plus baggage merging/sanitization in telemetry.rs. Security-sensitive surface: it parses externally supplied W3C trace-context fields (traceparent/tracestate/baggage) from request _meta, with explicit care taken not to graft tracestate across trace boundaries on invalid traceparent. No CODEOWNERS objections or outstanding review threads were found in the timeline, and tests are extensive (482 new lines of dedicated trace-context tests plus poll/cancellation coverage), but the subtlety of future-poll-scoped OTel context attachment and its interaction with span filtering is enough complexity that independent human review is still worthwhile rather than auto-approving.

let baggage = meta.get_baggage().map(normalize_baggage_list);
let carrier: HashMap<String, String> = [
("traceparent", meta.get_traceparent()),
("tracestate", meta.get_tracestate()),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I traced _meta.tracestate from request_parent_context through the W3C propagator and into the downstream GraphQL request, and I noticed traceparent gets reformatted from the parsed hex IDs, and baggage gets percent-encoded (and its metadata is checked by sanitize_baggage_for_http_injection). But TraceState::valid_value only rejects ,, = and long values, and trace_state().header() writes the value back as is. So a valid traceparent with a tracestate like vendor=k\u0001 reaches reqwest-tracing's HeaderValue::from_str(...).expect(...) and panics, and the client hangs until it times out. I wonder if we should drop tracestate here when HeaderValue::from_str rejects it, the same way we do for baggage metadata?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks! I reproduced it (panic in reqwest-tracing, then the client hangs until timeout) and fixed it in be2a989. I went slightly stricter than HeaderValue::from_str: _meta.tracestate now has to be text an inbound header could carry (visible ASCII or tab, the same set HeaderExtractor reads), because from_str still lets non-ASCII through to the downstream request. When it fails, the whole tracestate is dropped and the traceparent kept. control_char_state and non_ascii_state cover both cases.

@gocamille gocamille left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just have one minor comment! Thank you @inanna-apollo !

@inanna-apollo
inanna-apollo merged commit eaf0d51 into main Oct 2, 2026
19 checks passed
@inanna-apollo
inanna-apollo deleted the mcp-request-trace-context branch October 2, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants