Skip to content

feat(ag-ui): ask the human a question mid-run, and be answered - #3248

Open
vvlrff wants to merge 12 commits into
mainfrom
feat/ag-ui-human-input
Open

vvlrff wants to merge 12 commits into
mainfrom
feat/ag-ui-human-input

Conversation

@vvlrff

@vvlrff vvlrff commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Why are these changes needed?

A tool inside a served agent can call context.input(), but over AG-UI the human it wants is behind the client, and nothing carried the question there. With no hitl_hook configured, default_hitl_hook raised HumanInputNotProvidedError and the turn died pointing you at Agent(..., hitl_hook=...) — so the obvious setup was the broken one, and the only working one answered in the server process, where the human is not.

The protocol already describes the round trip: a run may finish with an interrupt outcome instead of a success, and a later run carries the answer back in resume. Nothing in ag2 produced or consumed it.

The question leaves as the outcome of the run that raised it. The exchange ends normally — not with an error — and the turn stays behind:

data: {"type":"RUN_FINISHED","threadId":"thread-1","runId":"run-1",
       "outcome":{"type":"interrupt","interrupts":[
         {"id":"e2c1...","reason":"human_input","message":"Book LH441? (yes/no)",
          "responseSchema":{"type":"string","title":"Answer"},
          "expiresAt":"2026-01-01T12:15:00+00:00",
          "metadata":{"ag2":{"proof":"g7Qk..."}}}]}}

The client answers with a new run id on the same thread — the thread is the unit of continuity, which is what the protocol's own examples and its TypeScript client do:

{"threadId": "thread-1", "runId": "run-2", "messages": [],
 "resume": [{"interruptId": "e2c1...", "status": "resolved", "payload": "yes",
             "metadata": {"ag2": {"proof": "g7Qk..."}}}]}

Continuing means holding the turn, not replaying it. The call is suspended inside the Python functioncontext.input stopped mid-way through the tool, with its locals, its loop position and whatever it had already done still live. A message history cannot rebuild that: replaying would re-enter the tool from the top and redo its side effects. So the turn is held in the serving process between the two exchanges, suspended exactly where it stopped.

That has consequences an operator needs up front, and they are documented rather than hidden: sticky routing is required (route by threadId; behind a round-robin balancer a resume lands on a process that knows nothing), a restart drops every held turn, and memory is proportional to retention — a held turn is a suspended coroutine carrying its whole conversation history, not a record.

The structural risk is that an interrupt is delivered as the terminating event of its own exchange, so the response closes by design while the agent is still suspended. Turn lifetime therefore had to move out of the request's task group, which is where it used to be cancelled.

This changes a default

A run with no human-input hook now pauses and asks the client, where it used to fail. A caller-supplied hook wins and behaves exactly as today: answered in process, no interrupt emitted, run finished in one exchange. The interrupt path is what an unconfigured run does instead of dying. Worth a line in the release notes; it also diverges from the ACP integration, which still refuses — justified, because ACP has nobody to ask.

A resume is authenticated

An interrupt id alone is a bare string in a request body, and an answer attributed to a human is the input an agent trusts most. Each interrupt carries a proof in the protocol's metadata envelope, which clients are required to echo back, so a compliant one already does this.

It is a capability, not a signature: the question it proves is suspended in this process, so the value to compare against is already in memory beside it — no key, nothing to rotate, secrets.compare_digest against what was issued, per interrupt. Giving up needs it too, since ending someone else's turn is not a lesser act than answering it. This is why the version floor moves: the resume side of that envelope exists only from ag-ui-protocol 0.1.21.

Every refusal is a RUN_ERROR with a machine-readable code, never a stream that just stops:

code What happened
INTERRUPT_NOT_HELD Nothing held for this thread: unknown id, already answered, or expired
INTERRUPT_NOT_OUTSTANDING Something is held, but not waiting on the interrupt this run names
INTERRUPT_NOT_PROVEN The proof is missing, malformed, or not the one issued
INTERRUPT_PAYLOAD_REFUSED The payload is not what the interrupt asked for

A refusal that is not the turn's own fault leaves the held turn resumable by a legitimate answer without restamping its clock, so a stream of bad resumes cannot keep a turn alive past the deadline its client was shown.

Both AG-UI transports

ag2/ag_ui/stream.py and ag2/a2ui/transports/ag_ui.py each emit their own run lifecycle and a client cannot tell them apart, so both are in scope. They share one serve_exchange in ag2/ag_ui/interrupts.py, which is the exchange — a transport supplies only how it launches a turn. If the two ever look different to a client, it will be because something was added outside that function.

Public API

A pausing agent needs no server-side setup. The tool just asks:

from ag2 import Agent, Context
from ag2.ag_ui import AGUIStream

@agent.tool
async def book_flight(context: Context, flight: str) -> str:
    """Book a flight, once the traveller confirms it."""
    answer = await context.input(f"Book {flight}? (yes/no)")
    return "Booked." if answer.strip().lower() == "yes" else "Cancelled."

stream = AGUIStream(agent)

Retention is where the operator tunes it, on either transport:

from ag2.ag_ui import AGUIStream, Retention
from ag2.a2ui import A2UIServer
from ag2.a2ui.transports import AgUiTransport

retention = Retention(ttl=900.0, max_held=128)  # the defaults

stream = AGUIStream(agent, retention=retention)
server = A2UIServer(agent, transport=AgUiTransport(retention=retention))

ttl is not an internal detail: it is what a client is shown as expiresAt. The advertised deadline is the earlier of ttl and the deadline implied by any timeout= passed to context.input — the protocol treats an absent deadline as a promise that the interrupt never expires, and clients reject late answers locally on the strength of it, so the figure on the wire is always the one that will in fact apply. Reaching that number needed HumanInputRequest.timeout; asyncio.wait_for exposes no deadline.

Tool-call approval rides the same path. ApprovalRequired now asks through Context.ask(request) — the seam context.input() delegates to, for callers needing a richer request than a bare question string — raising a ToolApprovalRequest. The interrupt carries reason="tool_approval" and the toolCallId, and a bare true/false payload is accepted for an approval and only for an approval: the client drew two buttons, the middleware reads words like "always", and translating between them beats teaching either side the other's vocabulary.

Shutdown. A2UIServer cancels held turns for you on ASGI shutdown, and waits for them to unwind rather than only scheduling the cancellation. build_asgi returns an HTTPEndpoint class rather than an app, so nothing can hang a hook on AGUIStream — call await stream.aclose() from your own, or use it as an async context manager.

This adds a required member to A2UITransport. A transport holding state past a single request has to be closed, and duck-checking for aclose with getattr hides that from the type checker, so aclose is declared on the protocol itself. RestTransport implements it as a no-op. The protocol shipped in 1.0.0, so a third-party transport without aclose stops satisfying it — at type-check time only: nothing isinstance-checks A2UITransport at runtime, so existing deployments keep working.

The four refusal codes below are re-exported from ag2.ag_ui alongside Retention, so nothing has to reach into ag2.ag_ui.interrupts for them.

A GET on the AG-UI route returns the capabilities document; the protocol defines the document but no event and no transport for it. A runnable server-plus-client script is in examples/ag_ui/human_input.py.

Related issue number

Closes #3202

Also in the diff, unrelated to the feature: @vvlrff joins the area:mcp CODEOWNERS entries, on the strength of the MCP work already landed.

Known limitations

  • One question is outstanding per turn. Concurrent asking — parallel subtasks each asking a human — is refused with HumanInputError rather than silently orphaning the first question. Serving several at once needs one outcome carrying them all and a resume routed per interrupt; that is follow-up work, not covered here.
  • Held turns cannot move to a shared backend. A suspended coroutine is not serialisable; surviving a restart is a different feature, not a setting.
  • "Already answered" is not distinguishable from "unknown" once a turn has finished and left the registry. Both are INTERRUPT_NOT_HELD, whose message names all three possibilities; a distinct code would need per-thread memory of spent ids.
  • examples/ag_ui/human_input.py has not itself been run against a live model, though every path it exercises now has: a scratch probe drove the round trip, tool approval and refusal, abandonment, all four refusal codes, two sequential questions, shutdown and expiry against gpt-5.6-luna on the Responses API, 46 assertions, all passing.

Checks

70 new tests across turn lifetime, the round trip, retention and deadlines, refusals, resume proof, tool approval, and end-to-end coverage of the second transport. Full suite locally: 5131 passed, 360 skipped, 2 xfailed; ruff check and ruff format clean, mypy gains no new kind of error. Tests drive in-process HTTP against the built ASGI application rather than the registry — the feature is two exchanges sharing state, and both transports look identical from that height. Deadlines run off an injectable clock instead of sleeping.

AI assistance

  • I understand the changes in this PR and can explain them in my own words.
  • I have verified that the PR description accurately reflects the actual diff.
  • If AI assistance was used, I reviewed, tested, and validated the generated code/text before submitting.

🤖 Generated with Claude Code

An AG-UI agent can now stop mid-run and ask. A tool calling
`context.input(...)` leaves the question as the outcome of the run that raised
it; the turn is held in the server, suspended exactly where it stopped, until a
later run on the same thread answers it through `resume`. The call resumes from
the line it stopped on, with its locals intact — nothing is replayed, because
the frame never died.

Tool-call approval rides the same path: an interrupt with
`reason="tool_approval"` carrying the `toolCallId`, answerable with a bare bool.

Both AG-UI transports serve this through one `serve_exchange`, so a client
cannot tell them apart, and a GET on the AG-UI route returns the capabilities
document.

Changes a default: a run with no human-input hook used to fail with "nobody
could be asked" and now pauses and asks the client. A supplied hook behaves
exactly as before. Held turns are bounded by `Retention(ttl=900, max_held=128)`,
the advertised `expiresAt` is the earlier of that bound and any
`context.input(timeout=)`, a resume is refused unless it echoes the proof issued
with its interrupt, and every refusal is a `RUN_ERROR` with a machine-readable
code rather than a stream that simply stops.

Requires `ag-ui-protocol>=0.1.21`, the first release whose resume entries carry
the metadata envelope the proof travels in.

One question is outstanding per turn: concurrent asking — parallel subtasks that
each ask a human — is refused with `HumanInputError` rather than silently
orphaning the first question. Serving several at once is follow-up work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added area:ag-ui AG-UI integration (ag2/ag_ui) area:docs Documentation (website/) area:a2ui A2UI support (ag2/a2ui) area:deps Python dependency updates (pyproject.toml, uv.lock) area:core Agent runtime core: ag2/*.py, ag2/events, ag2/response, ag2/streams area:middleware Middleware system and builtins (ag2/middleware) labels Sep 12, 2026
vvlrff and others added 4 commits September 13, 2026 00:05
Refactor the human-in-the-loop test suite to improve maintainability,
type coverage, and reliability.

- Flatten nested test classes in `test_interrupts.py` for better readability.
- Add explicit type annotations to test helpers in `test_refusals.py` and
  `test_resume_proof.py` to improve static analysis.
- Replace `asyncio.timeout` blocks with `asyncio.wait_for` in
  `test_turn_lifetime.py` to ensure consistent behavior during long-running
  test waits.
- Update `CODEOWNERS` to include `@vvlrff` for MCP-related files.
A proof arriving from a client went straight into `secrets.compare_digest`,
which raises `TypeError` rather than returning `False` when handed a non-ASCII
`str`. `serve_exchange` catches only `ResumeRefusedError`, so the generator died
after `RUN_STARTED` and left the client with no terminating event — and since
the turn had already been taken from the registry, it was then unreachable by
resume, sweep and eviction alike, held until the process went down. Proofs are
now compared as bytes, a turn taken for a resume is put back on any failure and
not only on a refusal, and a run already started on the wire is always
terminated on the wire.

Two smaller ones alongside it. `ServedTurns.release_all` was `async` but awaited
nothing, so shutdown scheduled its cancellations and returned before any held
turn had unwound; it now gathers and waits. And the guard against two questions
on one turn raised into a turn whose own exchange had already ended, so the
refusal reached no client and no log either; it now reports at `error` before
raising.

Also in this pass, from reviewing the two commits before it:

- Revert the `CODEOWNERS` change made in a9f66a1: it is unrelated to #3202.
- Drop `ToolApprovalRequest.tool_name`, which nothing read, and the `""` default
  on `tool_call_id`, which made an approval naming no call constructible.
- Declare `aclose` on the `A2UITransport` protocol instead of duck-checking it
  with `getattr`; `RestTransport` implements it as a no-op.
- Replace the per-turn `_discarder` closure with a class, per AGENTS.md.
- Cut `interrupts.__all__` from 29 names to 19 and re-export the four refusal
  codes from `ag2.ag_ui`, so nothing reaches into a private module for them.
- Rewrite `ag2/ag_ui/` docstrings against the openai SDK's conventions: no
  `Raises:`/`Returns:`/`Attributes:` sections, fields documented beneath
  themselves, backticks rather than Sphinx roles, and rationale moved into
  comments beside the code. 419 lines of docstring become 218.
Add a warning to the user guide explaining that paused tool calls
span multiple runs. Clarify that events are split across different
`runId`s and that clients must track open tool calls by `threadId` to
correctly associate `TOOL_CALL_RESULT` events with their corresponding
starts.
@vvlrff
vvlrff marked this pull request as ready for review September 16, 2026 17:53
@vvlrff
vvlrff requested a review from Lancetnik as a code owner September 16, 2026 17:53
vvlrff and others added 7 commits September 17, 2026 19:20
Refactor the AG-UI interrupt system to use more descriptive reason
codes and improve the robustness of the served agent lifecycle. This
includes a major overhaul of the testing infrastructure to support
end-to-end testing of served agents via real HTTP/ASGI transports.

- Rename `HUMAN_INPUT_REASON` to `INPUT_REQUIRED_REASON` and
  `TOOL_APPROVAL_REASON` to `TOOL_CALL_REASON` for better clarity.
- Add `HumanInputTimeoutError` to the exception hierarchy.
- Implement validation for `Retention` parameters (`ttl` and `max_held`).
- Introduce a new testing harness (`test/ag_ui/harness.py`) and
  serving utilities (`test/ag_ui/serving.py`) to drive agents over
  in-process HTTP.
- Restructure AG-UI tests into a `served` sub-package to distinguish
  between generator-based and transport-based testing.
- Add support for reasoning message mapping and empty chunk handling
  in the AG-UI protocol.
- Update documentation to reflect new interrupt reason codes and
  retention behavior.
Update the AG-UI documentation to reflect the new overview structure,
including a comprehensive guide on the AG-UI protocol, supported
capabilities, and basic server implementation.

- Rename `docs/user-guide/ag-ui/index` to `docs/user-guide/ag-ui/overview`
  and update all internal cross-references.
- Add a new `overview.mdx` providing a deep dive into AG-UI capabilities
  such as streaming, tool lifecycle events, and human-in-the-loop
  interrupts.
- Enhance `human_in_the_loop.mdx` and `approval_required.mdx` with notes
  explaining how agents served via `AGUIStream` handle interrupts
  differently than standard agents.
- Update `mint-json-template.json.jinja` to reflect the new file
  structure in the sidebar.
Replace the built-in `TimeoutError` with `asyncio.TimeoutError` in
`ag2/ag_ui/interrupts.py` to ensure compatibility with Python 3.10+
where these are distinct exception classes. This ensures that
timeouts triggered by the `ask` method are correctly caught.
@codecov

codecov Bot commented Sep 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.12230% with 12 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
ag2/ag_ui/interrupts.py 95.84% 7 Missing and 5 partials ⚠️
Files with missing lines Coverage Δ
ag2/a2ui/dispatch.py 93.82% <100.00%> (+0.49%) ⬆️
ag2/a2ui/server.py 97.05% <100.00%> (+1.05%) ⬆️
ag2/a2ui/transports/ag_ui.py 93.02% <100.00%> (+1.35%) ⬆️
ag2/a2ui/transports/base.py 100.00% <100.00%> (ø)
ag2/a2ui/transports/rest.py 85.48% <100.00%> (+0.23%) ⬆️
ag2/ag_ui/__init__.py 75.00% <100.00%> (+3.57%) ⬆️
ag2/ag_ui/asgi.py 86.66% <100.00%> (+2.05%) ⬆️
ag2/ag_ui/stream.py 91.46% <100.00%> (+0.59%) ⬆️
ag2/context.py 100.00% <100.00%> (ø)
ag2/events/__init__.py 100.00% <ø> (ø)
... and 3 more

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:a2ui A2UI support (ag2/a2ui) area:ag-ui AG-UI integration (ag2/ag_ui) area:core Agent runtime core: ag2/*.py, ag2/events, ag2/response, ag2/streams area:deps Python dependency updates (pyproject.toml, uv.lock) area:docs Documentation (website/) area:middleware Middleware system and builtins (ag2/middleware)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: Human input requests should reach the AG-UI client (interrupts / resume)

1 participant