[Jiaqi Zhang (JiaqiZhang-Dev)]

Summary

• run the hosted QA chat agent through the Microsoft Agent Framework harness
• add compaction, todos, session-scoped file memory, skills, memory context, and telemetry
• protect untrusted tool output and offload large opaque results to session memory
• use hosted Responses protocol 2.0 and isolate stateless requests in fresh logical sessions
• update PR-review guidance to inspect stored large outputs through harness file-memory tools

Validation

• 99 focused harness, chat-service, hosted-agent, and middleware tests passed after merging current main
• 37 file-backed evaluation tests passed
• targeted Ruff checks passed
• dev hosted-agent deployment succeeded
• isolated-session General run returned all responses without shared-session transport failures

Evaluation

• [Teams Chatbot] Support file-backed QA bot evaluations #17509 is merged into main, providing file-backed inputs and configurable concurrency
• the same scenarios can now be rerun against main and this branch for a comparable result

Relates to #17491

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
70 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

Unresolved critical issues break hosted startup and expose credentials and private session data through unrestricted shell access, shared result storage, and untrusted-text spotlighting bypasses.

7 open issues

• Dockerfiles omit the middleware package · New
• Prompt lookup fails in the hosted Docker layout · New
• Memory reads and grep results bypass spotlighting · New
• Mixed text-and-image results bypass spotlighting · New
• Shared result store exposes data across sessions · New
• Local shell execution is not isolated from host access · New
• Large results link to files the agent cannot read · New

What changed in this PR

This PR advances #17491 by moving the hosted chat agent onto the Microsoft Agent Framework harness and replacing custom MCP result parsers with shared middleware.

Changes:

• Adds harness-managed file access, shell execution, session memory, skills, and context compaction.
• Separates tool-output security from large-result storage across agent entrypoints.
• Updates dependencies, tests, and guidance for inspecting stored results.

Paths below are relative to tools/sdk-ai-bots/azure-sdk-qa-bot-agent/.

File	Description
utils/​tool_security.py	Removes the previous security middleware module.
utils/​azure_ai_foundry.py	Adds the shared hosted harness factory.
tools/​mslearn_mcp_tools.py	Removes custom result-parser registration.
tools/​github_mcp_tools.py	Removes GitHub-specific filtering and compaction parsers.
tools/​azsdk_mcp_tools.py	Removes custom result-parser registration.
tools/​ado_mcp_tools.py	Removes custom result-parser registration.
tools/​__init__.py	Removes shared MCP truncation logic.
tests/​tool_security_test.py	Removes tests for the previous middleware.
tests/​tool_output_storage_test.py	Tests normalization, offloading, and middleware composition.
tests/​tool_output_security_test.py	Tests spotlighting and tool exemptions.
tests/​tenant_config_test.py	Updates skill-discovery tests for framework APIs.
tests/​mslearn_mcp_tools_test.py	Updates expected MCP configuration.
tests/​mcp_parser_test.py	Removes legacy truncation-parser tests.
tests/​github_mcp_parser_test.py	Removes GitHub-specific parser tests.
tests/​agent_harness_test.py	Adds harness-provider and shell tests.
skills/​chat_agent/​pr-review/​SKILL.md	Updates guidance for inspecting stored results.
SECURITY.md	Updates the middleware link and description.
requirements.txt	Updates framework and MCP dependencies.
prompts/​harness_instruction.md	Adds shared harness and context-management guidance.
middleware/​tool_output_storage.py	Stores large results and returns file references.
middleware/​tool_output_security.py	Adds relocated spotlighting with harness exemptions.
middleware/​__init__.py	Introduces the middleware package.
agents/​chatbot_evolution_agent/​init.py	Registers security and storage middleware.
agents/​chat_agent/​init.py	Adopts the harness and agent-managed history.
agents/​azure_mcp_server_agent/​init.py	Updates skill loading and middleware registration.

_{Review effort: Balanced}