Skip to content

Prevent the Part 8 order agent from offering unsupported notifications #657

Description

@jongalloway

Issue type

enhancement

Requested priority

P3 - Polish, minor documentation, screenshots, or future improvement

Workshop area

Part 8 - Agent Framework Basics

Summary

During live validation, the Part 8 agent correctly called GetOrderStatus but then offered to notify the user when the order shipped or provide future tracking updates. The sample exposes no notification or tracking-subscription tool, so the offer cannot be fulfilled.

Attendee or instructor impact

The core lesson passes, but the output demonstrates a common agent-design problem: the model promises capabilities that are not represented by its tools. An instructor may need to explain why the offered follow-up cannot actually work.

Because model wording is nondeterministic, this is an intermittent polish issue rather than a guaranteed failure.

Reproduction or proposed change

From a fresh Part 8 project configured with gpt-5-mini, run the documented application and ask:

What's the status of order ORD-1001?
How about ORD-1002?
When will ORD-1001 arrive?

In the September 13 validation, the tool-derived dates and statuses were correct, but responses included offers such as:

Would you like me to notify you when that happens?

and:

Would you like me to notify you when it's out for delivery or provide tracking updates?

The application only has GetOrderStatus; it cannot schedule notifications or retrieve future updates.

Proposed change:

  1. Strengthen the agent instructions to state that it must not promise notifications, subscriptions, or future tracking updates.
  2. Tell it to limit answers to the data returned by available tools and suggest only actions the sample can perform.
  3. Optionally use this as a short teaching note about capability boundaries and tool-grounded agents.

Expected outcome

The agent answers order-status questions accurately without offering actions that have no corresponding application/tool implementation.

Validation

  • Build the Part 8 snapshot in Release with zero warnings.
  • Run several multi-turn order-status conversations against the configured model.
  • Confirm the agent invokes GetOrderStatus for each referenced order.
  • Confirm it preserves conversational context.
  • Confirm responses do not promise notifications, subscriptions, or unsupported future tracking.
  • Confirm no vulnerable packages.

Dependencies and related issues

No open duplicate found. This is independent of the Aspire 13.4.6 PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-decisionScope or product direction must be decided before implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions