Skip to content

feat(evaluators): 17516 request payload contract - #271

Open
namrataghadi-galileo wants to merge 15 commits into
feature/SAO-18049-add-children-to-stepfrom
feature/SAO-17516-request-payload-contract
Open

namrataghadi-galileo wants to merge 15 commits into
feature/SAO-18049-add-children-to-stepfrom
feature/SAO-17516-request-payload-contract

Conversation

@namrataghadi-galileo

@namrataghadi-galileo namrataghadi-galileo commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add an optional tools argument to @control for static definitions or a dynamic provider. Tool definitions are included only on LLM steps; omitting tools preserves existing decorator behavior.
  • Build canonical Galileo Core records from the complete Agent Control Step tree. Trace.children map to spans, Session.children map to traces, and tool/retriever children map to nested spans.
  • Use Step.children as the only source of execution hierarchy. Reject hierarchy fields in Step.context while preserving ordinary metadata.
  • Have Luna send the legacy inputs together with a canonical record when a supported runtime Step is available. Selector-selected values are passed into the shared records factory; the existing ScorerInvokeRecord request model and legacy request shape remain unchanged.
  • Normalize record fields with galileo-core models and cover serialization, request-model validation, and Core-model revalidation in tests.

Scope

  • User-facing/API changes: optional keyword-only tools on @control, accepting static definitions or a (args, kwargs) provider; SDK documentation updated. Existing calls without tools remain compatible.
  • Internal changes: recursive Galileo record conversion from Step.children; Luna structured-record dual-write alongside legacy inputs; field-level normalization and request validation; regression tests.
  • Out of scope: calling Orbit's execute_scorer, tool_calls, Agent/Workflow/Control nested spans, framework-specific integrations, persistence, and changes to Orbit or Luna's public request/response contracts. The scorer version remains caller-supplied; the client does not infer one.

Risk and Rollout

  • Risk level: moderate. Luna requests continue to include legacy inputs, and no database schema changes are included.
  • Rollback plan: revert this PR. No data migration is required.

Testing

  • Added or updated tests for @control tools, Step.children record trees, context hierarchy rejection, Luna request serialization, and Galileo Core serialization/revalidation.
  • Galileo package test suite: 187 passed using the existing package virtual environment.
  • Ruff, mypy, and git diff --check passed for the changed package files.
  • Repo-wide make check could not start because the configured Artifactory package index returned 403 while resolving google-re2.
  • Fresh CI is pending for the updated PR head. The previous run was against the pre-fix head and reported five Galileo test failures plus a generated TypeScript client check failure.
  • Manually verified behavior; validation is through automated tests and code review.

Checklist

  • Issue/spec: SAO-17516 (referenced in the PR title).
  • Updated SDK documentation for the decorator API and direct Step construction.
  • Follow-up noted: wiring Luna to Orbit's execute_scorer path and adding tool_calls are separate work.

@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.29412% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...agent_control_evaluator_galileo/records/factory.py 93.18% 3 Missing ⚠️
...src/agent_control_evaluator_galileo/luna/client.py 92.85% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Comment thread evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/evaluator.py Outdated
Comment thread evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/evaluator.py Outdated
Comment thread evaluators/contrib/galileo/pyproject.toml Outdated
Comment thread evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/client.py Outdated
Comment thread sdks/python/src/agent_control/control_decorators.py
…d-contract' into feature/SAO-17516-request-payload-contract

# Conflicts:
#	evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/client.py
#	evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/evaluator.py
#	evaluators/contrib/galileo/tests/test_luna_evaluator.py
@namrataghadi-galileo
namrataghadi-galileo changed the base branch from main to feature/SAO-18049-add-children-to-step October 6, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants