feat(evaluators): 17516 request payload contract - #271
Open
namrataghadi-galileo wants to merge 15 commits into
Open
namrataghadi-galileo wants to merge 15 commits into
namrataghadi-galileo wants to merge 15 commits into
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
josjeon
reviewed
Sep 28, 2026
josjeon
reviewed
Sep 28, 2026
wrisa
reviewed
Sep 29, 2026
wrisa
approved these changes
Sep 29, 2026
…d-contract' into feature/SAO-17516-request-payload-contract # Conflicts: # evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/client.py # evaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/evaluator.py # evaluators/contrib/galileo/tests/test_luna_evaluator.py
…O-17516-request-payload-contract
…O-17516-request-payload-contract
namrataghadi-galileo
changed the base branch from
main
to
feature/SAO-18049-add-children-to-step
October 6, 2026 00:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
toolsargument to@controlfor static definitions or a dynamic provider. Tool definitions are included only on LLM steps; omittingtoolspreserves existing decorator behavior.Steptree.Trace.childrenmap to spans,Session.childrenmap to traces, and tool/retriever children map to nested spans.Step.childrenas the only source of execution hierarchy. Reject hierarchy fields inStep.contextwhile preserving ordinary metadata.inputstogether with a canonicalrecordwhen a supported runtimeStepis available. Selector-selected values are passed into the shared records factory; the existingScorerInvokeRecordrequest model and legacy request shape remain unchanged.galileo-coremodels and cover serialization, request-model validation, and Core-model revalidation in tests.Scope
toolson@control, accepting static definitions or a(args, kwargs)provider; SDK documentation updated. Existing calls without tools remain compatible.Step.children; Luna structured-record dual-write alongside legacy inputs; field-level normalization and request validation; regression tests.execute_scorer,tool_calls, Agent/Workflow/Control nested spans, framework-specific integrations, persistence, and changes to Orbit or Luna's public request/response contracts. The scorer version remains caller-supplied; the client does not infer one.Risk and Rollout
Testing
@controltools,Step.childrenrecord trees, context hierarchy rejection, Luna request serialization, and Galileo Core serialization/revalidation.git diff --checkpassed for the changed package files.make checkcould not start because the configured Artifactory package index returned 403 while resolvinggoogle-re2.Checklist
execute_scorerpath and addingtool_callsare separate work.