Skip to content

Harden regular-runner context state and false completion guard#305

Open
mmprotest wants to merge 3 commits into
mainfrom
mmprotest/fix-regular-runner-task-completion-errors
Open

Harden regular-runner context state and false completion guard#305
mmprotest wants to merge 3 commits into
mainfrom
mmprotest/fix-regular-runner-task-completion-errors

Conversation

@mmprotest

Copy link
Copy Markdown
Owner

Motivation

  • Long regular-runner turns could be trimmed so aggressively that the model lost durable task context (objective, success predicate, verification state, next action) and returned generic conversational replies that the runner incorrectly treated as successful completion.
  • Existing execution-memento support was not sufficient when the saved memento was missing or incomplete, allowing unsafe prompt assembly and false positive termination.
  • A minimal, localized fix was required to ensure durable task state is always present for regular execution turns and to block obvious generic end-turn replies when verification is unresolved.

Description

  • Added build_fallback_execution_state_block(runner) to villani_code/execution_memento.py to render a compact, structured fallback block containing Objective, Success, Current verification, Next action, and Constraints.
  • Updated _inject_execution_state_memory in villani_code/state_runtime.py to prefer a valid saved memento but always inject the fallback block when the memento is missing or incomplete, and to include local evidence when available.
  • Strengthened trimming logic in _trim_regular_turn_messages to preserve all system messages, the earliest non-tool user message as a task anchor, and the recent tail (keeping tool-use/tool-result atomicity), and added a final invariant _ensure_regular_execution_state_invariant that guarantees Objective:, Success:, and Next action: markers appear in prepared messages.
  • Implemented a lightweight completion guard: should_block_regular_completion_on_generic_reply in state_runtime.py and a runner wrapper _should_block_regular_completion_on_generic_reply, and invoked this from the regular loop in villani_code/state.py to catch obvious generic no-tool replies when verification is still failing and either retry once with a focused continuation prompt or terminate as stalled_context_loss on repeat.
  • Added focused unit tests and small updates to tests/test_state_runtime.py and tests/test_loop.py to validate fallback injection, trimming behavior, memento preference, no-duplicate injection, tool-sequence integrity, and the false-completion guard.

Testing

  • Ran the targeted unit tests for the changed behavior and new cases: tests/test_state_runtime.py::test_prepare_messages_for_regular_runner_injects_execution_memento, tests/test_state_runtime.py::test_prepare_messages_for_model_does_not_duplicate_memento_injection, tests/test_state_runtime.py::test_prepare_messages_falls_back_when_memento_required_fields_missing, tests/test_state_runtime.py::test_prepare_messages_falls_back_when_memento_missing, tests/test_state_runtime.py::test_regular_trim_keeps_original_task_anchor_and_recent_tail, tests/test_state_runtime.py::test_prepare_messages_uses_saved_memento_over_fallback, and tests/test_loop.py::test_regular_runner_generic_reply_with_failing_verification_is_not_success, which all passed.
  • Also ran additional smoke tests covering tool-sequence/trimming integration (tests/test_state_runtime.py::test_prepare_messages_for_model_regular_trim_keeps_tool_sequence_integrity, tests/test_state_runtime.py::test_prepare_messages_for_model_repeated_calls_do_not_duplicate_injection) and loop tool-result behavior (tests/test_loop.py::test_loop_appends_tool_result_with_matching_id), which also passed.
  • No other tests were modified and the changes are limited to villani_code/execution_memento.py, villani_code/state_runtime.py, villani_code/state.py, and the localized tests mentioned above.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant