fix(litellm): survive malformed JSON in tool call arguments - #6867
Open
adnanahamed66772ndpc wants to merge 2 commits into
Open
fix(litellm): survive malformed JSON in tool call arguments#6867adnanahamed66772ndpc wants to merge 2 commits into
adnanahamed66772ndpc wants to merge 2 commits into
Conversation
A truncated or otherwise malformed arguments string in a LiteLLM tool call raised json.JSONDecodeError from _message_to_generate_content_response, aborting the whole invocation before any callback or tool boundary could handle it. Parse the arguments defensively, log a warning with the tool call id and function name, and fall back to empty args so downstream tool dispatch reports the mismatch back to the model for a retry. Fixes google#5896
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
When the upstream model returns malformed JSON in
tool_call.function.arguments(truncated strings, unclosed objects, partial streams committed to the message),_message_to_generate_content_responseraisedjson.JSONDecodeError. The exception propagated out of LiteLLM response parsing — upstream of any tool or callback boundary — so the entire invocation aborted with a generic streaming error and no information about the malformed tool call. This is a regression of #5008, still present atv2.0.0.Solution:
Parse tool call arguments defensively inside
_message_to_generate_content_response: onjson.JSONDecodeError, log a warning carrying the tool call id, function name and parse error, then fall back to empty args on the function-call part. Downstream tool dispatch then surfaces a structured argument mismatch back to the model, which can retry with corrected JSON — the agent loop survives.Strict parsing is preserved in
_parse_tool_call_argumentsitself because the streaming path (_finalize_tool_call_response) relies on it to detect arguments truncated byfinish_reason == "length"and report a dedicated MAX_TOKENS error response; only the final conversion step now recovers instead of crashing.Testing Plan
Unit Tests:
I have added or updated unit tests for my change.
All unit tests pass locally.
Added
test_message_to_generate_content_response_tool_call_with_malformed_json_arguments: truncated arguments produce a function-call part with empty args while preserving name and id.Added
test_message_to_generate_content_response_tool_call_malformed_json_warns: the recovery emits a warning-level record.Full model suite:
pytest tests/unittests/models/test_litellm.py→ 395 passed, including the existing Python-literal and unquoted-key repair tests which confirm no behavior change for valid inputs.Reproduced the exact crash from [LiteLlm] JSONDecodeError on malformed tool_call.function.arguments still crashes invocation at v2.0.0 (regression of #5008) #5896 before the fix (JSONDecodeError propagating out of
_message_to_generate_content_response) and verified it no longer reproduces after the fix.pre-commit run --files(ruff, isort, pyink, addlicense, compliance checks) andmypy src/google/adk/models/lite_llm.pyall pass.Manual End-to-End (E2E) Tests:
Not applicable — the fix is covered by unit tests that construct the LiteLLM message directly, exactly matching the minimal reproduction from the issue. No network or provider credentials are required.