Repository navigation
fix(sdk): return an error to the model for malformed tool-call arguments - #1168
santoshkumarradha merged 3 commits into
Conversation
When an LLM tool call carried invalid or truncated JSON arguments, both the
Python and Go SDKs silently replaced them with {} and still executed the
tool, so the tool ran with missing/default inputs and the model never learned
its call was broken. Report the parse error back as a tool message (like the
existing missing-"arguments" case) and skip execution. An empty arguments
string is still accepted as "no arguments", since some providers send that for
parameterless tools.
Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>
Performance
✓ No regressions detected |
santoshkumarradha
left a comment
There was a problem hiding this comment.
Thanks for tightening malformed tool-call handling. The intent looks right, but the Python CI is red because the existing error-path tests still index trace.calls[0] after this change. Invalid arguments now correctly avoid creating a tool-call record, so please update those assertions to inspect the tool error message / traced error instead, then rerun the Python suite.
Invalid arguments no longer create a tool-call record, so assert on the tool error message fed back to the model instead of trace.calls.
|
Thanks, updated. This comment was drafted with Claude Code. |
📊 Coverage gateThresholds from
✅ Gate passedNo surface regressed past the allowed threshold and the aggregate stayed above the floor. |
📐 Patch coverage gateThreshold: 80% on lines this PR touches vs
✅ Patch gate passedEvery surface whose lines were touched by this PR has patch coverage at or above the threshold. |
santoshkumarradha
left a comment
There was a problem hiding this comment.
Thanks for updating the error-path coverage. The tests now assert the reported tool error without assuming a call record exists, which matches the intended malformed-argument behavior. Focused Go and Python suites pass locally.
Root cause. In
execute_tool_call_loop(Python,tool_calling.py) andExecuteToolCallLoop(Go,tool_calling.go), a tool call whoseargumentsis not valid JSON (e.g. truncated bymax_tokens) was swallowed (except JSONDecodeError: func_args = {}/json.Unmarshalerror ignored) and the tool was then executed with empty arguments. The tool could run with missing or default inputs and the model never learned its call was broken. In the Python SDK, the sibling case of a missingargumentsfield already returns an error to the model.Fix. Invalid, truncated, or non-object arguments now append a tool-error message (
... has invalid or truncated JSON arguments (...). Please retry with valid JSON arguments.) and skip execution, in both SDKs. An empty arguments string is still treated as "no arguments", because some providers send""for parameterless tools.Tests. Python:
test_malformed_arguments_are_not_executed(3 cases) andtest_empty_string_arguments_run_with_no_args. Go:TestExecuteToolCallLoop_MalformedArgumentsAreNotExecuted; the old test that asserted a tool ran with empty input after{bad jsonwas reworked to use an empty-string argument. The new tests fail before the change and pass after;go test ./ai/andtests/test_tool_calling.pypass.This PR was prepared with the help of an AI assistant (Claude Code) and submitted by a human who is responsible for it.