fix: #2061 Sanitize invalid JSON in tool call arguments

[habema]

Resolves #2061

Fixes issue where invalid JSON in tool call arguments (stored in session history) would break API calls when using Anthropic models via LiteLLM.

Solution:

• Added _sanitize_tool_call_arguments() method that validates JSON before sending to APIs
• If invalid, sanitizes to "{}" as safe default (prevents API errors while maintaining compatibility)
• Tests

Not too sure if the logging should be more verbose, I'll leave that to you.

[seratch]

Thanks for sending this PR. Validating the JSON data format itself is fine (OpenAI SDK's code comment recommends doing so too), but I am unsure if falling back to an empty JSON data works well in the scenario, and it may be even more confusing for developers to debug. Sending an invalid JSON data to the model and receiving the error clearly telling the arguments are broken may be okay.

[habema]

@seratch Your concern is valid, but this issue causes sessions to break entirely. Also, when this issue occurs, the currently raised errors are just as confusing for debugging; it is not clear that the issue is in a certain tool call's arguments.

Examples:

litellm.BadRequestError: AnthropicException - Expecting ',' delimiter: line 1 column 42 (char 41)
Received Messages=[...] # Entire input list

litellm.BadRequestError: AnthropicException - Unterminated string starting at: line 1 column 47 (char 46)`
Received Messages=[...] # Entire input list

Perhaps this SDK could raise more informative error, or proceed with the sanitization but with a warning level instead of debug. I'm also open to hearing your suggestions.

[ihower]

@habema The json.loads error actually comes from LiteLLM. So if we want to improve the error message, the right place to fix it may be in LiteLLM, not here.

From a broader perspective, I think the root issue is related to the current session design.
As I mentioned in #1930 (comment), the current Session implementation saves those problematic items before the LLM request is sent. Once this happens, the conversation thread becomes permanently broken, because every later call will hit the same API error since the corrupted items are always included in the following request. The only workaround is to start a new thread.

Maybe we could provide a small helper function to clear out problematic session items, but deciding when to use it should probably be left to the developer, since we cannot know exactly which server-side API errors require clearing. It’s more like a retry or recovery feature.

Just my two cents.

[habema]

@ihower Thank you.

Yes, the litellm.BadRequestError is raised by LiteLLM. As mentioned in the original issue, this behavior is right, as the tool call's arguments violate Anthropic's API specs. That being said, the right place to fix such an issue is here, not LiteLLM, as this SDK is the one responsible for using LiteLLM's (and by extension Anthropic's) public API correctly.

As for the session's design, I think this can be very opinionated. In my opinion, I think the session should reflect the exact interactions that happen without alteration. If any sanitization is needed, this can be done inside the get_items() within a custom session implementation, which is how I managed to circumvent this issue.

On one side, this takes away the responsibility of handling such problems from this SDK's core internals, and delegates to the session, which can be implemented in a custom manner (and at this point I believe any production use-case will need its own custom session implementation). On the other side, and as @seratch mentioned earlier:

Sending an invalid JSON data to the model and receiving the error clearly telling the arguments are broken may be okay.

After all the broken JSON is what the agent sent.

Since I managed to solve this issue by implementing a function very similar _sanitize_tool_call_arguments() with my custom session, my only concern moving forward is making this easier to debug within the SDK. I am open to hearing suggestions regarding that, and if they are unrelated to the changes in this PR, this can be closed.