ToolEnvironment crashes on explicit `finish` tool calls: arguments is a JSON string, not a dict

[howl-anderson]

Summary

ToolEnvironment._extract_llm_answer raises AttributeError: 'str' object has no attribute 'get' when the model ends an episode with an explicit finish tool call (e.g. Qwen-format <tool_call>{"name": "finish", "arguments": {"response": "..."}}</tool_call>), instead of plain text.

Root cause

The two code paths in ToolAgent.update_from_model produce inconsistent types for function.arguments:

• Fallback path (no tool call parsed → synthetic finish): arguments is a dict — {"response": response}.
• Parser path (tool call parsed from text): arguments is json.dumps-stringified:

# tunix/rl/agentic/agents/tool_agent.py
args = tool_call.arguments
if isinstance(args, dict):
    args = json.dumps(args)

ToolEnvironment._extract_llm_answer only handles the dict shape:

# tunix/rl/agentic/environments/tool_environment.py
args = call["function"].get("arguments", {})
return args.get("response", "")   # AttributeError when args is a JSON string

So any model that learned to emit finish through the documented tool-call format (rather than by answering in plain text) crashes the rollout. Note the episode-termination check just above (name == "finish") matches fine — only the answer extraction breaks.

Repro

Drive TrajectoryCollectEngine with a scripted model_call whose final turn returns:

<tool_call>{"name": "finish", "arguments": {"response": "The answer is 42."}}</tool_call>

with a ToolAgent (qwen parser) + ToolEnvironment(reward_fn=...) pair:

  File ".../tunix/rl/agentic/environments/tool_environment.py", line 183, in _extract_llm_answer
    return args.get("response", "")
           ^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

Verified present at current main.

Suggested fix

Make _extract_llm_answer tolerant of the stringified form:

args = call["function"].get("arguments", {})
if isinstance(args, str):
    try:
        args = json.loads(args)
    except json.JSONDecodeError:
        return args
return args.get("response", "")

(or alternatively, stop stringifying arguments in ToolAgent.update_from_model — but other consumers may rely on the string form).

Fix

Proposed fix with regression tests: #2049