Skip to content

'charmap' codec can't decode byte 0x81 in position 1980: character maps to <undefined> #433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Damil001 opened this issue Apr 28, 2025 · 6 comments · May be fixed by #439
Open

'charmap' codec can't decode byte 0x81 in position 1980: character maps to <undefined> #433

Damil001 opened this issue Apr 28, 2025 · 6 comments · May be fixed by #439
Assignees
Labels
core Issues related to the core interface and implementation good first issue Good for newcomers

Comments

@Damil001
Copy link

Whenever I try to use LlmAgent with LiteLlm I am getting this error

'charmap' codec can't decode byte 0x81 in position 1980: character maps to

I have pasted the code below any help would be much appreciated

`import os
import asyncio
import warnings
import logging

from google.adk.agents import Agent
from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm # For multi-model support
from google.adk.sessions import InMemorySessionService
from google.adk.runners import Runner
from google.genai import types # For creating message Content/Parts

Ignore all warnings

warnings.filterwarnings("ignore")

Configure logging

logging.basicConfig(level=logging.ERROR)

Set API keys

os.environ['OPENAI_API_KEY'] = "My key here"

print("API Keys Set:")
print(f"Google API Key set: {'Yes' if os.environ.get('GOOGLE_API_KEY') and os.environ['GOOGLE_API_KEY'] != 'YOUR_GOOGLE_API_KEY' else 'No (REPLACE PLACEHOLDER!)'}")
print(f"OpenAI API Key set: {'Yes' if os.environ.get('OPENAI_API_KEY') and os.environ['OPENAI_API_KEY'] != 'YOUR_OPENAI_API_KEY' else 'No (REPLACE PLACEHOLDER!)'}")
print(f"Anthropic API Key set: {'Yes' if os.environ.get('ANTHROPIC_API_KEY') and os.environ['ANTHROPIC_API_KEY'] != 'YOUR_ANTHROPIC_API_KEY' else 'No (REPLACE PLACEHOLDER!)'}")

Configure ADK to use API keys directly (not Vertex AI for this multi-model setup)

os.environ["GOOGLE_GENAI_USE_VERTEXAI"] = "False"

Models

MODEL_GEMINI_2_0_FLASH = "gemini-2.0-flash"
MODEL_GPT_4O = "openai/gpt-4o"
MODEL_CLAUDE_SONNET = "anthropic/claude-3-sonnet-20240229"

print("\nEnvironment configured.")

Define tool

def get_weather(city: str) -> dict:
"""Retrieves the current weather report for a specified city.

Args:
    city (str): The name of the city (e.g., "Mumbai","Chennai","Delhi").

Returns:
    dict: A dictionary containing the weather information.
          Includes a 'status' key ('success' or 'error').
          If 'success', includes a 'report' key with weather details.
          If 'error', includes an 'error_message' key.
"""
print(f"--- Tool: get_weather called for city: {city} ---")

city_normalized = city.lower().replace(" ", "")  # Normalize input

mock_weather_db = {
    "delhi": {"status": "success", "report": "The weather in Delhi is sunny with a temperature of 35°C."},
    "mumbai": {"status": "success", "report": "It's humid in Mumbai with a temperature of 30°C."},
    "bangalore": {"status": "success", "report": "Bangalore is experiencing light showers and a temperature of 22°C."},
    "kolkata": {"status": "success", "report": "Kolkata is partly cloudy with a temperature of 29°C."},
    "chennai": {"status": "success", "report": "It's hot and humid in Chennai with a temperature of 33°C."},
}

if city_normalized in mock_weather_db:
    return mock_weather_db[city_normalized]
else:
    return {"status": "error", "error_message": f"Sorry, I don't have weather information for '{city}'."}

Example tool usage

print(get_weather("Mumbai"))

Set up agent

AGENT_MODEL = MODEL_GPT_4O
weather_agent = LlmAgent(
name="weather_agent_v1",
model= LiteLlm(AGENT_MODEL),
description="Provides weather information for specific cities.",
instruction=(
"You are a helpful weather assistant. Your primary goal is to provide current weather reports. "
"When the user asks for the weather in a specific city, "
"you MUST use the 'get_weather' tool to find the information. "
"Analyze the tool's response: if the status is 'error', inform the user politely about the error message. "
"If the status is 'success', present the weather 'report' clearly and concisely to the user. "
"Only use the tool when a city is mentioned for a weather request."
),
tools=[get_weather],
)

print(f"Agent '{weather_agent.name}' created using model '{AGENT_MODEL}'.")
`

@AlankritVerma01
Copy link
Contributor

Hey @Damil001, thanks for reporting this!

That error happens because Python is using your system’s default text encoding (on Windows that’s often “cp1252”) and it ran into a byte it can’t handle.

Quick fix: wherever you do

with open(path, "r") as f:
    data = json.load(f)

change it to

with open(path, "r", encoding="utf-8") as f:
    data = json.load(f)

That forces UTF-8 decoding and will avoid the “charmap” error.

I’m also rolling up a small PR right now to add encoding="utf-8" to all our file reads and writes in ADK so this won’t come up again. I’ll drop a link here when it’s ready.

@hangfei hangfei self-assigned this Apr 28, 2025
@hangfei hangfei added the core Issues related to the core interface and implementation label Apr 28, 2025
@hangfei
Copy link
Collaborator

hangfei commented Apr 28, 2025

Thanks. Please share your PR and we will review it once ready.

@AlankritVerma01
Copy link
Contributor

@hangfei I’ve just opened this PR #439 to ensure all open() calls specify encoding="utf-8".
Most files already had it, but I added it where it was missing.
There’s a bit of JavaScript in the repo too—unlikely to be affected, but worth keeping an eye on.

Please take a look when you get a chance. Thanks!

@Damil001
Copy link
Author

Hey @Damil001, thanks for reporting this!

That error happens because Python is using your system’s default text encoding (on Windows that’s often “cp1252”) and it ran into a byte it can’t handle.

Quick fix: wherever you do

with open(path, "r") as f:
data = json.load(f)
change it to

with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
That forces UTF-8 decoding and will avoid the “charmap” error.

I’m also rolling up a small PR right now to add encoding="utf-8" to all our file reads and writes in ADK so this won’t come up again. I’ll drop a link here when it’s ready.

Hi @AlankritVerma01 ,

I just wanted to clarify the issue, the agents works fine when I am using Gemini Model but when I try to swap the model with other LLM and use LiteLLM object I get this error, I am stuck with this for 2 days now, any help would be much appreciated.

Thanks

@AlankritVerma01
Copy link
Contributor

@Damil001 thanks for the additional context! I’m not entirely sure what’s going on yet—could you share a minimal snippet of the code you’re running (including how you instantiate and call LiteLlm) that reproduces this decode error? It’d also help to see exactly what the JSON reader in lite_llm.py is seeing—can you print out the raw bytes or repr of the data around lines 131–132 here:
https://github.com/google/adk-python/blob/main/src/google/adk/models/lite_llm.py#L131-L132

And just to double‐check: you’ve already pulled in the PR that adds encoding="utf-8" to all our open() calls and reinstalled, but the problem persists? If that’s true, we may need to dig into whether LiteLLM itself is doing any file reads (or string handling) under the hood that aren’t using UTF-8. Let me know what you find and we’ll take it from there!

@AlankritVerma01
Copy link
Contributor

@hangfei if all if well, should we go ahead with #439 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Issues related to the core interface and implementation good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants