Something with tokens has changed and some integration may stop working

[otomay]

Hey! I was using 0.4.6.2 with langchain and crewai. It was working, but today I updated g4f to latest version and stoped working consistenly with the following error (random, the code works ~15% of the time):

self.token_cost_process.sum_cached_prompt_tokens(
File "/Users/xxx/miniconda3/envs/crewai/lib/python3.12/site-packages/crewai/agents/agent_builder/utilities/base_token_process.py", line 21, in sum_cached_prompt_tokens
self.cached_prompt_tokens += tokens
TypeError: unsupported operand type(s) for +=: 'int' and 'NoneType'

Here is my code example (not a quickstart to run, just to ilustrate how I'm using g4f):

from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory

os.environ["OPENAI_API_KEY"] = "ABC"
os.environ["OPENAI_API_BASE"] = "http://localhost:1337/v1"

# ...

@agent
def agent_test(self) -> Agent:
    return Agent(
        config=self.agents_config['agent_test'],
        llm=ChatOpenAI(model="gpt-4o"),
    )

I installed both versions and made a post request to each one. Here is the response results for both versions:

0.4.6.2

ChatCompletion(id='chatcmpl-tVvRzHv6ijeAsMa5v1oPANSJRi1t', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Hello! How can I assist you today?', refusal=None, role='assistant', audio=None, function_call=None, tool_calls=None))], created=1740604376, model=None, object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=7, prompt_tokens=0, total_tokens=7, completion_tokens_details=None, prompt_tokens_details=None), provider=None)

0.4.7.5

ChatCompletion(id='chatcmpl-sq80MhdL8wkUBi2SuiSMNOq7u4A7', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Hello! How can I assist you today?', refusal=None, role='assistant', audio=None, function_call=None, tool_calls=None))], created=1740604456, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=10, prompt_tokens=153, total_tokens=163, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=None, audio_tokens=None, reasoning_tokens=None, rejected_prediction_tokens=None), prompt_tokens_details=PromptTokensDetails(audio_tokens=None, cached_tokens=None)), provider='PollinationsAI')

It's clearly that something with tokens has changed, but I can't see why stopped working.
Is there a way to fix it?

[hlohaus]

Thank you for the excellent example; it has been incorporated into the pydantic_ai support by adding completion_tokens_details=None and prompt_tokens_details=None. I will investigate further options. @otomay

[hlohaus]

@otomay I have fixed this issue. Also i have added a langchain integration:

from g4f.tools.langchain import ChatAI

#2766

[otomay]

tysm man, that's awesome! gonna try it as soon as posible

[otomay]

hey man o/ The last version on pypi doesn't come with the tools/integration folders

[hlohaus]

@otomay oh, i will fix that. It's a missing init.py file.