[Bug]: LiteLLM/OpiK integration not working

[jquinter]

What component(s) are affected?

• Python SDK
• Opik UI
• Opik Server
• Documentation

Opik version

• Opik version: 1.6.5

Describe the problem

By following the integration guidelines on both Opik and LiteLLM, I am unable to succesfully connect these two platforms.

The observed issue is simple: despite having set up a valid Opik's api key (confirmed to work via another platform, and curl requests) and opik workspace, nothing is received at Opik.

Reproduction steps and code snippets

• Use base docker image directly from ghcr.io/berriai/litellm:main-latest
• Tried both a Google Cloud Kubernetes deployment (on GKE), as well as a container running locally on my laptop (using podman)
• In both cases, these relevant environment variables were set: OPIK_API_KEY and OPIK_WORKSPACE
• What I have checked inside the running pod container: if I run pip freeze|grep opik, nothing is returned (this is, python package for pip is not installed in base docker image)

Error logs or stack trace

No error or debug message is emmited at all, regarding tracking or sending messages.

I noticed just one ocurrence of the following, but it seems to be related to another different issue

INFO:     192.168.127.1:43289 - "POST /health/test_connection HTTP/1.1" 200 OK
22:21:25 - LiteLLM Proxy:ERROR: proxy_track_cost_callback.py:205 - Error in tracking cost callback - User API key and team id and user id missing from custom callback.
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/proxy/hooks/proxy_track_cost_callback.py", line 174, in _PROXY_track_cost_callback
    raise Exception(
        "User API key and team id and user id missing from custom callback."
    )
Exception: User API key and team id and user id missing from custom callback.

Healthcheck results

I am running hosted opik, no healthcheck applies.

[jverre]

Hi @jquinter
Just to confirm, you are using the LiteLLM proxy rather than the the LiteLLM python library, right ?

[jquinter]

Hi @jquinter Just to confirm, you are using the LiteLLM proxy rather than the the LiteLLM python library, right ?

Right: I am using the proxy.

[HenryzhaoH]

Just met the same problem.
Got this Error message filled in my log after turning on the background_health_checks .

general_settings: 
  background_health_checks: True 

It seems that when doing health check, no user_api_key is set, but that should be ok.

Digging into this error, in file proxy_track_cost_callback.py#L134-139.

If _should_track_cost_callback returned false, an exception "User API key and team id and user id missing from custom callback." will be raised. That's the problem.

  if _should_track_cost_callback(
      user_api_key=user_api_key,
      user_id=user_id,
      team_id=team_id,
      end_user_id=end_user_id,
  ):
......
  else:
      raise Exception(
          "User API key and team id and user id missing from custom callback."
      )

Further more, setting disable_spend_updates in general_settings won't fix for this error but make it worse.
It will cause this exception been raised on every request, including the ones have user_api_key set.

Because _should_track_cost_callback will always return False.

def _should_track_cost_callback(
    user_api_key: Optional[str],
    user_id: Optional[str],
    team_id: Optional[str],
    end_user_id: Optional[str],
) -> bool:
    """
    Determine if the cost callback should be tracked based on the kwargs
    """

    # don't run track cost callback if user opted into disabling spend
    if ProxyUpdateSpend.disable_spend_updates() is True:
        return False

    if (
        user_api_key is not None
        or user_id is not None
        or team_id is not None
        or end_user_id is not None
    ):
        return True
    return False

[Lothiraldan]

Hi @jquinter, @HenryzhaoH,
Thanks for your patience. We wanted to update you that we’ll be investigating this issue this week. We’ll share any findings as soon as we have more information.

[Lothiraldan]

Hi @jquinter, @HenryzhaoH,

I’m also seeing the error:
User API key and team id and user id missing from custom callback
when enabling background_health_checks, even without enabling the Opik integration. Based on my findings, I believe this issue is unrelated to Opik integration or any callback.

That said, I haven’t been able to reproduce the missing traces in Opik when enabling background_health_checks. I tested enabling the Opik callback in both litellm_settings.callbacks and litellm_settings.success_callback. A couple of things to note:

• Traces can take a few seconds to appear in the UI.
• The LiteLLM configuration keys can be a bit confusing—at the top level of the LiteLLM settings, Opik can be configured under either callbacks (with an "s") or success_callback (without an "s").

To better understand your setup and try reproducing the issue more accurately, could you provide some additional details?

• LiteLLM version: Which version are you using? If you’re using a Docker image, you can get the exact image ID with docker compose image.
• Database setup: Do you have a database configured for your LiteLLM proxy installation?
• Configuration details: Could you share the relevant portion of your LiteLLM proxy configuration where Opik is enabled? If you prefer to share it privately, feel free to send it to me at boris@comet.com.
• LiteLLM Proxy usage: How are you calling the LiteLLM proxy? Which LiteLLM proxy endpoint and client library are you using? Any code snippets would be helpful.

Thanks again for your patience—I’ll do my best to get to the bottom of this!

[HenryzhaoH]

Hi @Lothiraldan,

Thank you for getting back to me!

After investigating, I believe this is actually a bug originating from LiteLLM rather than being related to OpiK. I've gone ahead and linked this issue to an existing bug report in the LiteLLM repository. Hopefully, they'll be able to roll out a fix soon.

Best regards.