The "Include LLM Information in Access Log" feature doesn't seem to be working.

[leekari5548]

Description

The "Include LLM Information in Access Log" feature doesn't seem to be working.

https://apisix.apache.org/docs/apisix/plugins/ai-proxy/#include-llm-information-in-access-log

Environment

• APISIX version (run apisix version): 3.13.0
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[SkyeYoung]

Hello, may I ask if you can provide the minimum environment and steps to reproduce?

[leekari5548]

Using the following configuration:

nginx_config:
  http:
    access_log_format: "$remote_addr - $remote_user [$time_local] $http_host \"$request_line\" $status $body_bytes_sent $request_time \"$http_referer\" \"$http_user_agent\" $upstream_addr $upstream_status $upstream_response_time \"$upstream_scheme://$upstream_host$upstream_uri\" \"$apisix_request_id\" \"$request_type\" \"$llm_time_to_first_token\" \"$llm_model\" \"$llm_prompt_tokens\" \"$llm_completion_tokens\""

The APISIX-3.13.0 Docker image fails to start, with the error content shown in the image:

After commenting out access_log_format, logs are collected using the clickhouse-logger with the following configuration:

{
  "database": "default",
  "endpoint_addrs": [
    "http://host.docker.internal:8123/"
  ],
  "format": {
    "body_bytes_sent": "$body_bytes_sent",
    "bytes_sent": "$bytes_sent",
    "connection": "$connection",
    "connection_requests": "$connection_requests",
    "content_length": "$content_length",
    "content_type": "$content_type",
    "hostname": "$hostname",
    "http_authorization": "$http_authorization",
    "http_referer": "$http_referer",
    "http_user_agent": "$http_user_agent",
    "http_x_forwarded_for": "$http_x_forwarded_for",
    "http_x_real_ip": "$http_x_real_ip",
    "llm_completion_tokens": "$llm_completion_tokens",
    "llm_model": "$llm_model",
    "llm_prompt_tokens": "$llm_prompt_tokens",
    "llm_time_to_first_token": "$llm_time_to_first_token",
    "pid": "$pid",
    "proxy_protocol_addr": "$proxy_protocol_addr",
    "proxy_protocol_port": "$proxy_protocol_port",
    "remote_addr": "$remote_addr",
    "remote_port": "$remote_port",
    "remote_user": "$remote_user",
    "request_body": "$request_body",
    "request_id": "$request_id",
    "request_length": "$request_length",
    "request_method": "$request_method",
    "request_time": "$request_time",
    "request_uri": "$request_uri",
    "sent_http_content_length": "$sent_http_content_length",
    "sent_http_content_type": "$sent_http_content_type",
    "sent_http_location": "$sent_http_location",
    "sent_http_server": "$sent_http_server",
    "server_protocol": "$server_protocol",
    "status": "$status",
    "time_iso8601": "$time_iso8601"
  },
  "logtable": "apisix_logs",
  "password": "123456",
  "user": "default"
}

There is no LLM-related data in ClickHouse.

[leekari5548]

APISIX Image: apache/apisix:3.13.0-debian

[SkyeYoung]

I used the same apisix image as you and tried it out, and it did indeed reproduce the issue.

It seems there are some discrepancies between the documentation and the functionality.

[Baoyuantop]

@SkyeYoung, have you confirmed that document https://apisix.apache.org/docs/apisix/plugins/ai-proxy/ doesn't actually work?

[SkyeYoung]

@Baoyuantop Yes, I checked. In fact, the configuration the user pointed out is exactly the one in the APISIX documentation.

[Baoyuantop]

@kayx23 Please help check this document.