chatgpt-proxy

A lightweight HTTP proxy server for OpenAI and Anthropic APIs. It wraps the official OpenAI and Anthropic SDKs and exposes simplified endpoints for chat completions, audio transcriptions, embeddings, and Claude messages with built-in authentication, logging, and metrics.

Features

Responses API — OpenAI's most advanced interface with tools, web search, file search, MCP, function calling, and streaming
Chat Completions — Full OpenAI Chat Completions API support including vision (images)
OpenAI-compatible endpoint — POST /v1/chat/completions with header-based auth for standard OpenAI SDKs and Spring AI (just override the base URL)
Anthropic Claude — Anthropic Messages API with streaming, vision, tool use, and extended thinking
Audio Transcriptions — Whisper-based speech-to-text
Embeddings — Generate text embeddings
Simple Chat — Simplified /chatgpt endpoint for quick prompts
Health Checks — Built-in health endpoints
Logging & Metrics — Request/response logs, token usage stats, error tracking
CORS — Enabled for all origins

Installation

npm install

Configuration

Create a .env or .env.local file in the project root:

OPENAI_API_KEY=sk-...
OPENAI_PROJECT_KEY=proj_...      # Optional
ANTHROPIC_API_KEY=sk-ant-...     # Required for /anthropic endpoints
SECURITY_KEY=your-secret-key     # Required for authenticated endpoints
OPENAI_PROXY_UPSTREAM_TIMEOUT_MS=600000
OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS=900000
OPENAI_PROXY_TRANSPORT_CONNECT_TIMEOUT_MS=30000
OPENAI_PROXY_TRANSPORT_HEADERS_TIMEOUT_MS=905000
OPENAI_PROXY_TRANSPORT_BODY_TIMEOUT_MS=905000
OPENAI_PROXY_SERVER_TIMEOUT_MS=935000
OPENAI_PROXY_KEEPALIVE_TIMEOUT_MS=65000
OPENAI_PROXY_MAX_PARALLEL_REQUESTS=32
OPENAI_PROXY_SSE_KEEPALIVE_INTERVAL_MS=15000

Reliability Controls

OPENAI_PROXY_UPSTREAM_TIMEOUT_MS sets the upstream OpenAI SDK timeout used when a caller does not provide timeout.
OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS caps caller-provided timeout values. Values above the cap are clamped.
OPENAI_PROXY_TRANSPORT_CONNECT_TIMEOUT_MS sets the explicit undici connect timeout for outbound OpenAI requests.
OPENAI_PROXY_TRANSPORT_HEADERS_TIMEOUT_MS sets the explicit undici response-headers timeout for outbound OpenAI requests.
OPENAI_PROXY_TRANSPORT_BODY_TIMEOUT_MS sets the explicit undici response-body idle timeout for outbound OpenAI requests.
OPENAI_PROXY_SERVER_TIMEOUT_MS sets the inbound client socket timeout (server.timeout, server.requestTimeout). It defaults to 30s above the latest upstream deadline and is clamped up to that floor if an operator configures a lower value.
OPENAI_PROXY_KEEPALIVE_TIMEOUT_MS sets how long an idle connection is held for reuse between requests (server.keepAliveTimeout), default 65s. This is deliberately independent of the request socket timeout: raising the upstream budget must not also make the proxy hoard idle sockets for the same span.
OPENAI_PROXY_MAX_PARALLEL_REQUESTS bounds concurrent OpenAI work inside the proxy. When the limit is reached, the proxy rejects new upstream work with 503 and Retry-After: 1.
OPENAI_PROXY_SSE_KEEPALIVE_INTERVAL_MS sets the SSE keep-alive interval for /openai2 streaming responses. While a stream is open and no upstream event has been forwarded during an interval, the proxy writes an SSE comment line (: keep-alive) so intermediary read timeouts do not kill the stream during long silent reasoning phases. Comment lines are ignored by standards-compliant SSE parsers.

By default, the transport headersTimeout and bodyTimeout are set above the proxy's maximum upstream timeout so undici does not terminate long-running /openai2 calls earlier than the configured OpenAI SDK budget unless an operator explicitly chooses a lower transport timeout.

The inbound socket timeout is layered above both. A client socket is idle for the whole of a long non-streaming upstream call, so if server.timeout were not strictly greater than every upstream deadline, Node would destroy the socket before the proxy could write its 504. Callers may therefore request the full OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS and still receive a classified 504.

Should the inbound socket cap fire anyway, the proxy handles it rather than letting the connection reset. It aborts upstream work and answers 504 with code OPENAI_PROXY_SOCKET_TIMEOUT, logging proxy.request.socket_timeout. A socket close is otherwise attributed by cause, recorded as disconnectCause on proxy.request.complete:

`disconnectCause`	Meaning	Result
`client`	The caller hung up while upstream work was still in flight	`499` `client_cancelled`, counted as a cancellation
`proxy_socket_timeout`	This proxy's own `server.timeout` fired	`504` `OPENAI_PROXY_SOCKET_TIMEOUT`
`upstream_deadline_elapsed`	The close landed after every upstream attempt could have timed out — typically an intermediary with its own cap	`504` `OPENAI_PROXY_TIMEOUT`

Only client increments the cancellation metric, so a middle hop or the proxy's own cap tearing down a socket is no longer reported as caller-initiated cancellation.

A hop in front of this proxy must be more patient than it is

The timeout ladder only works if every layer is strictly more patient than the one it wraps. This proxy owns the inner three; the hop in front owns the outermost and is not derived from anything here, so raising the upstream budget silently inverts the ladder unless that hop is raised too.

undici headers/body      905_000   = upstream max + transport grace
server.timeout           935_000   = latest upstream deadline + 30_000
server.headersTimeout    985_000   = server.timeout + 50_000
reverse proxy read       > 985_000  <-- NOT enforced from here

If the front hop's read timeout falls below server.headersTimeout, it answers first with its own error and every diagnostic this proxy reports — the resolved window, the timeout source, which timer fired — is lost, and the OPENAI_PROXY_SOCKET_TIMEOUT path becomes unreachable. proxy.server.started logs the resolved values under timeouts, so the required floor can be read from a running instance rather than recomputed by hand.

If a reverse proxy (nginx, etc.) sits in front of this service, it must pass text/event-stream responses through unbuffered and its read timeout must be above the keep-alive interval.

SSE cap characterization probe

POST /openai2/diagnostics/sse-cap is an authenticated, non-billable diagnostic endpoint for identifying whether the deployed proxy chain has an idle/read cap, a total-response cap, or SSE-comment buffering. It accepts the normal security_key plus one of three closed modes: active-data, comment-heartbeat, or silent-control. Probe duration is bounded to three minutes and the endpoint never calls a model provider.

Run all three modes concurrently through the same deployed host used for /openai2:

SSE_CAP_PROBE_URL=https://proxy.example.invalid \
SSE_CAP_PROBE_SECURITY_KEY=... \
npm run probe:sse-cap

The command sends Accept-Encoding: identity, records bounded event-arrival timings and opaque request IDs, and prints one of idle-cap-confirmed, total-response-cap-suspected, comment-buffering-suspected, cap-not-reproduced, or inconclusive. It never prints the target URL or security key. Run it against the deployed ingress; running directly against localhost does not characterize intermediate hops.

Default values:

default upstream timeout: 600000 ms
maximum upstream timeout: 900000 ms
transport connect timeout: 30000 ms
transport headers timeout: OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS + 5000
transport body timeout: OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS + 5000
maximum parallel requests: 32
SSE keep-alive interval: 15000 ms

Running

Development (with hot reload):

npm run local:watch

Production:

npm run start

The server starts on http://localhost:3002 by default.

Vision Notes

Local validation confirmed image-capable requests through /openai and /openai2 when the image is sent inline (image.base64 for /openai, or a data: URL in input_image.image_url for /openai2).
Externally hosted image URLs are forwarded unchanged and may still be rejected by the upstream model provider. In local testing, a Wikimedia image URL failed upstream with invalid_image_url.
Anthropic vision requests require a valid ANTHROPIC_API_KEY or per-request anthropic_api_key. Without Anthropic auth, the request does not reach Claude.

API Endpoints

`GET /`

Returns a simple HTML page to verify the server is running.

`POST /openai`

Main endpoint for OpenAI Chat Completions API.

Proxies requests to POST https://api.openai.com/v1/chat/completions.

Request Body (JSON)

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key
`project`	string	❌	OpenAI project ID
`organization`	string	❌	OpenAI organization ID
`image`	object	❌	Image for vision models (see below)
`model`	string	✅	Model ID (e.g., `gpt-4o`, `gpt-4o-mini`)
`messages`	array	✅	Array of message objects
`temperature`	number	❌	Sampling temperature (0-2)
`top_p`	number	❌	Nucleus sampling (0-1)
`max_tokens`	number	❌	Max tokens to generate
`max_completion_tokens`	number	❌	Max completion tokens
...	...	❌	Any other Chat Completions API parameters

Note: stream: true is not supported on this endpoint.

Image Object

{
  "url": "https://example.com/image.png"
}

or

{
  "base64": "iVBORw0KGgoAAAANSUhEUg..."
}

Example Request

curl -X POST http://localhost:3002/openai \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is TypeScript?"}
    ],
    "temperature": 0.7,
    "max_tokens": 500
  }'

Example with Image (Vision)

curl -X POST http://localhost:3002/openai \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "What is in this image?"}
    ],
    "image": {
      "base64": "iVBORw0KGgoAAAANSUhEUg..."
    }
  }'

Use inline base64 image data when you need the most reliable path through /openai. Remote image URLs are passed through unchanged and can still fail upstream.

Response

Standard OpenAI Chat Completion response:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "TypeScript is..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 100,
    "total_tokens": 125
  }
}

`POST /v1/chat/completions`

OpenAI-compatible Chat Completions endpoint — a drop-in target for standard OpenAI SDKs and Spring AI (spring-ai-openai). Unlike /openai, the authentication parameters are passed via HTTP headers (not in the body), so a client only needs to override its base URL and add two headers — no custom adapter code.

Proxies requests to POST https://api.openai.com/v1/chat/completions. The request body is forwarded to OpenAI as-is, and the OpenAI response is returned unchanged (same status code, same body).

Headers

Header	Required	Description
`Authorization: Bearer <openai_api_key>`	✅	OpenAI API key
`X-Security-Key: <security_key>`	✅	Must match the `SECURITY_KEY` env variable
`X-Project: <project>`	❌	OpenAI project ID (forwarded as `OpenAI-Project`)

Request Body (JSON)

Standard OpenAI Chat Completions body (model, messages, response_format, temperature, etc.). The body must not contain security_key / openai_api_key. response_format: { "type": "json_schema", ... } (structured output) is forwarded and works as usual.

Note: stream: true is not supported on this endpoint and returns 400 { "error": { "message": "Streaming is not supported on this endpoint" } }.

Errors

Errors are returned as OpenAI-compatible error objects ({ "error": { "message", "type", "code" } }), with the proxy's additional requestId and timeout diagnostic fields described under Error Responses:

Missing/invalid X-Security-Key → 401
Missing/malformed Authorization → 401
Upstream OpenAI errors → the upstream status code is forwarded
Upstream timeout → 504; network/transport failure → 502

Example Request

curl -X POST http://localhost:3002/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "X-Security-Key: your-secret-key" \
  -H "X-Project: proj_123" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "What is TypeScript?"}
    ]
  }'

Spring AI Configuration

Point the Spring AI OpenAI client at the proxy via base-url and add the proxy headers:

spring:
  ai:
    openai:
      base-url: https://<proxy-host>
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o

OpenAiChatOptions.builder()
    .httpHeaders(Map.of("X-Security-Key", key, "X-Project", project))
    .build();

Spring AI appends /v1/chat/completions to the configured base-url, so set base-url to the proxy origin (e.g. https://proxy.example.com) without a trailing path.

OpenAI SDK Configuration

Any standard OpenAI SDK works too — point base_url / baseURL at the proxy's /v1 and add the X-Security-Key header (streaming excluded).

Python (openai):

from openai import OpenAI

client = OpenAI(
    base_url="https://<proxy-host>/v1",
    api_key="<openai_api_key>",  # sent as Authorization: Bearer ...
    default_headers={"X-Security-Key": "your-secret-key", "X-Project": "proj_123"},
)

completion = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is TypeScript?"}],
)

Node (openai):

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<proxy-host>/v1",
  apiKey: "<openai_api_key>", // sent as Authorization: Bearer ...
  defaultHeaders: { "X-Security-Key": "your-secret-key", "X-Project": "proj_123" },
});

const completion = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "What is TypeScript?" }],
});

OpenAI SDKs append /chat/completions to base_url, so include the /v1 segment (e.g. https://proxy.example.com/v1). This lands on the same /v1/chat/completions route as the Spring AI configuration above.

`POST /openai2`

OpenAI Responses API endpoint — The most advanced interface for generating model responses with support for tools, files, web searching, MCP, function calling, and more.

Proxies requests to POST https://api.openai.com/v1/responses.

The proxy also exposes the other documented Responses API operations:

POST /openai2/compact → POST /v1/responses/compact
POST /openai2/input_tokens → POST /v1/responses/input_tokens
GET /openai2/:response_id → GET /v1/responses/:response_id
GET /openai2/:response_id/input_items → GET /v1/responses/:response_id/input_items
POST /openai2/:response_id/cancel → POST /v1/responses/:response_id/cancel
DELETE /openai2/:response_id → DELETE /v1/responses/:response_id

Request Body (JSON)

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key
`project`	string	❌	OpenAI project ID
`organization`	string	❌	OpenAI organization ID
`model`	string	✅	Model ID (e.g., `gpt-4o`, `gpt-4.1`, `o3`)
`input`	string/array	✅	Text, image, or file inputs to the model
`instructions`	string	❌	System/developer message inserted into context
`tools`	array	❌	Array of tools (web_search, file_search, function, mcp, etc.)
`tool_choice`	string/object	❌	How model should select tools (`auto`, `none`, `required`, or specific tool)
`stream`	boolean	❌	Enable Server-Sent Events streaming (default: `false`)
`temperature`	number	❌	Sampling temperature (0-2, default: 1)
`top_p`	number	❌	Nucleus sampling (0-1, default: 1)
`max_output_tokens`	integer	❌	Max tokens for response including reasoning
`max_tool_calls`	integer	❌	Max total calls to built-in tools
`parallel_tool_calls`	boolean	❌	Allow parallel tool calls (default: `true`)
`previous_response_id`	string	❌	ID of previous response for multi-turn conversations
`conversation`	string/object	❌	Conversation context (cannot use with `previous_response_id`)
`store`	boolean	❌	Store response for later retrieval (default: `true`)
`metadata`	object	❌	Up to 16 key-value pairs for additional info
`include`	array	❌	Additional output data to include (see below)
`text`	object	❌	Text response configuration (format, structured output)
`reasoning`	object	❌	Reasoning model configuration (effort, summary)
`truncation`	string	❌	Truncation strategy (`auto` or `disabled`)
`background`	boolean	❌	Run response in background (default: `false`)
`service_tier`	string	❌	Processing tier (`auto`, `default`, `flex`, `priority`)
`timeout`	number	❌	Proxy-specific OpenAI SDK request timeout in milliseconds for this single upstream call

Timeout Override Semantics

timeout is expressed in milliseconds.
For POST endpoints, pass timeout as a JSON number when possible. Numeric strings are also normalized safely.
For GET/DELETE Responses endpoints, pass timeout as a query parameter.
The proxy applies this value to the single upstream OpenAI SDK request for that operation. It does not carry over to later retrieve, list, cancel, or delete calls.
If no timeout is provided, the proxy uses OPENAI_PROXY_UPSTREAM_TIMEOUT_MS.
Missing, invalid, non-finite, or non-positive values fall back to OPENAI_PROXY_UPSTREAM_TIMEOUT_MS.
Values above OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS are clamped before the upstream SDK call is made.
If OPENAI_PROXY_UPSTREAM_MAX_TIMEOUT_MS is configured below OPENAI_PROXY_UPSTREAM_TIMEOUT_MS, the effective max becomes the default timeout.
The effective timeout is logged in the proxy's structured logs.

The timeout policy is applied across the OpenAI-backed proxy routes, including /openai, /openai2, /openai/audio/transcriptions, and /embeddings.

Error Responses

OpenAI-facing routes now return structured JSON errors instead of generic plain-text 500 responses:

{
  "error": {
    "message": "Timeout while waiting for upstream response",
    "type": "upstream_timeout",
    "code": "OPENAI_PROXY_TIMEOUT",
    "requestId": "a7a27871-9d49-40c0-8c7b-7d44d2770ce8",
    "incomingRequestId": "edge-request-id-from-x-request-id",
    "timeoutOrigin": "undici_headers_timeout",
    "effectiveTimeoutMs": 900000,
    "timeoutSource": "clamped",
    "requestedTimeoutMs": 1200000
  }
}

Correlation fields:

requestId is the proxy-local request UUID generated inside chatgpt-proxy.
incomingRequestId is the incoming x-request-id preserved from the caller or outer reverse proxy when present.
Error responses also include X-Proxy-Request-Id and, when available, X-Incoming-Request-Id response headers.

Timeout fields:

timeoutOrigin names the timer that actually fired — openai_sdk_timeout, anthropic_sdk_timeout, undici_connect_timeout, undici_headers_timeout, undici_body_timeout, proxy_socket_timeout, or unknown_timeout. Omitted when the failure was not a timeout.
effectiveTimeoutMs, timeoutSource, and requestedTimeoutMs mirror the response headers below. They appear once an upstream budget has been applied to the request, so a validation or overload failure raised before that carries none of them rather than reporting a window it never used.

Resolved Timeout Headers

Every response that reached the point of resolving an upstream timeout — successful ones included — carries the resolved facts as headers, so a caller can size its own budget from the window the proxy actually applied instead of inferring it:

Header	Value
`x-openai-proxy-upstream-timeout-ms`	The per-attempt window applied to the upstream call
`x-openai-proxy-timeout-source`	`default`, `provided`, `invalid`, or `clamped`
`x-openai-proxy-requested-timeout-ms`	What the caller asked for; omitted when no `timeout` was supplied
`x-openai-proxy-fetch-timeout-ms`	The undici response-headers timeout (`OPENAI_PROXY_TRANSPORT_HEADERS_TIMEOUT_MS`)
`x-openai-proxy-timeout-origin`	Which timer fired; present on timeout failures only

The *-ms headers carry bare integers with no units or suffixes, so Number() parses them directly. x-openai-proxy-timeout-origin is emitted as a header as well as error.timeoutOrigin because a caller's HTTP-error path can read headers without parsing the error body. All are listed in Access-Control-Expose-Headers, so browser callers can read them cross-origin.

Requests rejected before a timeout is resolved — auth failures, malformed bodies, 503 overload — carry no timeout headers, since no window was applied.

Failure categories:

OpenAI API errors with an upstream HTTP status preserve that status and include sanitized upstream metadata.
Transport timeouts without a valid upstream response return 504.
Transport failures such as DNS, TLS, socket reset, or other connection failures return 502.
Local overload from the concurrency guard returns 503 with Retry-After: 1.
Validation failures return 400.
Proxy auth failures remain 403.
Client disconnects abort upstream work and are logged as cancellations instead of generic server failures.

Retry Policy

Automatic retries are disabled for non-idempotent create-style calls such as /openai, /openai2, /openai2/compact, /openai/audio/transcriptions, and /embeddings to avoid duplicating billed work.
The official OpenAI SDK retry mechanism is still used on the safer read-only or idempotent operations exposed by the proxy:
- POST /openai2/input_tokens
- GET /openai2/:response_id
- GET /openai2/:response_id/input_items
- DELETE /openai2/:response_id
Retry attempts are logged with request ID, endpoint, attempt number, and sanitized failure details.

Structured Logs

Each OpenAI-backed request emits a structured completion log entry with:

request ID
incoming request ID when provided on x-request-id
OpenAI request ID when OpenAI returns x-request-id
endpoint and method
model when present
streaming flag
effective timeout and timeout source
timeout origin for timeout-like failures: openai_sdk_timeout, anthropic_sdk_timeout, undici_connect_timeout, undici_headers_timeout, undici_body_timeout, or unknown_timeout
top-level error name, code, and message for failed requests; timeout and transport failures use provider-agnostic messages (Timeout while waiting for upstream response, Transport failure while contacting upstream API)
sanitized error cause chain for failed requests when present
start time and duration
final result category and returned HTTP status
retry count
overload and cancellation flags

Secrets such as API keys, bearer tokens, proxy security keys, cookies, and access tokens are redacted before they are stored or printed.

Runtime Timeout Diagnostics

Operators can inspect the live timeout and correlation configuration without reading source:

curl "http://localhost:3002/debug/runtime?access_token=YOUR_ACCESS_TOKEN"

The guarded runtime snapshot includes:

default and maximum upstream timeout
explicit undici connect, headers, and body timeout values
Node HTTP server request, socket, keep-alive, and headers timeouts
maximum parallel request limit
request ID preservation behavior and response field names
the effective retry policy for each OpenAI-backed proxy route

Include Options

Specify additional output data to include:

web_search_call.action.sources — Include web search sources
code_interpreter_call.outputs — Include code interpreter outputs
file_search_call.results — Include file search results
message.input_image.image_url — Include input image URLs
message.output_text.logprobs — Include logprobs with messages
reasoning.encrypted_content — Include encrypted reasoning tokens

Tools Configuration

Web Search Tool:

{
  "type": "web_search_preview",
  "search_context_size": "medium"
}

File Search Tool:

{
  "type": "file_search",
  "vector_store_ids": ["vs_abc123"],
  "max_num_results": 20
}

Function Calling Tool:

{
  "type": "function",
  "name": "get_weather",
  "description": "Get current weather for a location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": { "type": "string", "description": "City name" }
    },
    "required": ["location"]
  }
}

MCP (Model Context Protocol) Tool:

{
  "type": "mcp",
  "server_label": "my-mcp-server",
  "server_url": "https://my-mcp-server.example.com",
  "allowed_tools": ["tool1", "tool2"]
}

Code Interpreter Tool:

{
  "type": "code_interpreter",
  "container": { "type": "auto" }
}

Example: Simple Text Request

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "Tell me a three sentence bedtime story about a unicorn."
  }'

Example: With System Instructions

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "instructions": "You are a helpful coding assistant. Always provide code examples.",
    "input": "How do I read a file in Python?"
  }'

Example: Web Search

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "What are the latest news about AI?",
    "tools": [
      { "type": "web_search_preview" }
    ],
    "include": ["web_search_call.action.sources"]
  }'

Example: File Search with Vector Store

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "What does the documentation say about authentication?",
    "tools": [
      {
        "type": "file_search",
        "vector_store_ids": ["vs_abc123"],
        "max_num_results": 10
      }
    ],
    "include": ["file_search_call.results"]
  }'

Example: Function Calling

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "What is the weather in San Francisco?",
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": { "type": "string", "description": "City name" },
            "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
          },
          "required": ["location"]
        }
      }
    ]
  }'

Example: MCP Server Integration

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "Search my Google Drive for Q4 reports",
    "tools": [
      {
        "type": "mcp",
        "server_label": "google-drive",
        "server_url": "https://mcp.example.com/google-drive",
        "allowed_tools": ["search_files", "read_file"]
      }
    ]
  }'

Example: Image Input

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": [
      { "type": "input_text", "text": "What is in this image?" },
      { "type": "input_image", "image_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg..." }
    ]
  }'

Local validation confirmed /openai2 with inline data: image URLs.

Example: Multi-turn Conversation

# First request
curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "My name is Alice."
  }'

# Response includes "id": "resp_abc123..."

# Second request with previous_response_id
curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "What is my name?",
    "previous_response_id": "resp_abc123..."
  }'

Example: Streaming Response

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "Write a short poem about coding.",
    "stream": true
  }'

Retrieve Query Parameters

GET /openai2/:response_id supports the documented Responses retrieval query parameters:

include
stream
include_obfuscation
starting_after
timeout (milliseconds, per upstream retrieve request, practical max 900000)

Input Items Query Parameters

GET /openai2/:response_id/input_items supports:

after
include
limit
order
timeout (milliseconds, per upstream list request, practical max 900000)

Example: Compact a Conversation

curl -X POST http://localhost:3002/openai2/compact \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-5",
    "input": "Summarize this long-running conversation."
  }'

Example: Count Input Tokens

curl -X POST http://localhost:3002/openai2/input_tokens \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "Count the tokens in this prompt."
  }'

Example: List Input Items

curl "http://localhost:3002/openai2/resp_abc123/input_items?security_key=your-secret-key&limit=20&order=desc"

Example: Structured Output (JSON Schema)

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "Extract the name and age from: John is 30 years old.",
    "text": {
      "format": {
        "type": "json_schema",
        "name": "person_info",
        "schema": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "age": { "type": "integer" }
          },
          "required": ["name", "age"]
        }
      }
    }
  }'

Example: Reasoning Model Configuration

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "o3",
    "input": "Solve this complex math problem: ...",
    "reasoning": {
      "effort": "high",
      "summary": "auto"
    }
  }'

Example: Combined Tools (Web Search + Function Calling)

curl -X POST http://localhost:3002/openai2 \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "gpt-4o",
    "input": "Find the current stock price of Apple and calculate a 10% increase",
    "tools": [
      { "type": "web_search_preview" },
      {
        "type": "function",
        "name": "calculate_percentage",
        "description": "Calculate percentage of a number",
        "parameters": {
          "type": "object",
          "properties": {
            "number": { "type": "number" },
            "percentage": { "type": "number" }
          },
          "required": ["number", "percentage"]
        }
      }
    ],
    "parallel_tool_calls": true
  }'

Response

Standard OpenAI Responses API response:

{
  "id": "resp_67ccd2bed1ec8190b14f964abc054267...",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "model": "gpt-4o-2024-08-06",
  "output": [
    {
      "type": "message",
      "id": "msg_67ccd2bf17f0819081ff3bb2cf6508e6...",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "In a peaceful grove beneath a silver moon...",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "reasoning": { "effort": null, "summary": null },
  "store": true,
  "temperature": 1.0,
  "tool_choice": "auto",
  "tools": [],
  "usage": {
    "input_tokens": 36,
    "input_tokens_details": { "cached_tokens": 0 },
    "output_tokens": 87,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 123
  }
}

`GET /openai2/:response_id`

Retrieve a stored response by ID.

Query Parameters

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key
`project`	string	❌	OpenAI project ID
`organization`	string	❌	OpenAI organization ID

Example Request

curl "http://localhost:3002/openai2/resp_abc123?security_key=your-secret-key"

`DELETE /openai2/:response_id`

Delete a stored response.

Query Parameters

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key

Example Request

curl -X DELETE "http://localhost:3002/openai2/resp_abc123?security_key=your-secret-key"

Response

{
  "id": "resp_abc123",
  "object": "response",
  "deleted": true
}

`POST /openai2/:response_id/cancel`

Cancel a background response (only for responses created with background: true).

Request Body (JSON)

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key

Example Request

curl -X POST http://localhost:3002/openai2/resp_abc123/cancel \
  -H "Content-Type: application/json" \
  -d '{ "security_key": "your-secret-key" }'

`GET /openai2/:response_id/input_items`

List input items for a response.

Query Parameters

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key

Example Request

curl "http://localhost:3002/openai2/resp_abc123/input_items?security_key=your-secret-key"

Response

{
  "object": "list",
  "data": [
    {
      "id": "msg_abc123",
      "type": "message",
      "role": "user",
      "content": [
        { "type": "input_text", "text": "Tell me a story." }
      ]
    }
  ],
  "first_id": "msg_abc123",
  "last_id": "msg_abc123",
  "has_more": false
}

`POST /chatgpt`

Simplified chat endpoint using the chatgpt library.

Request Body (JSON)

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`prompt`	string	✅	The user message
`model`	string	❌	Model ID (default: `gpt-4o-mini`)
`temperature`	number	❌	Sampling temperature (0-2)
`top_p`	number	❌	Nucleus sampling (0-1)
`max_tokens`	number	❌	Max tokens to generate
`max_completion_tokens`	number	❌	Max completion tokens

Example Request

curl -X POST http://localhost:3002/chatgpt \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "prompt": "Explain quantum computing in simple terms",
    "model": "gpt-4o-mini",
    "temperature": 0.8
  }'

`POST /openai/audio/transcriptions`

Audio transcription using OpenAI Whisper.

Request (multipart/form-data)

Field	Type	Required	Description
`file`	file	✅	Audio file (wav, mp3, m4a, etc.)
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`openai_api_key`	string	❌	Override the default API key
`project`	string	❌	OpenAI project ID
`organization`	string	❌	OpenAI organization ID
`model`	string	❌	Model ID (default: `whisper-1`)
`language`	string	❌	Language code (e.g., `en`, `es`)
`prompt`	string	❌	Optional prompt to guide transcription
`temperature`	number	❌	Sampling temperature (default: 0)
`response_format`	string	❌	`json`, `text`, `srt`, `verbose_json`, `vtt`
`timestamp_granularities[]`	string	❌	`word` and/or `segment`

Example Request

curl -X POST http://localhost:3002/openai/audio/transcriptions \
  -F "file=@audio.mp3" \
  -F "security_key=your-secret-key" \
  -F "model=whisper-1" \
  -F "language=en" \
  -F "response_format=json"

Response

{
  "text": "Hello, this is a transcription of the audio file."
}

`POST /embeddings`

Generate text embeddings.

Request Body (JSON)

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`input`	string or array	✅	Text(s) to embed
`model`	string	❌	Model ID (default: `text-embedding-3-large`)
`dimensions`	number	❌	Output dimensions
`encoding_format`	string	❌	`float` or `base64`

Example Request

curl -X POST http://localhost:3002/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "input": "The quick brown fox jumps over the lazy dog",
    "model": "text-embedding-3-small",
    "dimensions": 512
  }'

Response

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0023064255, -0.009327292, ...]
    }
  ],
  "model": "text-embedding-3-small",
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 9
  }
}

Anthropic (Claude) Endpoints

The proxy also supports Anthropic Claude models via the @anthropic-ai/sdk package. These endpoints are completely separate from the OpenAI routes.

Configuration

Add the Anthropic API key to your .env or .env.local file:

ANTHROPIC_API_KEY=sk-ant-...

If you do not configure Anthropic auth, /anthropic requests fail before they reach Claude.

`POST /anthropic`

Anthropic Messages API endpoint — Create a message using Claude models (non-streaming).

Request Body (JSON)

Field	Type	Required	Description
`security_key`	string	✅	Must match `SECURITY_KEY` env variable
`anthropic_api_key`	string	❌	Override the default Anthropic API key
`model`	string	✅	Model ID (e.g., `claude-sonnet-4-5-20250514`, `claude-opus-4-5-20250514`, `claude-haiku-4-5-20250514`)
`max_tokens`	number	✅	Maximum number of tokens to generate
`messages`	array	✅	Array of message objects (`role`: `user` or `assistant`)
`system`	string/array	❌	System prompt
`temperature`	number	❌	Sampling temperature (0-1)
`top_p`	number	❌	Nucleus sampling (0-1)
`top_k`	number	❌	Top-K sampling
`stop_sequences`	array	❌	Custom stop sequences
`tools`	array	❌	Tool definitions for function calling
`tool_choice`	object	❌	How model should use tools (`auto`, `any`, `tool`)
`output_config`	object	❌	Structured output configuration with JSON schema
`metadata`	object	❌	Request metadata (e.g., `user_id`)
`thinking`	object	❌	Extended thinking configuration
`timeout`	number	❌	Proxy-specific request timeout in milliseconds

Note: stream: true is not supported on this endpoint. Use /anthropic/stream instead.

Example Request

curl -X POST http://localhost:3002/anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Explain quantum computing in simple terms."}
    ]
  }'

Example with System Prompt

curl -X POST http://localhost:3002/anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 1024,
    "system": "You are a helpful coding assistant. Always provide code examples.",
    "messages": [
      {"role": "user", "content": "How do I read a file in Python?"}
    ]
  }'

Example with Vision (Image)

curl -X POST http://localhost:3002/anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 1024,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "image",
            "source": {
              "type": "base64",
              "media_type": "image/png",
              "data": "iVBORw0KGgoAAAANSUhEUg..."
            }
          },
          { "type": "text", "text": "What is in this image?" }
        ]
      }
    ]
  }'

Example with Tool Use

curl -X POST http://localhost:3002/anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 1024,
    "tools": [
      {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "input_schema": {
          "type": "object",
          "properties": {
            "location": { "type": "string", "description": "City name" }
          },
          "required": ["location"]
        }
      }
    ],
    "messages": [
      {"role": "user", "content": "What is the weather in San Francisco?"}
    ]
  }'

Example with Structured Output (JSON Schema)

The proxy fully supports Anthropic's output_config for structured output. Pass output_config with a format object containing the JSON schema to get type-safe, validated responses:

curl -X POST http://localhost:3002/anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 1024,
    "output_config": {
      "format": {
        "type": "json_schema",
        "schema": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "age": { "type": "integer" },
            "skills": {
              "type": "array",
              "items": { "type": "string" }
            }
          },
          "required": ["name", "age", "skills"]
        }
      }
    },
    "messages": [
      {"role": "user", "content": "Extract info: John is 30 years old and knows Python, TypeScript, and Rust."}
    ]
  }'

The response content will contain a JSON object matching the schema:

{
  "id": "msg_01...",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "{\"name\": \"John\", \"age\": 30, \"skills\": [\"Python\", \"TypeScript\", \"Rust\"]}"
    }
  ],
  "stop_reason": "end_turn",
  "usage": { "input_tokens": 40, "output_tokens": 35 }
}

Example with Extended Thinking

curl -X POST http://localhost:3002/anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 16000,
    "thinking": {
      "type": "enabled",
      "budget_tokens": 10000
    },
    "messages": [
      {"role": "user", "content": "Solve this complex math problem: ..."}
    ]
  }'

Response

Standard Anthropic Messages API response:

{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Quantum computing is..."
    }
  ],
  "model": "claude-sonnet-4-5-20250514",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 25,
    "output_tokens": 150
  }
}

`POST /anthropic/stream`

Streaming Anthropic Messages API endpoint — Create a message with Server-Sent Events streaming.

Takes the same request body as POST /anthropic (the stream field is ignored since this endpoint always streams).

Example Request

curl -X POST http://localhost:3002/anthropic/stream \
  -H "Content-Type: application/json" \
  -d '{
    "security_key": "your-secret-key",
    "model": "claude-sonnet-4-5-20250514",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Write a short poem about coding."}
    ]
  }'

Streaming Events

The stream emits Anthropic SSE events:

message_start — Contains the initial Message object with metadata
content_block_start — Start of a content block (text, tool_use, thinking)
content_block_delta — Incremental content (text_delta, input_json_delta, thinking_delta)
content_block_stop — End of a content block
message_delta — Final message metadata (stop_reason, usage)
message_stop — End of the message

Error Responses

Status	Description
`400`	Bad Request — Invalid input or streaming not supported
`403`	Forbidden — Invalid or missing `security_key`
`404`	Not Found — Unknown endpoint
`429`	Too Many Requests — Rate limit exceeded (health/log endpoints)
`500`	Internal Server Error — OpenAI API error or server issue

Docker

Build and push Docker image:

npm run docker

Or manually:

docker build -t chatgpt-proxy .
docker run -p 3002:3002 --env-file .env chatgpt-proxy

Server Configuration

Port: 3002
Request Timeout: 15 minutes (900,000 ms)
Keep-Alive Timeout: 15 minutes
Headers Timeout: ~16 minutes
OpenAI Transport Connect Timeout: 30 seconds by default
OpenAI Transport Headers Timeout: max upstream timeout + 5 seconds by default
OpenAI Transport Body Timeout: max upstream timeout + 5 seconds by default

Client timeout overrides cannot increase these server-side HTTP limits.

License

ISC

Name		Name	Last commit message	Last commit date
Latest commit History 60 Commits
build		build
src		src
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
eslint.config.js		eslint.config.js
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json

Folders and files

Latest commit

History

Repository files navigation

chatgpt-proxy

Features

Installation

Configuration

Reliability Controls

A hop in front of this proxy must be more patient than it is

SSE cap characterization probe

Running

Vision Notes

API Endpoints

GET /

POST /openai

Request Body (JSON)

Image Object

Example Request

Example with Image (Vision)

Response

POST /v1/chat/completions

Headers

Request Body (JSON)

Errors

Example Request

Spring AI Configuration

OpenAI SDK Configuration

POST /openai2

Request Body (JSON)

Timeout Override Semantics

Error Responses

Resolved Timeout Headers

Retry Policy

Structured Logs

Runtime Timeout Diagnostics

Include Options

Tools Configuration

Example: Simple Text Request

Example: With System Instructions

Example: Web Search

Example: File Search with Vector Store

Example: Function Calling

Example: MCP Server Integration

Example: Image Input

Example: Multi-turn Conversation

Example: Streaming Response

Retrieve Query Parameters

Input Items Query Parameters

Example: Compact a Conversation

Example: Count Input Tokens

Example: List Input Items

Example: Structured Output (JSON Schema)

Example: Reasoning Model Configuration

Example: Combined Tools (Web Search + Function Calling)

Response

GET /openai2/:response_id

Query Parameters

Example Request

DELETE /openai2/:response_id

Query Parameters

Example Request

Response

POST /openai2/:response_id/cancel

Request Body (JSON)

Example Request

GET /openai2/:response_id/input_items

Query Parameters

Example Request

Response

POST /chatgpt

Request Body (JSON)

Example Request

POST /openai/audio/transcriptions

Request (multipart/form-data)

Example Request

Response

POST /embeddings

Request Body (JSON)

Example Request

`GET /`

`POST /openai`

`POST /v1/chat/completions`

`POST /openai2`

`GET /openai2/:response_id`

`DELETE /openai2/:response_id`

`POST /openai2/:response_id/cancel`

`GET /openai2/:response_id/input_items`

`POST /chatgpt`

`POST /openai/audio/transcriptions`

`POST /embeddings`

`POST /anthropic`

`POST /anthropic/stream`