Skip to content

Refactor gateway request log persistence to avoid blocking response path #49

@wauputr4

Description

@wauputr4

Context

Review comment on PR #48 flagged that request logging () is awaited inline in many branches of , which adds synchronous database latency to API responses.

Proposed changes

  • Move request-log writes to an async background path (e.g., spawn task or bounded worker queue).
  • Keep failure handling resilient: never block/impact user-facing success path if logging fails.
  • Preserve existing request metadata (request id, status, provider, route, latency, error_code).
  • Add retry or bounded queue behavior to avoid unbounded task growth.

Acceptance Criteria

  • Hot path no longer waits for DB INSERT before returning response for chat completions.
  • Existing request log rows remain written with same schema and meaningful status/error_code.
  • Logging failures are recorded to tracing logs without changing API contract.
  • Regression test proves request handler latency path is unaffected by log DB failure.

Notes

Related comments are in PR #48: high-priority suggestion around blocking DB writes in handler flow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: gatewayGateway routing and OpenAI-compatible API surfacearea: observabilityTracing, logs, metrics, and diagnosticsenhancementNew feature or requestrustPull requests that update rust code

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions