Skip to content

P7: Implement Redis RPM, TPM, and concurrency primitives #23

@wauputr4

Description

@wauputr4

Goal

Implement Redis-backed counters and leases for requests per minute, tokens per minute, and concurrency limits.

Acceptance Criteria

  • Per-key RPM counter works with TTL.
  • Per-key TPM counter works with TTL.
  • Per-key, per-user, and per-provider concurrency leases are supported.
  • Crashed requests do not permanently hold leases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: limitsRate limits, token limits, and concurrency leasesenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions