feat(grpc): wire tenant token rate limiting into gRPC router - #2016

Open

XinyueZhang369 wants to merge 8 commits into

mainfrom

xz/grpc-rate-limit-reserve-guard

Commits on Aug 6, 2026

feat(grpc): wire tenant rate limiting into chat/generate/completion/messages
XinyueZhang369
committed
5fc1c71
test(grpc): add end-to-end integration tests for tenant rate limiting
XinyueZhang369
committed
b2a1ad2
fix(grpc): correct batched-completion undercount, preemption leak, alias handover, and impossible-request Retry-After
XinyueZhang369
committed
7fab6d8
fix(grpc): canonicalize request body, close unsettled streaming EOF, and stop double-charging n>1 prompts
XinyueZhang369
committed
746b430
fix(grpc): require every n>1 choice's Complete before settling streaming usage
XinyueZhang369
committed
d30d534
fix(grpc): stop multiplying cached_tokens per n>1 choice
XinyueZhang369
committed
7dc0e06
refactor(grpc): address PR review nitpicks on the rate-limit reserve guard
XinyueZhang369
committed
e8226e1
fix(grpc): resolve clippy fallout from rebasing onto the ZMQ stop-string work
XinyueZhang369
committed
0aa17d4