Kotlin SandboxPool ignores Retry-After on rate-limited warmup creates

[jianpingpei]

Summary

The Kotlin SDK preserves HTTP 429 information as SandboxRateLimitException, including the parsed Retry-After duration, but SandboxPool drops that semantic information when a warmup create fails.

As a result, rate limiting is accounted for as an ordinary warmup failure and future warmup submissions are paced only by the pool's generic degraded backoff, not by the server-provided retry window.

Current behavior

The adapter layer correctly creates a typed exception:

SandboxRateLimitException(
    statusCode = 429,
    retryAfter = parseRetryAfter(header("Retry-After")),
    ...
)

However, the rolling warmup path converts every throwable to the same outcome:

WarmupOutcome.Failure(failure)

and later handles every failure identically:

reconcileState.recordAsyncFailure(outcome.error.message)

There is no SandboxRateLimitException handling in either the pool or infrastructure.pool package. The exception type and retryAfter value are therefore lost at the layer that schedules subsequent create attempts.

The recent completion-reconcile rate bound prevents a failed warmup from immediately driving an unbounded completion loop, and the existing degraded state eventually applies generic exponential backoff after degradedThreshold. Those safeguards help, but they do not honor the server's explicit retry window:

• already available warmup slots may continue submitting while 429 failures are below the threshold;
• a configured periodic reconcile interval may retry before Retry-After expires;
• generic degraded backoff may be shorter or longer than the requested wait;
• 429 responses contribute to ordinary failure/degraded accounting even though they indicate capacity throttling rather than an unhealthy pool implementation.

In a high-concurrency deployment this produced a sustained large volume of repeated 429 create attempts during one rate-limit window.

Expected behavior

When a warmup create fails with SandboxRateLimitException:

1. Preserve and use retryAfter to establish a pool-level throttleUntil deadline.
2. Do not submit new warmups for that pool before the deadline.
3. If the header is missing or invalid, use a bounded default delay (for example 5-30 seconds).
4. Do not count rate-limit responses as ordinary degraded failures, or track them separately.
5. When multiple in-flight warmups receive 429, extend the deadline monotonically (for example, keep the latest throttleUntil) rather than shortening it.

The implementation should reuse the SDK's existing Retry-After parsing/capping semantics where possible. If pool leadership can move between processes, consider whether the throttle deadline needs to live in shared pool state rather than only in local ReconcileState.

Suggested tests

• A warmup creator throws SandboxRateLimitException(retryAfter = 2s); no further warmup is submitted during those two seconds.
• Warmup resumes after the deadline.
• A 429 does not increment the ordinary degraded failure count.
• Missing Retry-After uses the documented fallback.
• Concurrent 429 completions cannot shorten an existing throttle deadline.
• Pool stop/restart does not leak an obsolete throttle into a new run.