feat: add configurable rate limit retry delays [vibe by AI]

[bentcc]

Summary
Add two new configuration options to allow users to tune rate limit retry behavior without modifying source code.

Motivation
The current hardcoded 60-second default for rate limit retries can feel too long for users who:

• Have multiple accounts and prefer faster failover
• Want to reduce wait time when API doesn't return a retry-after header
• Are okay with potentially hitting more 429 errors in exchange for faster retries

Conversely, some users may want longer delays to preserve prompt cache by staying on the same account longer.

Changes
New Configuration Options

Option	Type	Default	Description
default_retry_after_seconds	1-300	60	Default retry delay when API doesn't return a retry-after header
max_backoff_seconds	5-300	60	Maximum cap for exponential backoff delay

Example Configuration
{
"default_retry_after_seconds": 10,
"max_backoff_seconds": 30
}

This configuration reduces the default wait from 60s to 10s, and caps exponential backoff at 30s.

Backward Compatibility

• Fully backward compatible - existing configs work without changes
• Default values match previous hardcoded behavior (60s each)
• No breaking changes to existing functionality

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

This pull request adds configurable parameters for rate-limit retry behavior. The changes introduce three new configuration options to the schema: switch_on_first_rate_limit, default_retry_after_seconds, and max_backoff_seconds. Function signatures for retryAfterMsFromResponse and getRateLimitBackoff are updated to accept corresponding parameters (defaultRetryMs and maxBackoffMs), with internal call sites modified to pass values derived from the new configuration options.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately and concisely summarizes the main change: adding configurable rate limit retry delays to the codebase.
Description check	✅ Passed	The description is comprehensive and directly related to the changeset, providing motivation, implementation details, configuration options, and backward compatibility assurances.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Greptile Summary

Added two new configuration options (default_retry_after_seconds and max_backoff_seconds) to make rate limit retry behavior tunable without code changes.

• Introduced default_retry_after_seconds (1-300s, default 60s) to control the fallback retry delay when the API doesn't provide a retry-after header
• Introduced max_backoff_seconds (5-300s, default 60s) to cap exponential backoff delays
• Modified retryAfterMsFromResponse to accept a configurable default retry delay parameter
• Updated getRateLimitBackoff to accept a configurable maximum backoff parameter
• Properly converts configuration values from seconds to milliseconds at the call site
• Maintains full backward compatibility with existing configurations

Confidence Score: 5/5

• This PR is safe to merge with no issues found
• The implementation is clean, well-structured, and maintains backward compatibility. Configuration validation is properly implemented with reasonable min/max bounds. The conversion from seconds to milliseconds is handled correctly at the call site, and default parameters ensure existing behavior is preserved.
• No files require special attention

Important Files Changed

Filename	Overview
src/plugin/config/schema.ts	Added two new configuration fields with appropriate validation: default_retry_after_seconds (1-300) and max_backoff_seconds (5-300), both defaulting to 60
src/plugin.ts	Modified retryAfterMsFromResponse and getRateLimitBackoff to accept configurable delays, properly converting seconds to milliseconds at call site

Sequence Diagram

sequenceDiagram
    participant Client
    participant Plugin
    participant API
    participant Config
    
    Client->>Plugin: Make API request
    Plugin->>API: Forward request
    API-->>Plugin: 429 Rate Limit
    
    Plugin->>Config: Get default_retry_after_seconds
    Config-->>Plugin: 60s (or custom value)
    
    Plugin->>Plugin: retryAfterMsFromResponse(response, defaultRetryMs)
    Note over Plugin: Check retry-after-ms header<br/>Check retry-after header<br/>Fallback to defaultRetryMs
    
    Plugin->>Config: Get max_backoff_seconds
    Config-->>Plugin: 60s (or custom value)
    
    Plugin->>Plugin: getRateLimitBackoff(accountIndex, quotaKey, serverRetryMs, maxBackoffMs)
    Note over Plugin: Calculate exponential backoff<br/>Cap delay with maxBackoffMs<br/>Track consecutive 429s
    
    Plugin->>Plugin: Wait delayMs
    Plugin->>API: Retry request

Loading

[greptile-apps]

Greptile found no issues!

From now on, if a review finishes and we haven't found any issues, we will not post anything, but you can confirm that we reviewed your changes in the status check section.

_{This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR".}

[tctinh]

Change merge target to dev, please fix conflict