Core-level cross-provider failover for in-flight agent turns

[millarch]

AI Disclosure

• This issue was written entirely by a human
• This issue was written with AI assistance and reviewed and edited by a human
• I have read the AI Policy and agree to its terms

AI Tool(s) Used

Letta Code

Human Verification

I have personally reviewed this request, verified that it addresses a real use case, and take responsibility for its contents.

Problem statement

Letta Code can have multiple authenticated frontier LLM providers available, but connecting more than one provider does not currently provide true high availability for an agent turn.

The existing autoSwapOnQuotaLimit setting helps with quota exhaustion. Other retryable failures — provider outage, timeout, transport/API error, or temporary capacity bottleneck — can still strand the turn on the active provider. A memory instruction telling the agent to switch models is insufficient because the active model may fail before the agent can execute any action. A watchdog that changes the model afterward also cannot recover the original inbound message.

This matters especially for always-on agents reached through messaging channels and for scheduled/headless jobs, where there may be no operator present to resend the request or switch models manually.

Proposed solution

Support an ordered, configurable fallback chain per agent, for example:

primary frontier model -> secondary frontier model -> optional tertiary model

When a retryable provider failure occurs, Letta core should:

1. Classify the failure (quota, timeout, outage, transport/API error, capacity).
2. Retry the same in-flight request on the next configured provider/model.
3. Preserve conversation state and tool-call safety/idempotency.
4. Temporarily route subsequent turns to the fallback.
5. Probe or wait for a configurable cooldown, then restore the primary.
6. Prevent retry loops with bounded attempts and backoff.
7. Emit observable logs/events and clearly indicate when fallback was used.

The behavior should be consistent across CLI, Desktop, messaging channels, cron/scheduled turns, headless execution, and both local and cloud agents where supported.

Alternatives considered

• Agent memory instructions: insufficient when the active model fails before it can act.
• External watchdog: can switch the model for future turns but cannot reliably replay the original failed interactive/channel request.
• Local mod or headless wrapper: useful for some workflows, but not a complete or durable solution across all execution surfaces.
• Direct patches to Letta internals: likely to be overwritten or broken by upgrades.

Additional context

Core-level in-flight failover would make multi-provider authentication a genuine availability feature rather than only a manual escape hatch. The design should avoid duplicate side effects from partially completed tool calls and expose fallback events for debugging and observability.

This is a resubmission of #4361 using the required template and disclosure fields.