[Automated] Draft docs (agentgateway): llm: automatically enable eviction for failover

[github-actions]

agentgateway/agentgateway#3462 — llm: automatically enable eviction for failover

Docs versions: standalone/main, kubernetes/main

What changes for users: AI backends and virtual models with more than one priority group now enable default eviction automatically. Server errors and connection failures can fail over without an explicit health.eviction block; health settings are still needed to classify responses such as 429 or tune eviction timing.

What the docs now say: The standalone Virtual models page now says default eviction is enabled for multi-priority failover and updates the health table. The Kubernetes Model failover shared page now adds 1.6.x-gated notes that default eviction applies without a health policy, while explicit policies remain useful for 429 handling and tuning.

Not verified: No cluster; the configuration steps are unrun.

How agentgateway-3462 was drafted, and what was not verified

• Code PR: llm: automatically enable eviction for failover agentgateway#3462
• Code issue: Virtual Provider doesn't failover on DNS resolution error agentgateway#2979
• Product: agentgateway (upstream) → agentgateway/website
• Docs version: standalone/main and kubernetes/main; documented in the development tree, which is where unreleased changes go.
• Category: scoped-edit / pr

Plan, and what changed it

• Planned: Update the code-path candidate pages for the behavior change to LLM failover eviction.
• Did: Updated the standalone virtual models failover section and the shared Kubernetes failover asset, and wrote a proposed release note.

Why a documentation change is necessary

The pull request changes the default for AI backends with multiple priority groups: the diff adds default health eviction in AIBackend::new, and the pull request body says failover was configurable but could not actually fail over without this change. The standalone virtual models page said readers must configure health.eviction, and the Kubernetes failover page said a health policy was required for failover. Those claims are now incomplete for main, because default eviction handles 5xx responses and connection failures.

What changed on disk

• assets/agw-docs/pages/agentgateway/llm/failover.md — Added 1.6.x-gated Kubernetes notes for automatic default eviction and when an explicit health policy is still needed.
• content/docs/standalone/main/documentation/llm/virtual-models.md — Updated the failover section, health table, and doc-test comments to describe default eviction.

How to verify this change

1. On agentgateway 1.6.x or later, follow the Kubernetes Model failover guide through the failover-demo backend and HTTPRoute, but skip the failover-demo-health policy.
2. Send the first request from the guide. The response should be HTTP 500 from the failing primary group, which triggers default eviction.
3. Send the same request again. The response should be HTTP 200 with $.model equal to gpt-4o-mini, showing that traffic moved to the fallback group without an explicit health policy.

Proposed release note

The work-list has no release-note kind in markers.kinds, so placement might require the release-note step to assign a section before publication.

What was not verified

The configuration was not applied to a cluster, so every command in this diff is unrun.

How the pages were chosen (triage report)

Draft a documentation change

agentgateway/agentgateway#3462 — llm: automatically enable eviction for failover

• Product: agentgateway (upstream) → agentgateway/website
• Docs version: standalone/main, kubernetes/main
• Code issue: Virtual Provider doesn't failover on DNS resolution error agentgateway#2979
• Summary (read from the body): Without this, a user can set failover but its impossible to actually failover.
• Not tested: no cluster was available, so every command in this diff is unrun
• No rule covers 3 touched file(s). The candidate pages below come from the files that DID match a rule, so they may scope this narrower than the change is. Read these before deciding the change is only about the pages listed:
  • crates/agentgateway/src/http/health.rs
  • crates/agentgateway/src/proxy/httpproxy.rs
  • crates/agentgateway/src/test_helpers/proxymock.rs

Draft branch: pr-tracker-draft-agentgateway-3462

[cloudflare-workers-and-pages]

Deploying agentproxy with    Cloudflare Pages

Latest commit:	a1d6035
Status:	✅  Deploy successful!
Preview URL:	https://90368b35.agentproxy.pages.dev
Branch Preview URL:	https://pr-tracker-draft-agentgatewa-sz12.agentproxy.pages.dev

View logs