[v5] RFC: backend capacity, model inventory, and placement

[hanthor]

Retitled and rewritten. The first draft led with an implementation and a
cute codename. Both were wrong. What matters is the four capabilities below —
and the one that makes the rest age well is that nothing about models can be
hardcoded. Models ship faster than any table in any repo survives.

Summary

Hive's governor decides when to kick an agent. Nothing decides which
backend and model it should be pointed at, or whether that backend can
serve right now. So a hive with a dry DeepSeek balance, a capped Codex
account, or a lapsed Claude session keeps kicking agents into backends that
cannot answer. They fail quietly and look healthy.

We have run an external component covering this for ~6 weeks. This RFC is not a
request to merge it — it is 1400 lines of bash against hive's HTTP API. It is a
report on the four capabilities the problem actually decomposes into, and
evidence for each.

The four capabilities

1. Usage probe — normalized headroom per backend

"How much is left" has a different answer shape per provider, and none is a
token count:

Backend	Where headroom lives	Shape
claude	GET /api/oauth/usage with the OAuth token	percent + resets_at, three limits
codex	/status typed into a live TUI	"NN% left, resets HH:MM on DD Mon"
agy	agy --print /usage	two pools × weekly/5-hour, percent remaining
deepseek	GET /user/balance	dollars of prepaid credit

A model-scoped cap is not provider exhaustion. Anthropic returns session,
weekly_all, and weekly_scoped — the last carries scope.model and caps one
model class. Taking the max across all three parked our entire fleet while
claude-sonnet-5 and claude-opus-5 both answered normally. Headroom must
come from unscoped limits, with scoped caps reported separately so
placement can avoid that model rather than the whole provider.

Probes that read a provider through a consumer of it will deadlock. This
bit us four separate times: google required an unpaused agent though its probe
is headless; openai skipped paused agents though a paused agent's CLI is still
running; contributors parked every worker and then could not measure the
provider needed to restore them; an emptied credential file reported "unknown"
instead of "exhausted". If hive builds this, that is the invariant to design
against.

2. Model inventory — what a backend can run right now

This is the capability we have least of, and the one that decides whether the
rest ages well. Today we hardcode model IDs in a table. That table is wrong the
week a model ships.

The backends are wildly inconsistent, which is exactly why it belongs behind
one interface:

Backend	Enumerable?	How
claude	yes, structured	GET /v1/models with the OAuth token — returns id + display_name
deepseek	yes, structured	OpenAI-compatible GET /models
agy	prose only	agy --print /models renders a human table, not a list
codex	no	--model takes a string; nothing enumerates

Verified live on our fleet — Anthropic returns claude-fable-5-1,
claude-opus-5, claude-sonnet-5, claude-opus-4-8, … and DeepSeek returns
deepseek-v4-flash, deepseek-v4-pro, deepseek-v4-flash-vision-exp.

Two of four can answer precisely. Hive is better placed than any operator to
normalize that and to push the other two upstream.

A related, cheap win: validate a model id against the inventory before
setting it. A model that a backend cannot run is a hard startup failure that
currently surfaces as a dead pane.

3. Tiering — ranked by benchmark, refreshed, never hardcoded

Placement needs to know which models are interchangeable in competence, so an
agent can move sideways rather than being silently downgraded. That ranking
cannot live in a repo:

• Our table has 14 hardcoded rungs. Every one is a guess with a shelf life.
• Our refresher (pull rankings from a benchmark API, rebuild the table) has
  never successfully run — it needs an API key nobody set, and it fails
  quietly, so the fleet has always used the stale built-in table.

Two things we got wrong that are worth encoding as warnings:

• Bigger is not better for agent work. DeepSeek's own release notes say not
  to move to V4-Pro for agent tasks on price grounds — Flash leads the agent
  suites. A naive "rank by size/price" tiering inverts this.
• Benchmarks measure agent+model pairs, not models. The same model swings
  ~8 points between harnesses. A ranking that ignores the harness does not
  predict the fleet running these CLIs.

So: auto-tier from an external agentic benchmark, refresh on a schedule, and
make the source pluggable — but treat the result as a proposal, not an
auto-adopted change. Which leads to:

4. Operator-editable policy

Ours is a bash heredoc. That is the wrong shape for something an operator must
tune. The decisions a human needs to make:

• which tier each agent requires (a competence floor, not a cost knob)
• provider preference order — ours puts free before metered, which is not
  universal
• per-provider exhaustion thresholds, since a shared subscription and a private
  one deserve different caution
• whether a newly-benchmarked model is adopted automatically or offered

And one behaviour worth stating explicitly, because it is counter-intuitive:
rotate on exhaustion, never on preference. If a provider is usable and the
current rung is legitimate for the agent's tier, leave it alone even when
something better-ranked exists — otherwise the fleet churns every tick chasing
whichever pool is momentarily cheapest.

What we would upstream first

1. Document the atomic PUT /api/config/agent/{name}/models as the
   preferred path over switch then model. Two calls with no transaction
   leave codex --model claude-opus-5 — unlaunchable — when the second times
   out. It is also ~2.4s versus ~45s. Pure docs; removes a live footgun.
2. A model-inventory interface a backend can implement, plus validation of
   a model id before it is set. Two of four backends can already answer.
3. A headroom interface, unscoped-vs-scoped separated, with the
   consumer-deadlock invariant written down.
4. Exhaustion as an agent condition, alongside Authenticated /
   Producing. Hive detects and reports agent state well; it has nowhere to
   say "the backend is out of quota".
5. Optional placement engine, off by default, consuming 2–4.

1–4 are useful even if nobody builds 5. We are not asking hive to adopt our
tier rankings — those are ours, and they change.

Environment

Two spokes + self-hosted hub on Talos, ACMM L5/L6, backends claude / agy /
codex / pi across Claude Max, Google AI Pro, ChatGPT Plus and DeepSeek prepaid.
Related: #5632, #5633, #5661, #5690, #5691.

[clubanderson]

Dispatching this RFC against v5 tip 67430b2222f7552a6fc93372ef49f21887f3e9ee.

Assessment of current v5:

• There is already a provider-rotation package in src/pkg/rotation/rotation.go: it has Headroom, Prober, Claude/Codex/agy/DeepSeek probes, fail-open semantics, Manager.ShouldRotate/NextBackend/Exhausted, and GET /api/providers/headroom response shaping. It is a strong starting point, but it still treats Claude's max percent across limits as the binding signal; this RFC's scoped-vs-unscoped distinction should be corrected before relying on it for placement.
• Config already has an opt-in config.RotationConfig in src/pkg/config/governor_config.go with provider classes, backend lists, threshold, high-volume subscription guard, and agent tier names.
• The dashboard already has runtime CLI model discovery in src/pkg/dashboard/cli_models.go and agy inventory in src/pkg/dashboard/cli_models_agy.go; inference model discovery is surfaced through Server.buildInferenceBackends in src/pkg/dashboard/server.go.
• The atomic backend+model endpoint already exists at PUT /api/config/agent/{name}/models (src/pkg/dashboard/api.go, handleAgentConfigModels) and validates backend names, but it does not validate model IDs against live inventory before persisting.
• Agent/heartbeat state already reports QuotaExhausted and spoke-level ProviderLimitReason/ProviderLimitRebuffs in src/pkg/hub/heartbeat.go, but this is detection/reporting rather than a normalized placement input.

Phased v5 sketch:

1. Docs/API contract first: document PUT /api/config/agent/{name}/models as the only safe switch+model mutation path, including why two separate calls can create an unlaunchable pair.
2. Model inventory interface: extract the dashboard discovery logic into a reusable package/service (or a narrow interface consumed by dashboard + config mutation), preserving per-backend fallbacks and cache semantics. Wire handleAgentConfigModels to reject a non-empty model when the backend has authoritative live inventory and the ID is absent; keep fallback behavior conservative so transient discovery failures do not brick current configs.
3. Headroom contract: evolve rotation.Headroom to carry unscoped provider headroom plus scoped/model caps separately. Update ClaudeProber so weekly_scoped limits affect model avoidance, not provider exhaustion; add tests with session/weekly_all/weekly_scoped combinations. Keep the existing fail-open invariant for probe errors.
4. Conditions and surfaces: map normalized headroom/exhaustion into agent status/heartbeat alongside the existing QuotaExhausted and ProviderLimitReason fields, so the hub and dashboard can distinguish provider exhaustion, scoped model cap, auth failure, and probe-unknown.
5. Placement engine: extend rotation.Manager from provider-only failover to backend+model placement: consume inventory, headroom, provider preference, agent tier floor, and high-volume rules; only rotate on confirmed exhaustion or invalid model, not preference churn. Keep it opt-in under governor.rotation.
6. Tier policy: add operator-editable tier/policy config and a pluggable refresher for benchmark rankings; store proposed rankings separately from adopted policy so a failed refresh or surprising benchmark cannot silently rewrite placements.

Open questions for @hanthor:

• For Claude usage, can you share a representative limits shape with session, weekly_all, and weekly_scoped so tests can pin the unscoped/scoped split without leaking account data?
• Which model inventories should be authoritative enough to reject at set time when discovery fails: Claude/DeepSeek only, or also agy/codex with their current fallbacks?
• Should model validation be blocking only for new edits, or should startup/reconcile also flag existing invalid configs as degraded without rewriting them?
• How should operator policy express tier floors: per agent name, role, ACMM pack default, or all three with provenance?
• Where should external benchmark rankings come from initially, and should the first implementation only ingest a checked-in/admin-uploaded JSON before adding a network refresher?

[hanthor]

Addendum: a fifth capability — pacing

Everything above treats headroom as an availability question: can this backend serve, yes or no. Running it for another day makes clear that answering only that question wastes the pool in two directions, and a threshold is structurally unable to see either.

Overburn. 22 agents took one Anthropic subscription from 64% to 69% in about 40 minutes against a weekly cap resetting six hours later. The threshold is silent until 90%, at which point the fleet parks — and then sits idle for hours holding quota it could not spend because it spent it early.

Underburn. A cap that resets unused is quota destroyed. A threshold has nothing to say about this at all; it is only ever a brake. Over a week this is the more expensive of the two.

Both are rate problems. So alongside probe / inventory / tiering / policy there is a fifth thing:

Pacing — spend each pool so it lands near-empty exactly at its reset, and no earlier.

This forces a correction to capability 1: headroom is not a scalar

The body already says a model-scoped cap is not provider exhaustion. Building the pacer surfaced a second, sharper version of the same mistake — the unscoped limits are not one number either. Measured 2026-09-02T16:55Z:

percent  37   resets 19:30   (5h session window)
percent  67   resets 23:00   (weekly)
percent 100   resets 23:00   scope.model="Fable"   (excluded — model-class cap)

Collapsing the unscoped pair to max gives "67%, resets 23:00". Correct for availability, useless for rate:

session:  63 points over 2.6h  ->  24.2 %/hr allowed
weekly:   33 points over 6.1h  ->   5.4 %/hr allowed

The binding constraint is whichever limit is closest to violating its own deadline, and that is not recoverable from the larger percent. Worse, the two percentages are on different scales — 1% of the session pool is not 1% of the weekly pool — so the allowed rates cannot be compared directly either.

The scale-free formulation is a ratio:

pressure = max over unscoped limits of ( observed_rate_i / allowed_rate_i )

pressure > 1 means some limit exhausts before it resets. pressure < 1 means some limit resets unspent. Both are failures; 1.0 is the target and a deadband makes it a band.

Concretely, this means the normalized headroom shape in capability 1 should be a list of {percent, resets_at, scope}, not a percent. A consumer asking "can this serve?" takes the max over unscoped entries — today's behaviour, unchanged. A consumer asking "how fast may we go?" takes the min over headroom_i / time_to_reset_i. One shape answers both; a scalar answers only the first. (Same note left on #5691, since the fleet-headroom heartbeat field there needs the identical shape.)

The probe endpoint rate-limits — measurement must be published once and shared

Worth stating as a design constraint because it is invisible until you have two consumers. Adding a second poller against api.anthropic.com/api/oauth/usage beside the existing 20-minute probe produced, immediately:

{"error":{"type":"rate_limit_error","message":"Rate limited. Please try again later."}}

And it degrades both readings — the new consumer's and the existing one's. So the fleet loses its availability signal in order to acquire a pacing signal. Measuring a provider harder is not the same as measuring it better.

This generalizes past our setup: any fleet-wide headroom feature has more than one consumer by construction. The probe must be a single producer publishing a reading that every consumer reads, never a function each consumer calls. This is a stronger constraint than the deadlock invariant already in the body, and it points the same way.

What a pacer needs that the other four capabilities don't provide

History, not a delta. percent is an integer, so a two-point difference is dominated by quantization — our "12 %/hr" was a 2-point delta over 10 minutes, whose true value is somewhere in 6–18 %/hr. A rate needs a fit over several samples, which means the probe layer needs somewhere to persist readings. Ours refuses to emit a verdict below 3 samples spanning an hour and reports learning rather than guessing; the first runs after a deploy must fall through to threshold behaviour, not to an invented rate.

Window-rollover detection. When a cap resets, percent drops 67 → 3, and a fit across that boundary yields a large negative rate that reads as extreme underburn — flooring the accelerator at exactly the moment a fresh window opens. Re-running our fixture with the guard removed gives −23.23 %/hr, so this is not hypothetical. Any persisted-usage feature needs the same guard.

Honest reporting of control authority. Our pacer actuates 22 agents across two spokes but not the contributor CLIs, which draw on the same pools. A controller that owns part of the load, sheds it, observes no improvement because the unowned part absorbed the slack, and sheds again, is a runaway. It states how many consumers it controls, and moves one notch per tick behind a deadband rather than computing a proportional correction to an error it does not fully own.

Saturation must be distinguishable from satisfaction. The actuator has a small finite range — only agents on a provider's expensive rung can be demoted. Once exhausted, the controller reports "hot" and makes no change, which is byte-identical to "hot, nothing needed". That is precisely the state needing a human, so it has to be named:

SATURATED: anthropic is hot but no notch was available
(0 of 11 seated agents demotable) — model-rung pacing is exhausted

Which actuator

Worth flagging because the obvious answer is wrong. Placement is not a pacing lever. Moving an agent to another provider only slows a burn if another provider has room — and there are long stretches where exactly one pool is usable (ours right now: DeepSeek at −$1.23, OpenAI capped for four days, Google inside its own 5-hour cooldown). Placement then has zero degrees of freedom while the burn continues unchanged.

The levers that work with one usable pool are model rung (opus → sonnet is a large change in burn against the same cap) and cadence. We actuate the first because the atomic PUT /api/config/agent/{name}/models already exists and is transactional; cadence would be the natural second, and is the one that keeps working after model rungs saturate.

Status

Running in our fleet as a separate 20-minute CronJob reading the shared publication, pacing both spokes. Happy to write this up as a hive-side design or a patch against whatever lands for capability 1 — the shape change (list of limits instead of a percent) is small and is the prerequisite for everything else here.

[clubanderson]

Progress note: item 1 from the earlier dispatch is now delivered by #5723 and merged to v5. The docs/OpenAPI now point backend+model changes at the atomic PUT /api/config/agent/{name}/models route and explicitly call out the two-call footgun plus the remaining phase-2 model-validation gap. No re-dispatch here; keeping the rest of the RFC open.

[clubanderson]

Closed by merged v5 PR #5784, which lands the accepted RFC document for backend capacity, model inventory, pacing, policy, and opt-in placement.

[clubanderson]

Roadmap anchor follow-up: the backend capacity / model inventory / placement RFC is now reflected in the v4 canonical roadmap and the public Now/Next/Later roadmap, with the v5 port merged too.

• Root roadmap v5 workstreams: https://github.com/hivecommons/hive/blob/v4/ROADMAP.md#v5--next-generation
• Public roadmap Now row: https://github.com/hivecommons/hive/blob/v4/src/docs/roadmap.md#now
• v5 public roadmap port: https://github.com/hivecommons/hive/blob/v5/src/docs/roadmap.md#now
• Production adopter entry for tunaos.org: https://github.com/hivecommons/hive/blob/v4/ADOPTERS.md#adopters-list

Merged in #5830 and #5831; #5773 is closed.