Skip to content

feat(tco-feed): serve the tier-weighted per-hardware score via view=scores / tco-feed 新增 view=scores,直接输出按层级加权的单芯片分数#601

Merged
Oseltamivir merged 2 commits into
masterfrom
feat/tco-feed-scores
Jul 21, 2026
Merged

feat(tco-feed): serve the tier-weighted per-hardware score via view=scores / tco-feed 新增 view=scores,直接输出按层级加权的单芯片分数#601
Oseltamivir merged 2 commits into
masterfrom
feat/tco-feed-scores

Conversation

@Oseltamivir

@Oseltamivir Oseltamivir commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

The points view of /api/v1/tco-feed serves per-tier frontier reads and leaves the weighted aggregation to spreadsheet SUMPRODUCT wiring. This PR adds view=scores: one row per hardware carrying the single number a TCO sheet consumes per chip — tier-weighted, workload-blended, output-equivalent tok/s/GPU — so the sheet's "InferenceMAX score" column becomes one Power Query fetch plus an XLOOKUP, replacing the old single-point read at 100 tok/s/user.

Methodology (all consumer-overridable query params; a published sheet's URL fully records its assumptions):

  • weights — per-tier traffic-mix weights, normalized to sum 1. Default 0.35,0.4,0.2,0.05 on the default tiers 30,50,75,100; equal weights on custom tiers.
  • workload_weights — per-workload blend weights, default equal split. Renormalized over covered workloads when a chip lacks a sweep — a missing workload is a flagged coverage gap (workloads_covered, blank score_<workload> column), not a zero.
  • alpha — input-token value ratio in the output-equivalent conversion out × (1 + α × ISL/OSL). Default 0.25; alpha=0 scores plain output throughput.

Semantics:

  • Unreachable tiers contribute 0 at full weight (chip genuinely can't serve that traffic segment), matching the points view's documented boundary behavior; unreachable_tiers/clamped_tiers columns surface how much of a score rests on boundary handling.
  • Scores aggregate the already-rounded points-view values, so view=scores is exactly SUMPRODUCT over view=points — reproducible by hand (verified: b200 8k1k 2992.49 both ways on fixtures).
  • Provenance: latest_date / oldest_frontier_date carried as max/min across covered workloads.

Validation hardening:

  • Duplicate workloads entries now 400 (they'd double-count in the blend).
  • weights/workload_weights/alpha passed with view=points → 400 rather than silently ignored.
  • New 400s for weight-count mismatch, negative weights, zero sums, alpha outside [0, 10].

view=points output is byte-identical to before (the duplicate-workloads rejection is the only behavior change).

Testing

  • 19 new unit tests in tco-feed.test.ts (36 total): parse defaults/normalization/rejections for the three new params, exact weighted-sum arithmetic at frontier knots, unreachable-at-full-weight, clamp inclusion, partial-coverage renormalization, α factors per workload, zero-covered-weight guard, provenance dates, dynamic-column CSV shape with blank uncovered fields.
  • End-to-end on the E2E_FIXTURES=1 dev server: scores JSON/CSV, custom tiers/weights/alpha, all six invalid-param 400 cases, and the SUMPRODUCT-reproducibility cross-check against view=points.
  • pnpm typecheck / lint / fmt clean; unit-suite failures are the three known environment-specific files (visit-tracking, chunk-load-recovery, use-feature-gate) that pass in CI, untouched here.

Overlay-run support: N/A — API-only change with no chart surface; both views are computed from official-run DB rows only.

中文说明

/api/v1/tco-feedpoints 视图只提供逐层级的前沿读数,加权聚合此前要靠电子表格里的 SUMPRODUCT 公式完成。本 PR 新增 view=scores 视图:每款硬件一行,直接给出 TCO 表格所需的单一数值——按 interactivity 层级加权、跨 workload 混合、经输出等效换算的 tok/s/GPU——表格中的 "InferenceMAX score" 列从此只需一次 Power Query 拉取加一个 XLOOKUP,取代旧的 100 tok/s/user 单点读数。

方法论(全部为消费方可覆盖的查询参数,已发布表格的 URL 完整记录其全部假设):

  • weights — 逐层级流量组合权重,归一化为和 1。默认层级 30,50,75,100 下默认为 0.35,0.4,0.2,0.05,自定义层级默认均分。
  • workload_weights — 跨 workload 混合权重,默认均分。当某款芯片缺少某个 workload 的数据时,仅在有数据的 workload 上重新归一化——缺失的 sweep 是被明确标记的覆盖缺口(workloads_covered 列、空白的 score_<workload> 列),而不是记零分。
  • alpha — 输出等效换算 out × (1 + α × ISL/OSL) 中的输入 token 价值比。默认 0.25alpha=0 即纯输出吞吐量。

语义:

  • 不可达层级按全权重计 0(芯片确实无法服务该流量段),与 points 视图既有的边界语义一致;unreachable_tiers/clamped_tiers 列展示分数中有多少依赖边界处理。
  • 分数基于 points 视图已舍入的数值聚合,因此 view=scores 与对 view=points 做 SUMPRODUCT 完全一致,可手工复算(已在 fixtures 上验证:b200 8k1k 两种算法均为 2992.49)。
  • 溯源信息:latest_date / oldest_frontier_date 取有数据 workload 的最大/最小值。

参数校验加强:

  • 重复的 workloads 现在返回 400(否则会在混合中重复计数)。
  • view=points 上传入 weights/workload_weights/alpha 返回 400,而非被静默忽略。
  • 权重个数不匹配、负权重、权重和为零、alpha 超出 [0, 10] 均返回 400。

view=points 的输出与之前逐字节一致(唯一的行为变化是拒绝重复 workload)。

测试: tco-feed.test.ts 新增 19 个单元测试(合计 36 个),覆盖三个新参数的默认值/归一化/拒绝路径、前沿节点处的精确加权求和、不可达层级全权重计零、钳位值纳入、部分覆盖时的重新归一化、逐 workload 的 α 系数、零覆盖权重防护、溯源日期以及动态列 CSV 结构;并在 E2E_FIXTURES=1 开发服务器上端到端验证了 JSON/CSV、自定义参数、全部六种非法参数 400 以及与 points 视图的 SUMPRODUCT 一致性。

叠加运行(overlay)支持:不适用——本改动仅涉及 API,无图表界面;两个视图均只基于官方运行的数据库行计算。


Note

Low Risk
API-only change to the documented tco-feed exception path, with extensive unit tests and stricter validation; default points view behavior is preserved except rejecting duplicate workloads.

Overview
Adds view=scores to /api/v1/tco-feed: one row per hardware with a single tier-weighted, workload-blended, output-equivalent tok/s/GPU for spreadsheet TCO models, instead of wiring SUMPRODUCT over view=points.

Scoring assumptions are explicit query params—weights, workload_weights, and alpha (with documented defaults)—and are echoed in JSON responses so a published Power Query URL records methodology. computeTcoScores aggregates rounded point values (SUMPRODUCT-equivalent), handles unreachable tiers as zero at full weight, and renormalizes workload weights when a chip lacks a sweep.

Validation: duplicate tiers/workloads now 400; score-only params with view=points return 400 instead of being ignored. view=points output is unchanged aside from duplicate-workload rejection.

Docs in AGENTS.md and route comments are updated; unit tests cover parsers, scoring math, and CSV shape.

Reviewed by Cursor Bugbot for commit 92cb8f6. Bugbot is set up for automated code reviews on this repo. Configure here.

The points view leaves the weighted aggregation to spreadsheet
SUMPRODUCT wiring; view=scores serves the single per-chip number
directly. Tier-weighted (weights=, default 0.35/0.4/0.2/0.05 traffic
mix on the default tiers, equal weights otherwise), workload-blended
(workload_weights=, default equal split, renormalized over covered
workloads so a missing sweep is a flagged coverage gap rather than a
zero), output-equivalent (alpha=, default 0.25 in out x (1 + a*ISL/OSL)).
Unreachable tiers contribute 0 at full weight, matching the points
view's boundary semantics. Scores aggregate the already-rounded points
values, so a consumer can reproduce them exactly by SUMPRODUCT over
view=points. Duplicate workloads now 400 (they would double-count in
the blend); weights/alpha on view=points 400 rather than being
silently ignored.

中文:tco-feed 新增 view=scores 视图,直接返回每款硬件的单一分数,
不再依赖电子表格里的 SUMPRODUCT 公式。分数按 interactivity 层级加权
(weights 参数,默认层级下为 0.35/0.4/0.2/0.05 的流量组合,自定义层级
默认均分)、跨 workload 混合(workload_weights 参数,默认均分,仅在有
数据的 workload 上重新归一化——缺失的 sweep 是被标记的覆盖缺口而非零
分)、并做输出等效换算(alpha 参数,默认 0.25,公式 out × (1 + α·ISL/OSL))。
不可达层级按全权重计 0,与 points 视图的边界语义一致。分数基于 points
视图已舍入的数值聚合,消费方可用 SUMPRODUCT 精确复算。重复的 workload
现在返回 400(否则会在混合中重复计数);在 view=points 上传入 weights/alpha
也返回 400 而非被静默忽略。
@Oseltamivir
Oseltamivir requested a review from adibarra as a code owner July 20, 2026 14:27
@vercel

vercel Bot commented Jul 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
inferencemax-app Ready Ready Preview, Comment Jul 20, 2026 2:34pm

Request Review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1d6c43b. Configure here.

Comment thread packages/app/src/lib/tco-feed.ts
…d guard

A repeated tier would collect weight from every duplicated slot and
silently skew the scores view, exactly like duplicate workloads —
flagged by Bugbot. 400 instead.

中文:重复的 tier 会在每个重复槽位上累计权重,悄悄扭曲 scores 视图的
分数,与重复 workload 的问题相同(Bugbot 发现)。现改为返回 400。
@Oseltamivir
Oseltamivir merged commit 3959bff into master Jul 21, 2026
26 checks passed
@Oseltamivir
Oseltamivir deleted the feat/tco-feed-scores branch July 21, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant