Skip to content

AGENTS.md

Latest commit

 

History

History
361 lines (265 loc) · 37.1 KB

AGENTS.md

File metadata and controls

361 lines (265 loc) · 37.1 KB
Raw

AGENTS.md

For detailed subsystem docs, see docs/index.md.

  • Pareto logic changes must update both InferenceX and InferenceX-app with matching regression tests and cross-linked PRs.

AI model disclosure

Every PR description must include an AI model disclosure section naming the exact model/version used to prepare the PR. List each contributing model and its role, including delegated agents. Tool names such as Claude Code, Cursor, or Perplexity Computer are not model identities. Copy the model identifier exposed by the runtime; do not guess an unavailable identifier. If the runtime does not expose the exact model, explicitly state that it could not be verified. Human-only PRs must state No AI used. Keep the disclosure current when later edits use another model.

Contribution requirements

PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题> (keep bracket prefixes at the front untranslated). In the PR/issue body, follow the English content with a ## 中文说明 section mirroring the summary; don't translate code blocks, logs, or stack traces — summarize around them. Commit messages must include a Chinese translation too: keep the subject line in English (conventional-commit style) and include the Chinese translation of the subject and key points in the commit body (e.g. a trailing 中文：<translation> paragraph); squash-merge commits inherit the bilingual PR title, which satisfies the subject requirement automatically.

Translation quality bar: write natural technical Chinese, not word-for-word machine translation (style reference: vllm-project/vllm-ascend README.zh.md). Preserve product names, hardware SKUs, framework/library names (Next.js, React Query, D3.js, Tailwind ...), flags, and code identifiers in English. Use parenthetical English clarification for acronyms on first use. Preferred terms: benchmark 基准测试, dashboard 仪表板, chart 图表, config 配置, throughput 吞吐量, latency 延迟, single-node/multi-node 单节点/多节点, evaluation 评估, artifact 产物. In first-party ML infrastructure UI and technical prose, keep established English technical terms and phrases that Chinese engineers normally use in English; decide from real industry usage and the surface, not from a closed list. warmup, seed, and offload are examples, not the whole category. Follow docs/chinese-copy.md for explanatory and quotation exceptions.

Chinese editorial decisions are context-aware. Follow docs/chinese-copy.md for audience, surface-specific register, terminology exceptions, the two independent fidelity/naturalness gates, the Claude-plus-manual-review workflow, and the PR checklist. Do not turn a preferred wording or pronoun choice into a global mechanical rule unless it is correct in every supported context.

The website itself is bilingual too — every indexable page must ship a Simplified Chinese sibling under /zh. See Chinese Website Pages below; a new page, tab, or blog post without its /zh version is 🔴 BLOCKING on PR review.

Cursor Bugbot re-reviews on EVERY push — each new commit to a PR can surface new inline comments, including on code an earlier review passed. Before merging, loop until convergence: wait for checks (the Bugbot review is one of the PR checks) → fetch unresolved review comments → fix or answer each with a reply → push → repeat until a push produces no new findings. Branch rules require all review threads resolved before merge, so resolve addressed threads as you go.

Project Overview

InferenceX App — Next.js 16 dashboard for ML inference benchmark data. DB-backed with Neon PostgreSQL, React Query for data fetching, D3.js for charts.

  • Framework: Next.js 16 (App Router, Turbopack)
  • Language: TypeScript (strict mode)
  • Styling: Tailwind CSS 4 + shadcn/ui (Radix UI primitives)
  • Charts: D3.js — shared library at src/lib/d3-chart/, scatter/GPU/bar charts
  • Data: Neon DB → API routes (/api/v1/*) → React Query hooks → Context providers
  • Deployment: Vercel with daily cron-triggered rebuilds
  • Analytics: PostHog (posthog-js) via @/lib/analytics — recommended on all interactive elements (autocapture provides baseline coverage)

Quick Start

bun install               # Install dependencies
bun run dev                # Dev server with Turbopack (http://localhost:3000)
bun run mcp                # MCP server exposing read-only benchmark tools
bun run build              # Production build
bun run typecheck          # TypeScript type checking (all packages)
bun run lint               # Lint with oxlint
bun run lint:fix           # Auto-fix lint issues
bun run fmt                # Format check with oxfmt
bun run fmt:fix            # Auto-fix formatting
bun run test:unit          # Vitest unit tests
bun run test:e2e           # Cypress smoke suite, the default local check
bun run test:e2e:full      # Full Cypress suite, normally covered by CI

Monorepo Structure

packages/
├── app/                  # Next.js frontend (@semianalysisai/inferencex-app)
│   ├── content/blog/     # MDX blog posts (frontmatter + content)
│   └── src/
│       ├── app/          # Pages, layouts, API routes (/api/v1/*)
│       │   └── blog/     # Blog list + [slug] post pages, OG image generation
│       ├── components/   # Tab sections: inference/, evaluation/, historical-trends/,
│       │                 #   throughput-calculator/, reliability/, gpu-specs/, blog/, ui/
│       ├── hooks/api/    # React Query hooks (use-benchmarks, use-availability, etc.)
│       └── lib/          # Utilities, constants, d3-chart/, chart-utils, blog, data-mappings
├── constants/            # Shared constants (GPU keys, model mappings, SEO)
├── db/                   # DB layer, ETL, migrations, queries, ingest scripts
└── mcp/                  # MCP server exposing read-only benchmark tools

Path alias: @/* → packages/app/src/

Data Architecture

Frontend → React Query hooks (src/hooks/api/) → /api/v1/* routes → Neon DB

API routes (packages/app/src/app/api/v1/):

  • benchmarks?model=X&date=YYYY-MM-DD — latest benchmark per (config, concurrency)
  • benchmarks/history?model=X&gpu=Y — historical benchmark data for trend charts
  • workflow-info?date=YYYY-MM-DD — runs, changelogs, configs for a date
  • availability — Record<model, dates[]>
  • reliability — raw ReliabilityRow[]
  • evaluations — raw EvalRow[]
  • server-log — retrieve benchmark runtime logs
  • invalidate — invalidate API cache (admin; ?scope=collectivex purges only that scope)
  • collectivex/latest, collectivex/runs, collectivex/runs/[runId] — CollectiveX sweep data from a separate Neon DB, populated lazily on read from GitHub Actions artifacts and served assembled through the shared reader (the one deliberate exception to the raw-rows rule below); runs/[runId] also handles admin DELETE. See CollectiveX.
  • tco-feed?model=dsv4&workloads=1024x1024,8192x1024&tiers=30,50,75,100&format=csv — per-hardware Pareto-frontier output-throughput reads at fixed interactivity tiers, for external spreadsheet TCO models (Excel Power Query); view=scores (optional weights, workload_weights, alpha) folds them into one tier-weighted, workload-blended, output-equivalent score per hardware
  • overview?tier=50&engine=community&compare=30d&ref=b200 — a compact, cached page-data response used only by /overview selector navigation

API routes return raw DB data — no presentation logic. Frontend handles all transformations. Exceptions: the CollectiveX routes assemble raw stored documents through the shared reader in packages/db/src/collectivex/ (see docs/collectivex.md for why); and tco-feed, which runs the calculator's frontier interpolation server-side because its consumers (spreadsheets) cannot execute the TS transforms — its assumptions (tier weights, workload mix, α) enter only as explicit query params with documented defaults, so a published sheet's URL fully records its methodology; and overview, which assembles the same OverviewPageData used for the initial server render so selector changes can update the matrix without transferring every model's raw benchmark history or triggering a React Server Component (RSC) round trip. It is a page-owned backend-for-frontend (BFF), not a reusable public data API.

API Documentation Synchronization

The public API reference at /api and /zh/api, plus the OpenAPI 3.1 document at /api/openapi.json, are generated from packages/app/src/lib/api-documentation.ts.

Any change to an API route, request parameter, response shape, status code, authentication, caching behavior, or shared API type MUST update the documentation registry in the same change. Keep packages/app/src/lib/api-route-catalog.ts synchronized with every handler under packages/app/src/app/api/; the catalog classifies unpublished routes and records review digests for handlers and shared contract sources. Do not update a digest without first confirming whether the human reference, Chinese copy, examples, or OpenAPI schema also need changes.

Run the synchronization guard from packages/app:

bun --env-file=../../.env vitest run src/lib/api-route-catalog.test.ts

Static content routes (no DB):

  • /blog — blog listing (statically generated from MDX files in content/blog/)
  • /blog/[slug] — blog post page with MDX rendering and OG image generation
  • /whitepaper — whitepaper index; /whitepaper/[slug] — research-paper landing page (registry in src/lib/whitepapers.ts; PDF, cover, hardware render, and light/dark figure PNGs under public/whitepaper/<slug>/). Load .claude/skills/write-inferencex-whitepaper/ to write or update a paper: it holds the numbers workflow, copy rules, the PDF/chart pipeline, and the registry checklist.
  • /feed.xml — RSS 2.0 feed
  • /llms.txt — LLM-readable site index
  • /llms-full.txt — full article content for LLM ingestion
  • /sitemap.xml — dynamic sitemap (includes blog posts)

Code Style & Tooling

  • Linter: oxlint — bun run lint / bun run lint:fix
  • Formatter: oxfmt — bun run fmt / bun run fmt:fix
  • Type checking: bun run typecheck (tsc --noEmit, strict mode)
  • Typography gate: bun run check:typography (CI + pre-commit) — no new arbitrary font sizes (text-[11px]) or letter-spacing (tracking-[0.16em]) in class strings, and no quoted font-size literals in src/lib/d3-chart/. Use text-2xs/text-3xs, tracking-eyebrow/tracking-eyebrow-wide/tracking-heading, the <Heading>/<Eyebrow> components, or CHART_TYPE from @/lib/d3-chart/typography (chart font sizes must stay in TS — CSS variables don't survive PNG export). Existing offenders live in packages/app/scripts/typography-allowlist.json and migrate on touch. See Typography.
  • Node: 24.x

Environment Variables

See .env.example. Key vars: GITHUB_TOKEN, DATABASE_READONLY_URL, DATABASE_WRITE_URL (admin only).

Testing

See Testing for full requirements, quality standards, and pre-commit checklist. Tests are mandatory — missing/low-quality tests are 🔴 BLOCKING on PR review.

E2E Runtime and PR Workflow

  • Prefer bun run test:e2e while iterating. It runs the local smoke suite and avoids blocking development on the full browser matrix.
  • Use bun run test:e2e:full only when a local full-suite run is useful. The merge gate runs the full suite in GitHub Actions.
  • A warm local full E2E run typically takes about 4–6 minutes because the component and integration suites run sequentially on one machine; cold dependency/browser setup can take longer.
  • GitHub Actions runs integration specs across four shards per browser for Chrome and Firefox (eight parallel E2E jobs), while component tests run in a separate job. Recent successful workflows complete in roughly 3–5 minutes.

Analytics Requirement

All interactive elements should have track() from @/lib/analytics (autocapture provides baseline coverage).

Convention: [section]_[action] — e.g., latency_zoom_reset, calculator_bar_selected, tab_changed

Prefixes: latency_, interactivity_, gpu_timeseries_, inference_, calculator_, evaluation_, reliability_, tab_, selector_, blog_, whitepaper_, social_

Tab Structure

Dashboard route keys, paths, primary/feature-gated/footer-only navigation, indexability, provider capabilities, /zh mirroring, and share-parameter scopes are defined once in packages/app/src/lib/dashboard-routes.ts. TabNav, DashboardShell, metadata, i18n, share URLs, and the sitemap derive from that registry; do not add a second route array.

Unofficial Run Support — Mandatory for Inference / Evaluation Features

Any new feature that operates on inference or evaluation chart data must also work for unofficial run overlays — not just the official run rendering path. The overlay path is a separate code branch (overlayData, processedOverlayData, overlayRooflines, activeOverlayHwTypes, overlayRunColor/overlayRunIndex from @/lib/overlay-run-style, useUnofficialRun() from @/components/unofficial-run-provider) that is easy to forget — features that only handle the official path silently degrade for users who load an unofficial run via ?unofficialrun=….

When adding a chart feature (toggle, label, overlay, filter, export, share-link param, tooltip enrichment, …):

  1. Implement it for both official and overlay data paths. Use overlayRunColor(runIndex) for overlay strokes / labels so they match the legend swatches; do not reuse the hw-derived color helper (getCssColor(resolveColor(hw))) for overlay items.
  2. Respect overlay visibility filters: activeOverlayHwTypes (hw toggles) and any per-run dismissal in unofficialRunInfos. Don't draw overlay items the user has hidden.
  3. Verify it manually with an unofficial run loaded — paste a ?unofficialrun=<github-actions-run-id> URL and confirm the new feature renders for overlay rooflines / points / rows, animates with zoom, and survives a per-run dismiss.
  4. Add at least one E2E or unit test that exercises the overlay path. The mock helper createMockUnofficialRunContext (cypress/support/mock-data.ts) and the cypress/e2e/inference-chart.cy.ts overlay setup are good starting points.
  5. Note overlay support explicitly in the PR description so reviewers can verify it ("works for both official runs and ?unofficialrun= overlays — verified at ").

If the feature genuinely cannot apply to overlays (e.g., it depends on data only ingested for official runs), say so explicitly in code comments and the PR description. Default to "must support overlays."

Chinese Website Pages (/zh) — Mandatory for All Indexable Surfaces

The site ships a hand-authored Simplified Chinese sibling for every indexable page under the /zh route prefix (/ ↔ /zh, /about ↔ /zh/about, /blog/<slug> ↔ /zh/blog/<slug>, …) so the site is crawled and indexed in Chinese as well as English. There is no i18n framework — each /zh page is a real page that reuses the shared helpers in packages/app/src/lib/i18n.ts (zhAlternates, enAlternates, ZH_OG_LOCALE, ZH_MIRRORED_ROUTES) and src/lib/tab-meta-zh.ts. The translation quality bar above applies to all site content.

Every new indexable page, dashboard tab, or blog post MUST ship its Chinese version in the same PR:

  1. New page → create packages/app/src/app/zh/<route>/page.tsx with fully translated content and metadata. Metadata: alternates: zhAlternates('<en-path>') plus openGraph.locale: ZH_OG_LOCALE. Switch the English page's alternates to enAlternates('<en-path>') so both sides carry bidirectional hreflang. Register non-dashboard routes in ZH_MIRRORED_ROUTES (src/lib/i18n.ts) so the header nav and EN↔中文 toggle link to them, and add them to the sitemap via localizedPair().
  2. New dashboard tab → add one entry to src/lib/dashboard-routes.ts, then add exact entries to TAB_META_ZH, TAB_INTRO_ZH, and TAB_LABELS_ZH in src/lib/tab-meta-zh.ts. Create src/app/zh/(dashboard)/<tab>/page.tsx mirroring the English page with tabMetadataZh('<tab>') and a <ZhTabIntro tab="<tab>" /> block above the chart; the registry automatically supplies /zh route matching and indexable sitemap entries. The chart's own UI strings must follow rule 5. Registry and metadata tests enforce route parity and dictionary completeness.
  3. New blog post → the translation packages/app/content/blog/zh/<same-filename>.mdx is REQUIRED in the same PR. Translate frontmatter title/subtitle and the body; keep date, publishDate, modifiedDate, tags, and the filename/slug identical (English and Chinese posts pair by filename; visibility gating always follows the English post's publishDate). Rewrite internal /blog/<slug> links to /zh/blog/<slug>; never alter numbers, code blocks, or <Figure>/<JsonLd> structure. The /zh/blog listing, hreflang, and sitemap pick the file up automatically.
  4. Editing an existing English page or post → update its Chinese sibling in the same PR. Omitting the required Chinese sibling update is a 🔴 BLOCKING review issue. Fidelity or wording issues inside an updated Chinese sibling follow the advisory workflow below; serious findings require Chinese maintainer confirmation.
  5. ALL user-visible UI strings MUST have a Chinese equivalent — no carve-outs for "chart internals" or "option labels". This includes: headers/footers, card titles/descriptions, control and filter labels, buttons, toggles (Log Scale, Optimal Only, …), nudges, dropdown OPTION display names (Y-axis metric names, token types, scale modes), searchable-select placeholders ("Search…"), table column headers and action buttons ("Prompts"), modal/drawer chrome, legend footnotes, and empty/loading/error messages. Mechanism: client components call useLocale() (src/lib/use-locale.ts) and read from a component-local STRINGS = { en, zh } dict; server components take an optional locale prop passed from the /zh page; registry-defined display names (e.g. Y_AXIS_METRICS, legend toggle configs) carry a labelZh field resolved through a locale-aware label helper at render time. The en values must keep the exact original strings so English pages stay byte-identical.
  6. What stays English: brand/product names, hardware SKUs, model/framework/precision names, units (tok/s/user, GB/s, $/M tok), code identifiers and flags — per the translation quality bar — plus established English technical terms and phrases that Chinese ML infrastructure engineers normally use in English, as documented contextually in docs/chinese-copy.md (warmup, seed, and offload are non-exhaustive examples), and DB-stored content (benchmark rows, dataset conversation text, run logs), which is data, not UI.
  7. Compare slug narrative sync: the per-slug compare pages are mirrored at /zh/compare/[slug] and /zh/compare-per-dollar/[slug]; their Chinese prose templates live in src/lib/compare-ssr-zh.ts, a 1:1 port of the English templates in compare-ssr.ts. The variant compare pages (/zh/compare-precision/[slug] and /zh/compare-spec-decode/[slug]) have their Chinese templates in src/lib/compare-variant-ssr-zh.ts, porting compare-variant-ssr.ts. Any PR that changes the English narrative templates MUST update the zh port in the same commit.
  8. Every route gets a /zh sibling, including hidden or feature-gated ones (/agentx, /ai-chart, /current-inferencex-image, /feedback, agentic detail pages). Noindex routes keep their noindex on both sides. The only exceptions are feed.xml and llms.txt (single-language machine feeds) plus per-post OG images. Chinese posts reuse the English post's OG image because the renderer font has no CJK glyphs.

Claude reviews every changed user-visible Chinese string, but the Chinese maintainer makes the final decision. Load review-zh-copy before opening or reviewing any PR that touches user-visible Chinese text, including refactors whose filenames do not contain zh. The skill evaluates semantic fidelity and natural Chinese as separate gates and follows docs/chinese-copy.md. Its findings never block another contributor's merge. Do not mention @edwingao28 for a clean review, routine coverage, or an ordinary wording or naturalness suggestion. Mention @edwingao28 only for a high-confidence semantic or factual error, changed attribution or speaker voice, or unresolved high-impact ambiguity, and ask for confirmation. The Chinese maintainer makes the final editorial decision.

Chinese copy has a narrow objective CI guard. packages/app/src/lib/zh-objective-guard.test.ts checks direction-aware App Router page parity (the documented Chinese-only /zh/[...notFound] catch-all is the sole current exception), explicit en/zh object-literal key shape, Blog filename and non-translatable MDX structure, and English-byte preservation for PRs labeled chinese-copy-only. packages/app/src/lib/zh-copy.test.ts retains its separately hand-checked mechanical cases. New objective rules must use the same parsing logic in the real source scan and their mutation tests. These guards do not judge fluency, clause order, sentence structure, register, contextual pronouns, marketing tone, quotation voice, English-token ratios, or contextual terminology; those require page context and human review. See docs/chinese-copy.md for the complete boundary.

Chart Interpolation — TS and Python Helpers MUST Stay in Sync

The blog-writing workflow (.claude/skills/write-inferencex-blog/) ships a Python port of the chart's interpolation algorithm at .claude/skills/write-inferencex-blog/iso_interactivity.py. It exists so iso-interactivity tables in blog posts produce exactly the same numbers readers see when they hover the rendered chart. Linear-interpolation shell scripts will produce visibly different values — Cursor Bugbot has flagged this on prior posts.

The Python helper is a 1:1 port of these three TypeScript functions:

  • paretoFrontUpperLeft — packages/app/src/components/calculator/interpolation.ts
  • monotoneSlopes (Steffen 1990, matches d3.curveMonotoneX) — same file
  • hermiteInterpolate — same file

Plus the wrapper interpolateMetricAtInteractivity in packages/app/src/components/inference/hooks/useInterpolatedTrendData.ts which composes them with the "no extrapolation → return null" rule.

Plus recoverReciprocalNumerator in interpolation.ts, which decides whether a metric is splined directly or derived from the interpolated throughput. $/M tok and J/token are a per-chip constant over a throughput, so independently splining the metric breaks that identity between knots; both TS and Python spline the throughput and re-derive instead. See docs/tco-calculator.md for the reproducible measurement.

Rule: any PR that changes any of those four TypeScript functions MUST also update .claude/skills/write-inferencex-blog/iso_interactivity.py in the same commit. Drift between the TS and Python implementations means the blog tables will silently diverge from the live chart on the very next post — readers will see one number in the table and a different one in the chart they click through to. This includes:

  • Changing the Pareto frontier definition (upper-left → lower-left, or adding tie-breaking rules)
  • Switching from Steffen's monotone slopes to a different spline construction (Fritsch-Carlson, natural cubic, etc.)
  • Loosening or tightening the extrapolation rule (currently: return null outside [min x, max x])
  • Changing which metrics are derived from throughput rather than splined, or the tolerance that decides whether the data obeys metric x throughput = constant
  • Adjusting the Y-clamp behavior that prevents spline overshoot

The Python file has a header comment explaining the pipeline and a _cli() entrypoint for stdin/stdout JSON usage. When you update it, keep the structure 1:1 with the TS so future readers can diff the two files line by line. Run the helper against a known dataset and confirm the outputs match what the chart renders before merging.

Model Parameter Counts (verified)

Authoritative total / active parameter counts for every model in the dashboard. Use these when updating MODEL_CONFIG labels in packages/app/src/lib/data-mappings.ts or any blog/docs prose. Verify against the HF model card before adding a new model — point releases (e.g. K2 → K2.5, GLM-4.5 → GLM-5) often keep or change sizes in non-obvious ways.

Model Total Active HF ID Source
DeepSeek-R1-0528 671B 37B deepseek-ai/DeepSeek-R1-0528 HF model card
DeepSeek-V4-Pro 1.6T 49B deepseek-ai/DeepSeek-V4-Pro HF model card
DeepSeek-V4.1-Flash 552B 8B / 16B deepseek-ai/DeepSeek-V4.1-Flash HF model card
Kimi-K2.5 1T 32B moonshotai/Kimi-K2.5 HF model card
Kimi-K2.6 1T 32B moonshotai/Kimi-K2.6 HF model card
Kimi-K2.7-Code 1T 32B moonshotai/Kimi-K2.7-Code HF model card
Qwen3.5-397B-A17B 397B 17B Qwen/Qwen3.5-397B-A17B HF model card
Qwen3.8-Flash-Next 176B 6B Qwen/Qwen3.8-Flash-Next-FP8 HF model card
Qwen3.8-27B 27B 27B (dense) Qwen/Qwen3.8-27B HF model card
GLM-5 744B 40B zai-org/GLM-5 HF model card
GLM-5.1 744B 40B zai-org/GLM-5.1-FP8 HF model card (same base as GLM-5)
MiniMax-M2.5 230B 10B MiniMaxAI/MiniMax-M2.5 HF model card
MiniMax-M2.7 230B 10B MiniMaxAI/MiniMax-M2.7 NVIDIA M2.7 blog
gpt-oss-120b 120B 5.1B openai/gpt-oss-120b HF model card
Llama-3.3-70B-Instruct 70B 70B (dense) meta-llama/Llama-3.3-70B-Instruct HF model card

Common mislabel traps (have all bitten this repo at least once — do not repeat):

  • DeepSeek-V4.1-Flash is 552B, and its active count is two numbers. The Causal Encoder-Decoder split means 8B activates during prefill and 16B during decode — quoting a single "active" figure loses that. The 196B Engram conditional-memory table is excluded from the 552B backbone total: it is sparsely accessed by token lookup, not resident per-token compute, so it is treated like a separate MTP head rather than like Qwen3.8's n-gram embedding table (which IS counted). Hugging Face safetensors metadata says 485B, which matches neither figure. Do not fold it into the dsv4 bucket — V4.1-Flash is a different architecture (CED + CSA2), not a V4-Pro point release.
  • Qwen3.8-Flash-Next is 176B, not 125B. The model card leads with "125B with 6B activated", but that is the main model only; the 51B n-gram embedding table brings the total to 176B. The separate 4B MTP head sits outside both figures. It is a Qwen4-architecture preview (GatedDeltaNet + Qwen Sparse Attention, 512 experts, 10 routed + 1 shared), not a Qwen3.5 point release, so it gets its own DB bucket.
  • GLM-5 ≠ 355B. 355B is GLM-4.5. GLM-5 jumped to 744B / 40B active (256-expert MoE with DSA).
  • MiniMax-M2.5/M2.7 ≠ 456B. 456B is the older MiniMax-Text-01 / M1 (32 large experts). The M2 series is a different architecture: 230B / 10B active, 256 small experts.
  • DeepSeek-R1 is 671B, not 685B. HF metadata shows 685B because the bundled MTP head adds ~14B; the core MoE is 671B / 37B active.
  • Kimi K2.5, K2.6, and K2.7-Code are post-training refinements, not new pre-trained sizes. Same 1T / 32B / 384-expert backbone as the original K2. K2.7-Code is a coding-focused refinement of the same backbone.

Common Development Tasks

Modify chart appearance/behavior

  • D3 scatter plot: src/components/inference/ui/ScatterGraph.tsx
  • D3 GPU graph: src/components/inference/ui/GPUGraph.tsx
  • Chart layout/errors: src/components/inference/ui/ChartDisplay.tsx
  • Shared D3 library: src/lib/d3-chart/ (setup, axes, grid, watermark, layers)

Change chart filters/state

  • State: src/components/inference/InferenceContext.tsx
  • Controls: src/components/inference/ui/ChartControls.tsx
  • Filter logic: src/components/inference/hooks/useChartData.ts

Add/modify a metric

  1. Register the field, bilingual labels/titles, polarity, and custom-source metadata in src/components/inference/metric-registry.ts (METRIC_REGISTRY).
  2. Add or reuse the derived field in buildDerivedChartFields (src/lib/chart-utils.ts) and the corresponding optional InferenceData field. Historical trends request the same selective builder; do not duplicate formulas.
  3. Add the metric to METRIC_CONTROL_GROUPS when it belongs in a selector group. Chart definitions and metric key types derive from the registry.
  4. Add subtitle/disclaimer copy in ChartDisplay.tsx if the metric depends on assumed constants.
  5. Add the disaggregated-config caveat banner for per-GPU or per-MW metrics.
  6. Verify both official rows and ?unofficialrun= overlays, including clipping, CSV/table behavior where applicable, zoom, labels, and Chinese UI.

Add a new blog post

  1. Create packages/app/content/blog/<slug>.mdx with frontmatter: title, subtitle, date (required), tags, modifiedDate (optional)
  2. Write content using Markdown + custom MDX components (Figure, Blur)
  3. Create the Simplified Chinese translation at packages/app/content/blog/zh/<slug>.mdx (required — see Chinese Website Pages)
  4. No code changes needed — the post automatically appears in the blog list, sitemap, RSS feed, llms.txt, and gets a generated OG image; the zh file appears on /zh/blog with hreflang pairing

See Blog for content format, available MDX components, and design details.

Modify blog components

  • Blog library (posts, headings, reading time): src/lib/blog.ts
  • Blog list page: src/app/blog/page.tsx
  • Blog post page: src/app/blog/[slug]/page.tsx
  • MDX components: src/components/blog/mdx-components.tsx
  • TOC sidebar: src/components/blog/blog-toc.tsx
  • OG image generation: src/app/blog/[slug]/og-image-render.tsx
  • RSS feed: src/app/feed.xml/route.ts
  • SEO constants: packages/constants/src/seo.ts

Add a new model or GPU

First ask for the PR / GitHub Actions run URL — see Adding Entities for the full workflow. Never ask other questions before getting the URL.

Adding a new tab

  1. Add the canonical route entry to packages/app/src/lib/dashboard-routes.ts, including navigation group, indexability, provider capabilities, locale mirroring, and share scopes.
  2. Add explicit English metadata in src/lib/tab-meta.ts and Chinese metadata, intro, and label entries in src/lib/tab-meta-zh.ts.
  3. Create the English and Chinese App Router pages. dashboard-routes.test.ts enforces structural parity.
  4. Add the tab content and any section-specific provider only when the route actually needs one; declare dashboard-wide provider needs in the registry.
  5. Use ChartLegend with variant="sidebar", sorted by HW_REGISTRY sort order, default expanded.
  6. Track interactive actions with the {tabname}_ analytics prefix.

Bumping dependencies

Workflow for a periodic dep bump. Branch: chore/bump-deps-YYYY-MM-DD. Commit each step separately so failures are easy to bisect.

  1. Bump versions: bun update --interactive --recursive. Review every proposed update across workspaces. Press l to select the latest version when an update must cross the declared semver range.
  2. Resolve install errors: follow Bun's reported remediation and keep package-manager settings in bunfig.toml only when the documented configuration requires them.
  3. Audit security: bun audit checks the installed packages recorded in bun.lock. For each finding, update the affected dependency with bun update or bun update --latest, then rerun the audit. Do not suppress an advisory without a documented reason.
  4. Fix lint/format: bun run lint:fix && bun run fmt:fix. New rules from oxlint version bumps may not have autofixers (e.g. require-unicode-regexp, unicorn/no-negated-condition) — fix manually. For mechanical bulk changes, delegate to a subagent and verify with bun run typecheck.
  5. Final check: bun run lint && bun run fmt && bun run typecheck && bun run security all pass. Pre-commit hook reruns these.

Subsystem Docs

Detailed design rationale (the "why" and "how", not the "what") lives in docs/:

  • Index — index of all docs MUST ALWAYS READ IN CASE OF RELEVANT INFORMATION
  • Architecture — Client-first design, route navigation, URL state, caching, color system
  • D3 Charts — 4-effect architecture, zoom refs, tooltip lifecycle
  • Data Pipeline — DB schema reasoning, ETL design, spline interpolation
  • Pitfalls — Token type bugs, schema evolution, stale closures, zoom loss
  • GPU Specs — Topology invariants, unit conventions, hardware gotchas
  • TCO Calculator — Interpolation, composite keys, cost matrix
  • Adding Entities — Checklists for adding models, GPUs, precisions, sequences, frameworks
  • Testing — Requirements, quality standards, pre-commit checklist
  • Data Transforms — BenchmarkRow → AggDataEntry → InferenceData pipeline, hardware key construction, derived metrics
  • State Ownership — Context provider state map, availability filtering cascade, comparison dates, URL params
  • Blog — MDX content system, SEO features, TOC sidebar, reading progress, analytics events

Claude AI Agents

@claude (.github/workflows/claude.yml)

Three jobs: a lightweight Haiku route classifier runs on any @claude mention in an issue/comment and emits a profile; its output gates implement or review. (The review job also triggers directly on PR open/sync, with no comment to route.)

  • @claude <anything> — route picks a profile (ui / code / docs / question / review) and, for implement profiles, a browser (playwright / chrome / none).
    • implement job (ui / code / docs / question): provisions only what's needed — dev server, Playwright browser, and Cypress binary install on demand only for browser/UI work, so docs/DB/backend/question tasks stay fast. ui gets focused browser verification plus the ?unofficialrun= overlay, then passes bun run test:e2e; the full suite remains covered by the merge checks. Creates claude/issue-{N}-* branches and can push.
    • review job (review profile, or any PR open/sync): a read-only, verifying review. It checks out the PR head, starts a local dev server backed by the real read-only DB, and uses the Playwright MCP on http://localhost:3000 to confirm the changed UI actually works (renders real data, interactions behave, no console errors). It does not re-run the test suite — typecheck/lint/test:unit and the fixtures-based e2e are already covered by the dedicated tests-*/lint workflows; the review reads their status and folds failures into the review as 🔴 BLOCKING — plus the static diff review (bugs, security, missing tests). Never edits or pushes. A review-phrased ask in any wording (e.g. "@claude take a look at this PR") routes here, not just the exact @claude review. Prompt: .github/claude/review-prompt.md.
  • Explicit overrides (skip the classifier): @claude review → review; @claude chrome → Chrome DevTools MCP; @claude frontend → full Playwright + dev server; @claude general (or lite) → lean no-browser. If the router guesses wrong, re-run with the override.
  • implement and review share a claude-<PR/issue number> concurrency group, so reviews and implementation on the same PR serialize instead of clobbering each other.

The model is set once via the workflow-level CLAUDE_MODEL env (claude-opus-4-8); the router uses CLAUDE_ROUTER_MODEL (claude-haiku-4-5).