Skip to content

collectivex.md

Latest commit

 

History

History
158 lines (138 loc) · 11 KB

collectivex.md

File metadata and controls

158 lines (138 loc) · 11 KB
Raw

CollectiveX

Design rationale for the CollectiveX tab's data pipeline. Unlike every other tab (Neon DB → ETL ingest → /api/v1/*), CollectiveX uses lazy ingest-on-read: its database is a durable cache of GitHub Actions, populated by the API routes themselves.

Why lazy ingest instead of the main pipeline

  • Sweep artifacts expire after 14 days. The sweep workflow (collectivex-sweep.yml in the harness repo) uploads a matrix artifact (cxsweep-matrix-{run_id}) and per-cell result artifacts (cxshard-{cell}-{run_id}-{attempt}) with 14-day retention. Persisting on first view makes a run outlive its artifacts once anyone has looked at it.
  • The sweep JSON contract is expected to change. The DB stores the RAW documents verbatim; the shared reader (packages/db/src/collectivex/reader.ts) is the single transform point and runs at API-read time, so a reader fix retroactively applies to already-stored runs — no re-ingest. A contract change = reader change + a bump of the numeric version in the harness's experimental/CollectiveX/configs/sweep.json.
  • No CI plumbing. There is no ingest workflow, no cross-repo dispatch, and no GH secrets. Runs launched via gh api on any harness branch appear on the dashboard within the CDN TTL of someone viewing the page — only the workflow identity is checked, never the branch.

How it works

packages/app/src/lib/collectivex-lazy-ingest.ts exposes three ensure* functions the routes call before reading the DB (packages/db/src/queries/collectivex.ts):

  • ensureLatestCollectiveXRun — walk GitHub's completed sweep runs newest-first; stop at the first live requested-version run; persist it if absent.
  • ensureCollectiveXRunsList — progressively backfill every requested-version run whose 14-day artifacts may still be downloaded. Each request changes at most eight rows; an incomplete response is uncached and the client refetches until the workflow history is exhausted. Known rows do not consume the batch, so discovery advances past the newest runs. GitHub discovery is limited to runs created within the last 44 days: the 30-day workflow rerun window plus 14-day artifact retention, covering the oldest rerun whose artifacts can still exist without rescanning permanent workflow history on every cold-origin request.
  • ensureCollectiveXRun — fetch one run by id, or compare a stored run's run_attempt against GitHub and refresh it when a rerun is newer. Only completed runs are persisted.

Key invariants:

  • Writes are atomic and race-safe: one CTE statement with ON CONFLICT (run_id) DO NOTHING; concurrent first-viewers can't double-ingest or expose a partial run. A GitHub re-run (newer run_attempt) is replaced through a FOR UPDATE-guarded refresh statement.
  • Reads after lazy persistence use the primary: discovery state checks, writes, and each route's post-discovery query use DATABASE_COLLECTIVEX_WRITE_URL. This guarantees that a newly inserted or refreshed run is visible in the same request even when DATABASE_COLLECTIVEX_READONLY_URL points at a lagging replica. Ordinary read-only consumers can still use the read URL.
  • Deletion tombstones (cx_runs.deleted_at, documents freed): discovery must never resurrect a deleted run. Re-ingesting via the CLI (bun run admin:db:ingest:collectivex <run-url-or-id>) clears the tombstone — that CLI is the operator tool for pre-warming runs before artifact expiry, backfills, and un-deletes.
  • "Latest" orders by run_id (monotonic with run creation, matching the discovery walk) — not by completion time, where a long-failing older run would shadow a newer successful one.
  • Vendor scope is explicit: the shared reader accepts only amd and nvidia result shards (case-insensitive). Other vendor values are omitted from chart series, coverage, and SKU summaries; runs with no supported-vendor cases stay hidden.
  • GitHub being down never takes the page down: routes serve whatever the DB holds and only surface an error when there is no stored fallback.
  • Run-list completeness is progressive: /api/v1/collectivex/runs returns discovery_complete: false while another bounded ingest pass is required. Those responses use private, no-store; the client polls once per second until the field becomes true. Stored runs remain visible indefinitely, while never-ingested runs disappear with their upstream artifacts and can no longer be reconstructed.
  • Caching: responses carry the collectivex CDN tag with a 60s s-maxage (freshness bound for lazy discovery). Run deletion and POST /api/v1/invalidate?scope=collectivex purge only that tag; the main dashboard's blob cache is untouched by CollectiveX operations.
  • Env: DATABASE_COLLECTIVEX_READONLY_URL is the ordinary read-only connection and may point at a replica. DATABASE_COLLECTIVEX_WRITE_URL is the primary used by lazy persistence, its consistent reads, deletion, and migrations via bun run admin:db:migrate:collectivex. COLLECTIVEX_ADMIN_SECRET is the delete route Bearer token (deliberately not INVALIDATE_SECRET, since it is remembered in browser localStorage), and GITHUB_TOKEN authorizes artifact reads.

Multi-run explorer

The frontend loads every stored live run summary for the selected benchmark version and keeps refetching while recent GitHub history is still being ingested. The summary query has no arbitrary row cap and does not load artifact documents. Each table row has a visibility checkbox; checking a run fetches its cached dataset through /api/v1/collectivex/runs/[runId]. Checked datasets are combined client-side, and the EP, phase, kernel mode, precision, SKU, and backend controls filter their combined series.

Series ids are namespaced by GitHub Actions run id so the same matrix case from two runs remains independently toggleable. Configuration color stays consistent across runs; run identity is encoded by the active selection order: the first checked run is solid and each additional checked run gets the next non-repeating dash pattern. Removing a run compacts those style slots, so a lone remaining run is always solid. Active patterns appear in both the run table and legend, keeping run ids out of visible legend labels while retaining them in the legend item's accessible title. The newest run with measured cases is checked by default; newer incomplete sweeps remain listed but cannot blank the initial explorer. Deletion is available per row or as one confirmed action for all currently shown runs; both paths keep the same tombstone semantics described above. The Runs card's suite filter narrows the table to runs containing EP or KV cases; a run containing both appears under both filters, and changing the filter does not alter the checked-run selection. The bottom of the page carries the curated known-support matrix: every SKU × library pairing for throughput (normal) and low-latency kernels at EP8 and EP16, independent of which runs are checked. Green degrees are known to work on the fleet; red degrees are known NOT to work, each carrying a numbered note with the investigated reason (upstream issue references included, e.g. ROCm/mori#610); gray degrees do not exist for that pairing (vendor-mismatched library, a kernel with no such mode, or a pool the library was never brought up on). The table is maintained by hand in known-support.ts, mirroring the InferenceX repo's platform_config.json registry plus its wall investigations — when a registry row flips or a wall falls upstream, the cell and its note change here.

The raw-rows exception

CollectiveX routes return the assembled dataset (reader over stored matrix + docs) instead of raw rows. The reader is shared between the app and the CLI through the db package (@semianalysisai/inferencex-db/collectivex/*), so ingest-time validation and read-time assembly can never drift; shipping raw docs to the client would only move the same shared transform across the wire.

vLLM swap_blocks

English | 中文

Standalone backend=swap-blocks sweep runs are discovered through the same matrix/shard artifact path. Their execution-only matrix (include with one or more swap-blocks cells) maps to app contract version 1. The shared reader validates collectivex-swap-blocks-v1 documents, including correctness, payload arithmetic, finite positive ordered latency percentiles, sample counts and the run's source SHA. EP/KV documents keep their existing reader path. No database migration is needed: raw JSON documents remain the source of truth.

The additive swap_blocks dataset field holds runtime provenance and measured points; swap_cases labels their run summaries separately from EP/KV. One verified artifact counts as one case; each measured (direction, layout, block bytes, block count, seed) is one point. Budget-excluded combinations are reported separately and are never plotted as measurements. The run-table suite filter includes swap_blocks; checked runs share the normal comparison selection and run dash styles. The chart selects direction, layout and latency percentile, plots block bytes in B/KiB/MiB/GiB, and keeps bandwidth linear with a zero baseline. Latency is logarithmic. Bandwidth is copied payload divided by host-observed latency, including submission and CUDA synchronization, counting payload once even for d2d. Both /collectivex and /zh/collectivex expose the controls and tooltips.

Bandwidth charts draw dashed nominal hardware rooflines for visible GPUs, including across checked comparison runs. Host transfers use 64 GB/s one-way PCIe 5.0 x16 on the registered x86 pools (including HGX B300's CPU uplink). GB200/GB300 use 225 GB/s per GPU per direction: the 900 GB/s bidirectional Grace C2C link is split across two GPUs. Same-GPU copies use half the GPU Specs HBM bandwidth because each payload byte is read and written. These are hardware references, not measured limits: warm-cache copies may exceed the HBM reference; host memory, placement and protocol overhead reduce host rates. Unknown SKUs get no assumed roofline. Rooflines disappear in latency mode or when all matching series are hidden; bandwidth stays linear and includes the reference in its domain.

References: H200 specifications, AMD MI355X specifications, Grace C2C ports per GPU, GB200 bidirectional C2C, HGX CPU uplink topology.

Multi-pool artifacts must carry runtime.sku matching a requested matrix cell. Older single-pool artifacts may omit it. Measurement identity includes SKU so equivalent grid points from different GPU pools remain independent.