Skip to content

fix(inference): official legend X works with overlay loaded + explicit hide/show affordances / 修复叠加层下图例移除按钮并明确隐藏/恢复操作提示 #597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Oseltamivir merged 2 commits into from
Jul 20, 2026
Merged

fix(inference): official legend X works with overlay loaded + explicit hide/show affordances / 修复叠加层下图例移除按钮并明确隐藏/恢复操作提示 #597

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dc6ef91
fix(inference): make official legend X work while an unofficial overl…
Oseltamivir Jul 20, 2026
6fcc13f
feat(legend): explicit hide/show affordances on legend rows
Oseltamivir Jul 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions packages/app/cypress/e2e/overlay-legend-remove.cy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/**
* Clicking an official SKU's legend "X" must remove that series even while an
* unofficial-run overlay is loaded.
*
* Regression: with an overlay active, the chart reads official visibility from
* `localOfficialOverride` (the unified overlay-mode selection), but the legend
* X routed straight to InferenceContext's `removeHwType`, which mutates
* `activeHwTypes` — a set the chart ignores in overlay mode. The click
* appeared to do nothing. The legend toggle already had the overlay-aware
* split (`unifiedToggle`); the X now shares it (`handleRemoveHwType`).
*/
import { unlockAgenticGate } from '../support/e2e';
import {
countVisible,
interceptOverlayRun,
OVERLAY_RUN_ID,
REAL_CONFIGS,
} from '../support/overlay-fixtures';

describe('Official legend X works while an unofficial overlay is loaded', () => {
before(() => {
interceptOverlayRun();
cy.visit(`/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90`, {
onBeforeLoad(win) {
win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now()));
unlockAgenticGate(win);
},
});
cy.wait('@unofficialRun');
cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1);
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(
'have.length',
REAL_CONFIGS.length,
);
});

it('shows official points and an official legend entry initially', () => {
cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => {
expect(countVisible($dots), 'visible official points').to.be.greaterThan(0);
});
cy.get('[data-testid="chart-legend"]').contains('B300').should('exist');
// Active row: the hover affordance is the "Hide" X with an explicit tooltip.
cy.get('[data-testid="chart-legend"] [role="button"][aria-label^="Hide"][aria-label*="B300"]')
.should('have.attr', 'title')
.and('match', /^Hide B300/u);
});

it('clicking the official SKU X hides its points but keeps the overlay', () => {
// The X only becomes opaque on row hover (CSS group-hover), which Cypress
// events don't trigger — force the click on the always-present element.
// Target the OFFICIAL row's X: the overlay run row is listed first and has
// its own (no-op) X, so `.first()` would hit the wrong one. The official
// label is "B300 (SGLang)" — case-sensitive match excludes the overlay
// row's lowercase branch name.
cy.get('[data-testid="chart-legend"] [role="button"][aria-label^="Hide"][aria-label*="B300"]')
.first()
.click({ force: true });

// Every official point belongs to the removed B300 series → all hidden.
cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => {
expect(countVisible($dots), 'visible official points after remove').to.eq(0);
});
// The overlay series is untouched (Optimal Only default keeps 4 of 5).
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => {
expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length - 1);
});
// Inactive row: the hover affordance flips to the "+" restore indicator
// (explicit "clicking the name brings it back"), and the Hide X is gone.
cy.get('[data-testid="chart-legend"] [title^="Show B300"]').should('exist');
cy.get(
'[data-testid="chart-legend"] [role="button"][aria-label^="Hide"][aria-label*="B300"]',
).should('not.exist');
});

it('re-activating the SKU from the legend restores the official points', () => {
cy.get('[data-testid="chart-legend"]').contains('B300').click();
cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => {
expect(countVisible($dots), 'visible official points after re-add').to.be.greaterThan(0);
});
});
});
127 changes: 18 additions & 109 deletions packages/app/cypress/e2e/overlay-optimal-only.cy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,116 +2,25 @@
* Overlay (unofficial run) points must respect the "Optimal Only" toggle the
* same way official points do.
*
* Regression: with Optimal Only ON, official non-pareto points are hidden via
* `isPointVisible`, but overlay X markers rendered every point unconditionally.
* On the agentic interactivity chart this made an e2e-dominated config (TP8
* C=4 in the GLM5.2 B300 hicache run) look like a pareto point: its X marker
* stayed visible sitting on the dashed roofline (the monotone spline between
* C=8 and C=2 passes within ~0.5% of it) while the official twin was hidden.
*
* Fixture values are the real run-29682242847 numbers:
* conc, p90_intvty, tput_per_gpu, p90_e2el
* C=4 is dominated on e2e by C=8 (12874 tok/s @ 33.1s vs 9415 @ 48.0s), so
* with Optimal Only ON exactly 4 of the 5 overlay X's must stay visible.
* Regression: with Optimal Only ON (the default — `i_optimal !== '0'`),
* official non-pareto points are hidden via `isPointVisible`, but overlay X
* markers rendered every point unconditionally. On the agentic interactivity
* chart this made an e2e-dominated config (TP8 C=4 in the GLM5.2 B300 hicache
* run) look like a pareto point: its X marker stayed visible sitting on the
* dashed roofline (the monotone spline between C=8 and C=2 passes within
* ~0.5% of it) while the official twin was hidden.
*/
import { unlockAgenticGate } from '../support/e2e';

const DEFAULT_MODEL_DB_KEY = 'dsv4';
const AGENTIC_DATE = '2026-07-19';
const OVERLAY_RUN_ID = '29682242847';
const OVERLAY_RUN_URL = `https://github.com/SemiAnalysisAI/InferenceX/actions/runs/${OVERLAY_RUN_ID}`;

const REAL_CONFIGS: [number, number, number, number][] = [
[48, 10.6, 17199, 126.9],
[8, 68.5, 12874, 33.1],
[4, 88.3, 9415, 48], // e2e-dominated by C=8 → NOT optimal
[2, 111.1, 5018, 30],
[1, 130.2, 2600, 25.8],
];

const metricsFor = (intvty: number, tput: number, e2el: number): Record<string, number> => ({
median_itl: 1 / (intvty * 1.2),
p90_itl: 1 / intvty,
p99_itl: 1 / (intvty * 0.8),
median_e2el: e2el * 0.8,
p90_e2el: e2el,
p99_e2el: e2el * 1.3,
median_ttft: 0.5,
p90_ttft: 1,
p99_ttft: 2,
tput_per_gpu: tput,
output_tput_per_gpu: tput * 0.3,
input_tput_per_gpu: tput * 0.7,
});

let idCursor = 900000;
const b300Rows = (runUrl: string | null) =>
REAL_CONFIGS.map(([conc, intvty, tput, e2el]) => ({
id: runUrl ? 0 : idCursor++,
hardware: 'b300',
framework: 'sglang',
model: DEFAULT_MODEL_DB_KEY,
precision: 'fp4',
spec_method: 'none',
disagg: false,
is_multinode: false,
prefill_tp: 8,
decode_tp: 8,
num_prefill_gpu: 8,
num_decode_gpu: 8,
isl: null,
osl: null,
conc,
offload_mode: 'on',
benchmark_type: 'agentic_traces',
image: 'sglang:test',
metrics: metricsFor(intvty, tput, e2el),
workers: null,
date: AGENTIC_DATE,
run_url: runUrl,
}));

const availability = [
{
model: DEFAULT_MODEL_DB_KEY,
isl: null,
osl: null,
precision: 'fp4',
hardware: 'b300',
framework: 'sglang',
spec_method: 'none',
disagg: false,
benchmark_type: 'agentic_traces',
date: AGENTIC_DATE,
},
];

const countVisible = ($els: JQuery<HTMLElement>): number =>
[...$els].filter((el) => getComputedStyle(el).opacity !== '0').length;
import {
countVisible,
interceptOverlayRun,
OVERLAY_RUN_ID,
REAL_CONFIGS,
} from '../support/overlay-fixtures';

describe('Overlay points respect Optimal Only (agentic interactivity)', () => {
before(() => {
cy.intercept('GET', '/api/v1/availability', { body: availability }).as('availability');
cy.intercept('GET', '/api/v1/benchmarks*', { body: b300Rows(null) }).as('benchmarks');
cy.intercept('GET', '/api/unofficial-run*', {
body: {
runInfos: [
{
id: OVERLAY_RUN_ID,
name: 'add-glm5.2-b300-agentic-hicache',
branch: 'add-glm5.2-b300-agentic-hicache',
sha: 'abc000',
createdAt: `${AGENTIC_DATE}T00:00:00Z`,
url: OVERLAY_RUN_URL,
conclusion: 'success',
status: 'completed',
isNonMainBranch: true,
},
],
benchmarks: b300Rows(OVERLAY_RUN_URL),
evaluations: [],
},
}).as('unofficialRun');
interceptOverlayRun();
cy.visit(`/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90`, {
onBeforeLoad(win) {
win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now()));
Expand All @@ -136,27 +45,27 @@ describe('Overlay points respect Optimal Only (agentic interactivity)', () => {
it('hides the e2e-dominated overlay point in the default Optimal Only view', () => {
cy.get('#scatter-hide-non-optimal').should('have.attr', 'data-state', 'checked');
// Official parity check: 4 of 5 official dots visible.
cy.get('[data-testid="inference-chart-display"] svg .dot-group').then(($dots) => {
cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => {
expect(countVisible($dots), 'visible official points').to.eq(REAL_CONFIGS.length - 1);
});
// The overlay must hide its C=4 too — 4 of 5 X markers visible.
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').then(($pts) => {
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => {
expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length - 1);
});
});

it('shows all overlay points when Optimal Only is turned off', () => {
cy.get('#scatter-hide-non-optimal').click();
cy.get('#scatter-hide-non-optimal').should('have.attr', 'data-state', 'unchecked');
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').then(($pts) => {
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => {
expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length);
});
});

it('re-hides the e2e-dominated overlay point when Optimal Only is re-enabled', () => {
cy.get('#scatter-hide-non-optimal').click();
cy.get('#scatter-hide-non-optimal').should('have.attr', 'data-state', 'checked');
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').then(($pts) => {
cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => {
expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length - 1);
});
});
Expand Down
108 changes: 108 additions & 0 deletions packages/app/cypress/support/overlay-fixtures.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
/**
* Shared fixtures for `?unofficialrun=` overlay e2e specs.
*
* The benchmark values are the real numbers from GitHub run 29682242847
* (GLM5.2 B300 agentic hicache, offload=on rows):
* conc, p90_intvty (tok/s/user), tput_per_gpu, p90_e2el (s)
* C=4 is dominated on e2e by C=8 (12874 tok/s @ 33.1s vs 9415 @ 48.0s), which
* makes the set a ready-made probe for the e2e-restricted frontier behaviors.
*/
export const DEFAULT_MODEL_DB_KEY = 'dsv4';
export const AGENTIC_DATE = '2026-07-19';
export const OVERLAY_RUN_ID = '29682242847';
export const OVERLAY_RUN_BRANCH = 'add-glm5.2-b300-agentic-hicache';
export const OVERLAY_RUN_URL = `https://github.com/SemiAnalysisAI/InferenceX/actions/runs/${OVERLAY_RUN_ID}`;

export const REAL_CONFIGS: [number, number, number, number][] = [
[48, 10.6, 17199, 126.9],
[8, 68.5, 12874, 33.1],
[4, 88.3, 9415, 48], // e2e-dominated by C=8 → NOT optimal
[2, 111.1, 5018, 30],
[1, 130.2, 2600, 25.8],
];

export const metricsFor = (intvty: number, tput: number, e2el: number): Record<string, number> => ({
// intvty is ALWAYS derived as 1/itl by the agentic aliases — feed itl.
median_itl: 1 / (intvty * 1.2),
p90_itl: 1 / intvty,
p99_itl: 1 / (intvty * 0.8),
median_e2el: e2el * 0.8,
p90_e2el: e2el,
p99_e2el: e2el * 1.3,
median_ttft: 0.5,
p90_ttft: 1,
p99_ttft: 2,
tput_per_gpu: tput,
output_tput_per_gpu: tput * 0.3,
input_tput_per_gpu: tput * 0.7,
});

let idCursor = 900000;
export const b300Rows = (runUrl: string | null) =>
REAL_CONFIGS.map(([conc, intvty, tput, e2el]) => ({
id: runUrl ? 0 : idCursor++,
hardware: 'b300',
framework: 'sglang',
model: DEFAULT_MODEL_DB_KEY,
precision: 'fp4',
spec_method: 'none',
disagg: false,
is_multinode: false,
prefill_tp: 8,
decode_tp: 8,
num_prefill_gpu: 8,
num_decode_gpu: 8,
isl: null,
osl: null,
conc,
offload_mode: 'on',
benchmark_type: 'agentic_traces',
image: 'sglang:test',
metrics: metricsFor(intvty, tput, e2el),
workers: null,
date: AGENTIC_DATE,
run_url: runUrl,
}));

export const availability = [
{
model: DEFAULT_MODEL_DB_KEY,
isl: null,
osl: null,
precision: 'fp4',
hardware: 'b300',
framework: 'sglang',
spec_method: 'none',
disagg: false,
benchmark_type: 'agentic_traces',
date: AGENTIC_DATE,
},
];

/** Intercept availability + benchmarks + unofficial-run with the B300 fixture. */
export const interceptOverlayRun = () => {
cy.intercept('GET', '/api/v1/availability', { body: availability }).as('availability');
cy.intercept('GET', '/api/v1/benchmarks*', { body: b300Rows(null) }).as('benchmarks');
cy.intercept('GET', '/api/unofficial-run*', {
body: {
runInfos: [
{
id: OVERLAY_RUN_ID,
name: OVERLAY_RUN_BRANCH,
branch: OVERLAY_RUN_BRANCH,
sha: 'abc000',
createdAt: `${AGENTIC_DATE}T00:00:00Z`,
url: OVERLAY_RUN_URL,
conclusion: 'success',
status: 'completed',
isNonMainBranch: true,
},
],
benchmarks: b300Rows(OVERLAY_RUN_URL),
evaluations: [],
},
}).as('unofficialRun');
};

export const countVisible = ($els: JQuery<HTMLElement>): number =>
[...$els].filter((el) => getComputedStyle(el).opacity !== '0').length;
22 changes: 21 additions & 1 deletion packages/app/src/components/inference/ui/ScatterGraph.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -587,6 +587,26 @@ const ScatterGraph = React.memo(
[overlayData, unifiedToggle, toggleHwType],
);

// Legend "X" (remove) — same overlay split as handleToggleHwType. With an
// overlay loaded the chart reads localOfficialOverride, which the context's
// removeHwType (activeHwTypes) never touches, so routing the X through it
// left the official series visibly un-removed. Commit the removal through
// the unified selection instead; context state stays untouched so
// dismissing the overlay restores the pre-overlay official selection, same
// as the toggle path.
const handleRemoveHwType = useCallback(
(key: string) => {
if (!overlayData) {
removeHwType(key);
return;
}
const next = new Set(resolvedUnifiedSelection);
next.delete(key);
commitUnifiedSelection(next);
},
[overlayData, removeHwType, resolvedUnifiedSelection, commitUnifiedSelection],
);

// --- Theme ---
const hardwareConfig = hardwareConfigOverride || contextHardwareConfig;
const activeHwKeys = useMemo(() => {
Expand Down Expand Up @@ -2820,7 +2840,7 @@ const ScatterGraph = React.memo(
variant="sidebar"
onItemHover={handleLegendHover}
onItemHoverEnd={handleLegendHoverEnd}
onItemRemove={showAllHardwareTypes ? undefined : removeHwType}
onItemRemove={showAllHardwareTypes ? undefined : handleRemoveHwType}
legendItems={[
// Overlay legend: one entry per loaded unofficial run that actually
// contributes points to this chart. Colored from the shared palette
Expand Down
Loading
Loading