Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions agentops-cli/src/commands/dashboard.js
Original file line number Diff line number Diff line change
Expand Up @@ -561,6 +561,7 @@ function validateDashboardUx() {
schema_version_coverage: true,
exporter_failure_review: true,
pattern_drilldowns: true,
run_centric_ui: true,
empty_state_dashboards: emptyStateDashboards
},
errors
Expand Down
18 changes: 18 additions & 0 deletions agentops-cli/src/commands/product.js
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,24 @@ function productAudit(options = {}) {
checks.push(check('dashboard-drilldowns', links.ok, [`${links.checked_links} nav/data links checked`], links.errors));
const ux = validateDashboardUx();
checks.push(check('dashboard-operator-ux', ux.ok, ['Home, Runs, Replay, transcript, patterns, recommendations, and empty states checked'], ux.errors));
checks.push(check(
'run-centric-ui-contract',
ux.ok
&& ux.contracts?.run_centric_ui === true
&& fileIncludes('docs/agentops-architecture-product-audit.md', [
'Run-Centric UI',
'Session explorer as first screen',
'Trace waterfall through Run Replay',
'Ask AgentOps panel'
]),
[
'grafana/dashboards/v2/01-agentops-home.json',
'grafana/dashboards/v2/02-runs-explorer.json',
'grafana/dashboards/v2/03-run-replay.json',
'docs/agentops-architecture-product-audit.md'
],
ux.errors
));

checks.push(check(
'azure-ingest-privacy-plan',
Expand Down
1 change: 1 addition & 0 deletions agentops-cli/test/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -4331,6 +4331,7 @@ test('product audit proves the local AgentOps control-room contract', () => {
'evals-insights-recommendations',
'grafana-v2-pack',
'kql-library',
'run-centric-ui-contract',
'content-transcript-opt-in',
'first-run-loop',
'ask-agentops-response-flow'
Expand Down
19 changes: 10 additions & 9 deletions docs/agentops-architecture-product-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -1203,16 +1203,17 @@ Success criterion:
The default UI answers: what happened, why did it fail or cost money, what changed, what should I do next?
```

Required work:
Implemented:

- Session explorer as first screen.
- Trace waterfall.
- Policy/safety strip.
- Tool/MCP waterfall.
- Context/tokens panel.
- Recommendation panel.
- Eval/benchmark linkage.
- Ask AgentOps panel.
- Session explorer as first screen through the Home **Session Health** panel and the **Runs Explorer** dashboard.
- Trace waterfall through Run Replay's **Replay timeline** and `OpenTrace` drilldowns.
- Policy/safety strip through Run Replay's **Policy, privacy, tests, and GitHub outcome** panel plus the Safety, Privacy & Policy dashboard.
- Tool/MCP waterfall through Run Replay's **Agent, skill, and MCP lineage** panel plus the Tools & MCP Risk dashboard.
- Context/tokens panel through Run Replay's **Context and cache posture** panel and the Models, Cost & Tokens dashboard.
- Recommendation panel through Home **Recommended next actions**, Run Replay **Latest recommendation**, and Insights **Recommendation artifacts**.
- Eval/benchmark linkage through Evals & Quality scorecards, before/after comparison, artifact review, hidden checks, policy review, semantic checks, and promotion approvals.
- Ask AgentOps panel through Run Replay **Ask AgentOps context** and hosted Ask AgentOps action links.
- `agentops product audit` now includes `run-centric-ui-contract` so this UI shape remains a local release gate.

### 3. Agent Improvement Loop

Expand Down
Loading