feat: add benchmark diagnostics dashboard#604
Merged
Conversation
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds two new benchmarking API endpoints ( Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Memory diff vs base
Top 20 startup allocators
JTN-610 · backend=base:memray, pr:memray · informational only, does not block merge. Hard RSS budgets are enforced separately by JTN-608. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/static/scripts/settings/diagnostics.js`:
- Around line 349-405: refreshBenchmarks can complete out-of-order and overwrite
the UI for a newer benchmark window selection; modify refreshBenchmarks to
ignore stale responses by associating each invocation with a unique token (e.g.,
incrementing requestId or an AbortController) and checking that token before
applying results (use getBenchmarkWindow() at start to capture the intended
window and compare before render) or abort previous fetches when starting a new
call; apply the same pattern to the other similar function referenced around
lines 545-547 so only the latest request updates the DOM (reference
refreshBenchmarks, getBenchmarkWindow, renderPanel, setPanelFailure).
- Around line 307-311: The benchStages panel can hold a stale drill-down
selection when the refresh list changes; update the code that toggles benchmark
panels (e.g., setBenchmarkPanelsLoading) and the refresh-list update path (the
block around lines 380-398) to explicitly reset/clear the benchStages state
whenever the refresh list is reloaded or becomes empty—call the UI helper that
clears selections or resets panel content for "benchStages" (e.g.,
ui.clearPanel?, ui.resetPanel?, or ui.setPanelContent with an empty state) right
after updating benchSummary/benchPlugins/benchRefreshes so the stage drill-down
is cleared whenever the refresh window or rows change.
- Around line 329-340: The fetch handler that populates the stage panel treats
any JSON body as success; change the logic after calling
fetch(`/api/benchmarks/stages?...`) to check resp.ok and handle non-OK responses
by reading the JSON error (or resp.statusText) and displaying a clear failure
message in the same panel instead of calling buildStagesTable; specifically,
after const resp = await fetch(...) and const data = await resp.json(), if
(!resp.ok) set panel.textContent (or append an error div) to include the
returned error message/status and return early so
buildStagesTable(Array.isArray(data?.items) ? data.items : []) is not called for
error responses.
- Around line 353-388: The fetch results for summaryResp, pluginsResp, and
refreshesResp are treated as successful even when the server returns errors,
causing 404/500 pages to appear as an empty benchmark state; update the code
after the Promise.all to check each response.ok, parse any error JSON (or text)
from summaryResp, pluginsResp, and refreshesResp when not ok, and render an
error panel via renderPanel("benchSummary"/"benchPlugins"/"benchRefreshes") with
the extracted error message instead of collapsing into the empty-state path;
keep the existing logic that uses summary, plugins, refreshes when responses are
ok (variables summary, plugins, refreshes) and only fall back to the empty
message when all three are OK but contain no data.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: c4c1fcb5-2ea5-4594-8f52-8413cb3cb7b8
📒 Files selected for processing (13)
docs/benchmarking.mdsrc/blueprints/settings/_benchmarks.pysrc/schemas/endpoint_map.pysrc/schemas/responses.pysrc/static/openapi.jsonsrc/static/scripts/settings/diagnostics.jssrc/static/styles/main.csssrc/static/styles/partials/_settings-console.csssrc/templates/settings.htmltests/contract/test_frontend_api_contract_coverage.pytests/contract/test_response_shapes.pytests/static/test_benchmark_display.pytests/unit/test_settings_blueprint.py
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Base Branch Confirmation
origin/main(not a stale long-lived branch)origin/mainbefore openingParent-Fork Sync Checklist
fatihak/InkyPi, changes were cherry-picked by featureCompatibility/Release Checklist
pytestrelevant suites pass locallysuccess:false,error,code,details,request_id)src/static/**,src/templates/**): ran browser tests (SKIP_BROWSER=0 .venv/bin/python -m pytest tests/) and all passedTesting
PYTHONPATH=src pytest -q tests/unit/test_settings_blueprint.py::TestBenchmarkAPIs tests/contract/test_response_shapes.py::test_benchmarks_summary_shape tests/contract/test_response_shapes.py::test_benchmarks_plugins_shape tests/contract/test_response_shapes.py::test_benchmarks_refreshes_shape tests/contract/test_response_shapes.py::test_benchmarks_stages_shape tests/contract/test_frontend_api_contract_coverage.py tests/static/test_benchmark_display.py tests/integration/test_observability_api.pypython3 -m ruff check src/blueprints/settings/_benchmarks.py src/schemas/responses.py src/schemas/endpoint_map.py tests/contract/test_response_shapes.py tests/unit/test_settings_blueprint.py tests/static/test_benchmark_display.pypython3 -m black --check src/blueprints/settings/_benchmarks.py src/schemas/responses.py src/schemas/endpoint_map.py tests/contract/test_response_shapes.py tests/unit/test_settings_blueprint.py tests/static/test_benchmark_display.pyPYTHONPATH=src python3 -m mypy src/blueprints/settings/_benchmarks.py src/schemas/responses.py src/schemas/endpoint_map.pynode --check src/static/scripts/settings/diagnostics.jspython3 -m json.tool src/static/openapi.json >/tmp/inkypi-openapi-check.jsonpython3 scripts/build_css.pybash scripts/lint.shSummary by CodeRabbit
New Features
/api/benchmarks/refreshesand/api/benchmarks/stagesendpoints for querying benchmark data.Documentation
Bug Fixes