Context
A performance or reliability regression shows up in more than one place, and each place tells a partial story. Sentry has the runtime symptom — error rate, span duration, the release it started on. Grafana has the fleet view. The MetaMask CI Health dashboard (DevLake) has the build-side picture: job duration, failure rate, queue time.
Today, correlating them is manual and therefore inconsistent. Someone notices a Sentry spike, guesses at a release window, eyeballs a dashboard, and writes a conclusion whose reasoning nobody can re-run. The conclusion is usually a claim about causation — "the regression came from X" — supported by two screenshots and an assertion that the timelines line up.
Goal
A skill that produces an attribution carrying its own falsifier, rather than a narrative. Given a suspected regression, it should establish:
- Which release/commit window the symptom actually starts in, per source.
- Whether the sources agree on that window — and say so explicitly when they do not.
- What would have to be true for the correlation to be coincidental, and whether that holds.
The last item is the point. Two metrics moving together across one deploy is not evidence; the skill should report how many prior deploys show the same co-movement.
Requirements
- Operates on queries, not screenshots — every figure must be reproducible from a query the reader can re-run.
- Names its sources explicitly. A number without a source and a time window is not usable.
- States the disagreement case. If Sentry places the onset in one release and CI data in another, that discrepancy is the finding.
- Agent-agnostic; no dependency on a particular chat surface.
Open — needs resolving before this can be built
Programmatic read access to the DevLake Grafana instance. The dashboard is session-cookie authenticated (grafana_session) and there is no confirmed API-token path or o11y-side equivalent yet. Options to check, roughly in order of preference:
- A Grafana service account token on the DevLake instance.
- Querying the underlying DevLake datastore directly, bypassing the dashboard.
- An existing internal o11y interface that already fronts this data.
Without one of these, the CI half of the correlation cannot be automated and the skill degrades to the Sentry half only.
Naming
regression-attribution.
Not telemetry-correlation, the first candidate: correlation is the technique, and a correlation is worth producing only if it attributes the regression to something. Skill names in this repo say what gets produced — a proof, an audit, a cost — and this one produces an attribution. The goal above is already written in those terms.
Deliberately not vendor-named, unlike the sibling sentry-grafana-correlation, which joins a single trace across Sentry and Grafana Tempo by trace_id. That skill is about those two stores and the split between them, so its vendor names are load-bearing. This one is about a regression's onset window and is expected to span more sources over time — it already spans three.
Relationship to sentry-grafana-correlation
Distinct skills, easily confused. That one answers "what happened in this one request, end to end". This one answers "which release did this regression start in, and is the co-movement coincidental". The first is a single-trace join; the second is a time-series argument across releases.
Context
A performance or reliability regression shows up in more than one place, and each place tells a partial story. Sentry has the runtime symptom — error rate, span duration, the release it started on. Grafana has the fleet view. The MetaMask CI Health dashboard (DevLake) has the build-side picture: job duration, failure rate, queue time.
Today, correlating them is manual and therefore inconsistent. Someone notices a Sentry spike, guesses at a release window, eyeballs a dashboard, and writes a conclusion whose reasoning nobody can re-run. The conclusion is usually a claim about causation — "the regression came from X" — supported by two screenshots and an assertion that the timelines line up.
Goal
A skill that produces an attribution carrying its own falsifier, rather than a narrative. Given a suspected regression, it should establish:
The last item is the point. Two metrics moving together across one deploy is not evidence; the skill should report how many prior deploys show the same co-movement.
Requirements
Open — needs resolving before this can be built
Programmatic read access to the DevLake Grafana instance. The dashboard is session-cookie authenticated (
grafana_session) and there is no confirmed API-token path or o11y-side equivalent yet. Options to check, roughly in order of preference:Without one of these, the CI half of the correlation cannot be automated and the skill degrades to the Sentry half only.
Naming
regression-attribution.Not
telemetry-correlation, the first candidate: correlation is the technique, and a correlation is worth producing only if it attributes the regression to something. Skill names in this repo say what gets produced — a proof, an audit, a cost — and this one produces an attribution. The goal above is already written in those terms.Deliberately not vendor-named, unlike the sibling
sentry-grafana-correlation, which joins a single trace across Sentry and Grafana Tempo bytrace_id. That skill is about those two stores and the split between them, so its vendor names are load-bearing. This one is about a regression's onset window and is expected to span more sources over time — it already spans three.Relationship to
sentry-grafana-correlationDistinct skills, easily confused. That one answers "what happened in this one request, end to end". This one answers "which release did this regression start in, and is the co-movement coincidental". The first is a single-trace join; the second is a time-series argument across releases.