feat(mcp-core): Add snapshot inspection tools for LLM debugging#935
feat(mcp-core): Add snapshot inspection tools for LLM debugging#935NicoHinderling wants to merge 12 commits into
Conversation
Add getSnapshotDetails and getSnapshotImage methods to support upcoming MCP tools for inspecting preprod snapshot test failures. getSnapshotDetails fetches comparison metadata as JSON from the organizations preprodartifacts snapshots endpoint. getSnapshotImage fetches raw image binary from the projects preprodartifacts images endpoint, returning the blob and content-type. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add MCP tool that retrieves preprod snapshot comparison details, including changed/added/removed images, diff percentages, VCS info, and approval status. Parses snapshot URLs to extract org slug and snapshot ID automatically. Co-Authored-By: Claude <noreply@anthropic.com>
Add a new MCP tool that fetches before/after/diff images for a specific image in a preprod snapshot comparison. Supports changed, added, and removed view types, returning mixed TextContent and ImageContent arrays with base64-encoded images. Co-Authored-By: Claude <noreply@anthropic.com>
The preprod image endpoint uses /files/images/{id}/ not
/preprodartifacts/images/{id}/. Also avoid encoding the imageId
since diff image keys contain slashes.
The use_sentry handler tests had hardcoded tool counts that didn't account for the two new snapshot tools (get_snapshot_details and get_snapshot_image).
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
A couple of quick thoughts (i didnt read code yet):
|
|
do we need some kind of way to search for snapshots? not sure the workflows |
yes it's gated by their org token, so hopefully that's sufficient
hmm im not familiar with either so i'll have to look into that
right now we're only building out the flow where the LLM knows the exact snapshot in question (ex. CI failure) but yeah that could be a logical next step |
|
@NicoHinderling what do you mean by org token? the gates we have right now basically discover if the account is using a feature. you can prompt your agent to look for it and itll find it - but i believe we lock profiles and some other stuff behind that. its not critical if it uses shared tool definitions, but def valuable if we have no other choice but to expose more tools |
…SnapshotUrl fetchImage pushed label and image to a shared array with an await in between, allowing concurrent Promise.all calls to interleave content. Now returns parts array and flattens deterministically after all fetches complete. Extracts duplicated parseSnapshotUrl into shared url-helpers module.
Snapshot URLs are now auto-detected by parseSentryUrl and routed through get_sentry_resource, same as issues, traces, and replays. get_snapshot_details is marked internalOnly so it no longer appears as a standalone tool. get_snapshot_image remains standalone since it returns image content blocks.
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
New "preprod" skill (off by default) controls visibility of the get_snapshot_image tool. Customers opt in to see snapshot image inspection capabilities. Snapshot details remain accessible via get_sentry_resource for anyone with a snapshot URL.
Remove viewType parameter — the tool now searches all categories (changed, added, removed) to find the image by name and determines what to fetch automatically. Simplifies the interface to just snapshot URL + image name.
- Add renamed image support to get_snapshot_details and get_snapshot_image - Fix region prefix filtering in parseSnapshotUrl - Add non-image content type guard in fetchImage - Add "preprod" to knownSegments for path-based org extraction - Remove deprecated withApiErrorHandling usage - Delegate parseSnapshotUrl to parseSentryUrl instead of reimplementing - Fix misplaced JSDoc comment on parseSnapshotUrl - Regenerate toolDefinitions.json and skillDefinitions.json
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0b08058. Configure here.
| "unknown", | ||
| group: pair.head_image?.group || null, | ||
| })), | ||
| hint: "Use get_snapshot_image with snapshot_url and image_name to view actual images.", |
There was a problem hiding this comment.
Snapshot hint references tool from different disabled skill
Medium Severity
get_snapshot_details has skills: ["inspect"] and returns a hint telling the LLM to call get_snapshot_image, but get_snapshot_image is in the preprod skill which is disabled by default. Since inspect is enabled by default, users encountering snapshot URLs through get_sentry_resource will receive this hint pointing to a tool they can't access unless they've also enabled the preprod skill. The LLM would attempt a tool call that doesn't exist in its available tool set.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 0b08058. Configure here.
2 participants
Add MCP support for inspecting preprod snapshot test failures, so LLMs can debug failed CI snapshot tests from a URL.
How it works:
https://sentry.sentry.io/preprod/snapshots/231949/)get_sentry_resourcewith that URL — snapshot URLs are now auto-detected alongside issues, traces, replays, etc. Returns a JSON summary: which images changed (with diff percentages), which were added/removed, VCS context, and approval status.get_snapshot_image— the one new standalone tool. This fetches the actual before/after/diff PNGs and returns them asImageContentblocks that multimodal LLMs can see.Why
get_snapshot_imageis standalone:get_sentry_resourceauto-detects resources from a URL alone. But fetching a specific image requires additional parameters (imageName,viewType) that aren't in the URL — so it can't be folded intoget_sentry_resource.Changes:
get_sentry_resourcenow recognizes/preprod/snapshots/{id}/URLs and routes to snapshot details (no new tool exposed)get_snapshot_imageis the only new tool visible to the LLM (+1 tool count)get_snapshot_detailsexists as an internal-only handler behindget_sentry_resourceparseSnapshotUrllives in sharedurl-helpers.tsSentryApiServicefor the existing preprod endpointsTested e2e against real Sentry snapshot data.