Skip to content

fix: AI Conversations input/output correlation (0.2.11 + 0.2.12)#24

Merged
Joshkop merged 1 commit into
mainfrom
fix/ai-conversations-output-capture
May 21, 2026
Merged

fix: AI Conversations input/output correlation (0.2.11 + 0.2.12)#24
Joshkop merged 1 commit into
mainfrom
fix/ai-conversations-output-capture

Conversation

@Joshkop
Copy link
Copy Markdown
Owner

@Joshkop Joshkop commented May 21, 2026

Summary

Two related fixes to make Sentry AI Conversations show the right input/output pair per turn.

  • 0.2.11 (already on branch): Stop hook now closes the current turn so the transcript JSONL flush race doesn't drop gen_ai.output.messages; late-flush retry now triggers when recordOutputs is on and response is null; synthesized sessions refuse ordinal matches.
  • 0.2.12 (this commit): align transcript-reader's real-turn index with what UserPromptSubmit actually counts.

0.2.12 root cause

Claude Code's transcript records client-only slash commands (/model, /clear) as a synthetic user-line triple (<local-command-caveat> + <command-name> + <local-command-stdout>), but never fires UserPromptSubmit for them. The transcript reader was treating each of those lines as a real turn, so the collector's turnIndex (incremented per UserPromptSubmit) drifted off-by-N from transcript ordinals. Real UserPromptSubmit hook events don't carry prompt_id, so selectTurn fell back to ordinal — and the ordinal pointed to the wrong transcript turn.

Visible symptoms (conversation 9763be13-…, span 9d1cb6a39ecde5ba):

  • Early turns: gen_ai.input.messages set, no gen_ai.output.messages, claude_code.usage_extraction.status = turn_had_no_usage
  • Later turn: gen_ai.output.messages present but it was an earlier turn's response

Fix

  • src/transcript-reader.ts: pre-pass marks every prompt_id that has any <local-command-*> content as client-only; main loop skips isMeta:true lines, client-only-pid lines, and bare <local-command-*> lines.
  • src/server.ts + src/types.ts: accept camelCase promptId on UserPromptSubmit events as defense-in-depth alongside prompt_id.

Test plan

  • New failing test in tests/transcript-reader.test.ts reproduces the off-by-N with a /model synthetic triple followed by two real prompts; passes after the fix.
  • Full suite: 232 tests passing, tsc --noEmit clean.
  • Live verification: run a session that uses /model mid-conversation and confirm in Sentry AI Conversations that each turn's input matches its output.

🤖 Generated with Claude Code

Synthetic user-line triples for client-only slash commands (/model, /clear)
inflated transcript-reader's real-turn index, drifting off-by-N from the
collector's turnIndex. With UserPromptSubmit lacking prompt_id, ordinal
fallback returned the wrong turn — early turns lost outputs entirely, later
turns carried earlier turns' responses. Pre-pass now marks any prompt_id
with <local-command-*> content as client-only and skips those user lines
(plus isMeta:true). Also accept camelCase promptId as defense-in-depth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Joshkop Joshkop force-pushed the fix/ai-conversations-output-capture branch from 7192022 to e8f2529 Compare May 21, 2026 13:23
@Joshkop Joshkop merged commit b803d03 into main May 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant