diff --git a/develop-docs/sdk/telemetry/traces/modules/ai-agents.mdx b/develop-docs/sdk/telemetry/traces/modules/ai-agents.mdx index 8d3b1c46661e9c..29f1b1f5b4ac44 100644 --- a/develop-docs/sdk/telemetry/traces/modules/ai-agents.mdx +++ b/develop-docs/sdk/telemetry/traces/modules/ai-agents.mdx @@ -77,9 +77,9 @@ Additional attributes on the span: This span represents a request to an AI model or service that generates a response or requests a tool call based on the input prompt. -- Span `op` MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.chat"`) +- Span `op` MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.request"`) - Span `name` SHOULD be `{gen_ai.operation.name} {gen_ai.request.model}"`. (e.g. `"chat o3-mini"`) -- Attribute `gen_ai.operation.name` MUST be the operation performed.[3] (e.g. `"chat"`.) +- Attribute `gen_ai.operation.name` MUST be the operation performed.[3] (e.g. `"summarize"`.) - All [Common Span Attributes](#common-span-attributes) SHOULD be set (all `required` common attributes MUST be set). Additional attributes on the span: @@ -154,7 +154,7 @@ Some attributes are common to all AI Agents spans: | Attribute | Type | Requirement Level | Description | Example | | :---------------------- | :----- | :---------------- | :--------------------------------------------------------------------------------------- | :-------------------- | -| `gen_ai.operation.name` | string | required | The name of the operation being performed. **[2]** | `"chat"` | +| `gen_ai.operation.name` | string | required | The name of the operation being performed. **[2]** | `"summarize"` | | `gen_ai.request.model` | string | required | The name of the AI model a request is being made to. | `"gpt-4o"` | | `gen_ai.response.model` | string | optional | The name of the AI model the response was created with. | `"gpt-4o-2024-08-06"` | | `gen_ai.agent.name` | string | optional | The name of the agent this span belongs to. | `"Weather Agent"` | @@ -164,7 +164,7 @@ Some attributes are common to all AI Agents spans: | Value | Description | | :------------------- | :---------------------------------------------------------------------- | -| `"chat"` | Chat completion operation such as OpenAI Chat API | +| `"summarize"` | Chat completion operation such as OpenAI Chat API | | `"responses"` | OpenAI Responses API | | `"create_agent"` | Create GenAI agent | | `"embeddings"` | Embeddings operation such as OpenAI Create embeddings API | @@ -193,5 +193,7 @@ Some attributes are common to all AI Agents spans: | `"xai"` | xAI | ## Size Limits + In addition to the [general size limits](https://docs.sentry.io/concepts/data-management/size-limits/) for events, spans, and attachments, there are specific size limits for AI integrations: -- Input messages in AI integrations recorded in the span attribute `gen_ai.request.messages` are limited to 20kB per span. Messages are trimmed and truncated if they exceed the limit, retaining the most recent messages. \ No newline at end of file + +- Input messages in AI integrations recorded in the span attribute `gen_ai.request.messages` are limited to 20kB per span. Messages are trimmed and truncated if they exceed the limit, retaining the most recent messages. diff --git a/docs/platforms/javascript/common/configuration/integrations/langchain.mdx b/docs/platforms/javascript/common/configuration/integrations/langchain.mdx index dd13f5072ada84..4f4f713d73035a 100644 --- a/docs/platforms/javascript/common/configuration/integrations/langchain.mdx +++ b/docs/platforms/javascript/common/configuration/integrations/langchain.mdx @@ -144,7 +144,7 @@ Sentry.init({ By default this integration adds tracing support for LangChain operations including: -- **Chat model invocations** (`gen_ai.chat`) - Captures spans for chat model calls +- **Chat model invocations** (`gen_ai.request`) - Captures spans for chat model calls - **LLM invocations** (`gen_ai.pipeline`) - Captures spans for LLM pipeline executions - **Chain executions** (`gen_ai.invoke_agent`) - Captures spans for chain invocations - **Tool executions** (`gen_ai.execute_tool`) - Captures spans for tool calls diff --git a/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module-browser.mdx b/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module-browser.mdx index cd38cd73bf4704..5f7380a272a204 100644 --- a/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module-browser.mdx +++ b/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module-browser.mdx @@ -39,7 +39,7 @@ notSupported: With Sentry AI Agent Monitoring, you can monitor and debug your AI systems with full-stack context. You'll be able to track key insights like token usage, latency, tool usage, and error rates. AI Agent Monitoring data will be fully connected to your other Sentry data like logs, errors, and traces. -As a prerequisite to setting up AI Agent Monitoring in browser applications, you'll need to first set up tracing. +As a prerequisite to setting up AI Agent Monitoring in browser applications, you'll need to first set up tracing. @@ -108,7 +108,6 @@ Sentry.startSpan( name: `invoke_agent ${myAgent.name}`, attributes: { "gen_ai.operation.name": "invoke_agent", - "gen_ai.system": myAgent.modelProvider, "gen_ai.request.model": myAgent.model, "gen_ai.agent.name": myAgent.name, }, @@ -168,7 +167,6 @@ Sentry.startSpan( name: `chat ${myAi.model}`, attributes: { "gen_ai.operation.name": "chat", - "gen_ai.system": myAi.modelProvider, "gen_ai.request.model": myAi.model, }, }, @@ -253,7 +251,6 @@ if (result.toolCalls && result.toolCalls.length > 0) { op: "gen_ai.execute_tool", name: `execute_tool ${tool.name}`, attributes: { - "gen_ai.system": myAi.modelProvider, "gen_ai.request.model": myAi.model, "gen_ai.tool.type": "function", "gen_ai.tool.name": tool.name, @@ -322,7 +319,6 @@ if (result.handoffTo) { op: "gen_ai.handoff", name: `handoff from ${myAgent.name} to ${otherAgent.name}`, attributes: { - "gen_ai.system": myAgent.modelProvider, "gen_ai.request.model": myAgent.model, }, }, diff --git a/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module.mdx b/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module.mdx index 2b659d748ec314..90d4270460d883 100644 --- a/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module.mdx +++ b/docs/platforms/javascript/common/tracing/instrumentation/ai-agents-module.mdx @@ -36,152 +36,96 @@ The JavaScript SDK supports automatic instrumentation for some AI libraries. We - Vercel AI SDK - OpenAI -- Anthropic +- Anthropic - Google Gen AI SDK - LangChain - +- LangGraph + ## Manual Instrumentation If you're using a library that Sentry does not automatically instrument, you can manually instrument your code to capture spans. For your AI agents data to show up in the Sentry [AI Agents Insights](https://sentry.io/orgredirect/organizations/:orgslug/insights/ai/agents/), two spans must be created and have well-defined names and data attributes. See below. ## Spans -### Invoke Agent Span +### AI Request Span - - +This span represents a request to a LLM model or service that generates a response based on the input prompt. + + + -#### Example of an Invoke Agent Span: +#### Example AI Request Span ```javascript -// some example agent implementation for demonstration -const myAgent = { - name: "Weather Agent", - modelProvider: "openai", - model: "o3-mini", - async run() { - // Agent implementation - return { - output: "The weather in Paris is sunny", - usage: { - inputTokens: 15, - outputTokens: 8, - }, - }; - }, -}; +const messages = [{ role: "user", content: "Tell me a joke" }]; -Sentry.startSpan( +await Sentry.startSpan( { - op: "gen_ai.invoke_agent", - name: `invoke_agent ${myAgent.name}`, + op: "gen_ai.request", + name: "request o3-mini", attributes: { - "gen_ai.operation.name": "invoke_agent", - "gen_ai.system": myAgent.modelProvider, - "gen_ai.request.model": myAgent.model, - "gen_ai.agent.name": myAgent.name, + "gen_ai.request.model": "o3-mini", + "gen_ai.request.messages": JSON.stringify(messages), }, }, async (span) => { - // run the agent - const result = await myAgent.run(); - - // set agent response - // we assume result.output is a string - // type of `gen_ai.response.text` needs to be a string - span.setAttribute("gen_ai.response.text", JSON.stringify([result.output])); - - // set token usage - // we assume the result includes the tokens used - span.setAttribute("gen_ai.usage.input_tokens", result.usage.inputTokens); - span.setAttribute("gen_ai.usage.output_tokens", result.usage.outputTokens); + // Call your LLM here + const result = await client.chat.completions.create({ + model: "o3-mini", + messages, + }); - return result; + span.setAttribute( + "gen_ai.response.text", + JSON.stringify([result.choices[0].message.content]) + ); + // Set token usage + span.setAttribute("gen_ai.usage.input_tokens", result.usage.prompt_tokens); + span.setAttribute( + "gen_ai.usage.output_tokens", + result.usage.completion_tokens + ); } ); ``` -### AI Client Span +### Invoke Agent Span - - +This span represents the execution of an AI agent, capturing the full lifecycle from receiving a task to producing a final response. + + + -#### Example AI Client Span +#### Example Invoke Agent Span ```javascript -// some example implementation for demonstration -const myAi = { - modelProvider: "openai", - model: "o3-mini", - modelConfig: { - temperature: 0.1, - presencePenalty: 0.5, - }, - async createMessage(messages, maxTokens) { - // AI implementation - return { - output: - "Here's a joke: Why don't scientists trust atoms? Because they make up everything!", - usage: { - inputTokens: 12, - outputTokens: 24, - }, - }; - }, -}; - -Sentry.startSpan( +await Sentry.startSpan( { - op: "gen_ai.chat", - name: `chat ${myAi.model}`, + op: "gen_ai.invoke_agent", + name: "invoke_agent Weather Agent", attributes: { - "gen_ai.operation.name": "chat", - "gen_ai.system": myAi.modelProvider, - "gen_ai.request.model": myAi.model, + "gen_ai.request.model": "o3-mini", + "gen_ai.agent.name": "Weather Agent", }, }, async (span) => { - // set up messages for LLM - const maxTokens = 1024; - const prompt = "Tell me a joke"; - const messages = [{ role: "user", content: prompt }]; - - // set chat request data - span.setAttribute("gen_ai.request.messages", JSON.stringify(messages)); - span.setAttribute("gen_ai.request.max_tokens", maxTokens); - span.setAttribute( - "gen_ai.request.temperature", - myAi.modelConfig.temperature - ); - span.setAttribute( - "gen_ai.request.presence_penalty", - myAi.modelConfig.presencePenalty - ); - - // ask the LLM - const result = await myAi.createMessage(messages, maxTokens); + // Run the agent + const result = await myAgent.run(); - // set response - // we assume result.output is a string - // type of `gen_ai.response.text` needs to be a string span.setAttribute("gen_ai.response.text", JSON.stringify([result.output])); - - // set token usage - // we assume the result includes the tokens used + // Set token usage span.setAttribute("gen_ai.usage.input_tokens", result.usage.inputTokens); span.setAttribute("gen_ai.usage.output_tokens", result.usage.outputTokens); - - return result; } ); ``` ### Execute Tool Span +This span represents the execution of a tool or function that was requested by an AI model, including the input arguments and resulting output. + @@ -189,129 +133,48 @@ Sentry.startSpan( #### Example Execute Tool Span ```javascript -// some example implementation for demonstration -const myAi = { - modelProvider: "openai", - model: "o3-mini", - async createMessage(messages, maxTokens) { - // AI implementation that returns tool calls - return { - toolCalls: [ - { - name: "random_number", - description: "Generate a random number", - arguments: { max: 10 }, - }, - ], - }; +await Sentry.startSpan( + { + op: "gen_ai.execute_tool", + name: "execute_tool get_weather", + attributes: { + "gen_ai.tool.name": "get_weather", + "gen_ai.tool.input": JSON.stringify({ location: "Paris" }), + }, }, -}; - -const prompt = "Generate a random number between 0 and 10"; -const messages = [{ role: "user", content: prompt }]; + async (span) => { + // Call the tool + const result = await getWeather({ location: "Paris" }); -// First, make the AI call -const result = await Sentry.startSpan( - { op: "gen_ai.chat", name: `chat ${myAi.model}` }, - () => myAi.createMessage(messages, 1024) + span.setAttribute("gen_ai.tool.output", JSON.stringify(result)); + } ); - -// Check if we should call a tool -if (result.toolCalls && result.toolCalls.length > 0) { - const tool = result.toolCalls[0]; - - await Sentry.startSpan( - { - op: "gen_ai.execute_tool", - name: `execute_tool ${tool.name}`, - attributes: { - "gen_ai.system": myAi.modelProvider, - "gen_ai.request.model": myAi.model, - "gen_ai.tool.type": "function", - "gen_ai.tool.name": tool.name, - "gen_ai.tool.description": tool.description, - "gen_ai.tool.input": JSON.stringify(tool.arguments), - }, - }, - async (span) => { - // run tool (example implementation) - const toolResult = Math.floor(Math.random() * tool.arguments.max); - - // set tool result - span.setAttribute("gen_ai.tool.output", String(toolResult)); - - return toolResult; - } - ); -} ``` ### Handoff Span +This span marks the transition of control from one agent to another, typically when the current agent determines another agent is better suited to handle the task. + -#### Example of a Handoff Span +#### Example Handoff Span ```javascript -// some example agent implementation for demonstration -const myAgent = { - name: "Weather Agent", - modelProvider: "openai", - model: "o3-mini", - async run() { - // Agent implementation - return { - handoffTo: "Travel Agent", - output: - "I need to handoff to the travel agent for booking recommendations", - }; - }, -}; - -const otherAgent = { - name: "Travel Agent", - modelProvider: "openai", - model: "o3-mini", - async run() { - // Other agent implementation - return { output: "Here are some travel recommendations..." }; - }, -}; - -// First agent execution -const result = await Sentry.startSpan( - { op: "gen_ai.invoke_agent", name: `invoke_agent ${myAgent.name}` }, - () => myAgent.run() +await Sentry.startSpan( + { op: "gen_ai.handoff", name: "handoff from Weather Agent to Travel Agent" }, + () => {} // Handoff span just marks the transition ); -// Check if we should handoff to another agent -if (result.handoffTo) { - // Create handoff span - await Sentry.startSpan( - { - op: "gen_ai.handoff", - name: `handoff from ${myAgent.name} to ${otherAgent.name}`, - attributes: { - "gen_ai.system": myAgent.modelProvider, - "gen_ai.request.model": myAgent.model, - }, - }, - () => { - // the handoff span just marks the handoff - // no actual work is done here - } - ); - - // Execute the other agent - await Sentry.startSpan( - { op: "gen_ai.invoke_agent", name: `invoke_agent ${otherAgent.name}` }, - () => otherAgent.run() - ); -} +await Sentry.startSpan( + { op: "gen_ai.invoke_agent", name: "invoke_agent Travel Agent" }, + async () => { + // Run the target agent here + } +); ``` - - - +## Common Span Attributes + + diff --git a/docs/platforms/javascript/common/tracing/span-metrics/examples.mdx b/docs/platforms/javascript/common/tracing/span-metrics/examples.mdx index 7dfb41385a7fb0..61e78d17318182 100644 --- a/docs/platforms/javascript/common/tracing/span-metrics/examples.mdx +++ b/docs/platforms/javascript/common/tracing/span-metrics/examples.mdx @@ -1,6 +1,6 @@ --- title: Example Instrumentation -description: +description: Examples of using span metrics to debug performance issues and monitor application behavior across frontend and backend services. sidebar_order: 10 @@ -55,7 +55,7 @@ Sentry.startSpan( span.setAttribute('order.id', data.orderId) span.setAttribute('payment.provider', data.paymentProvider) Sentry.logger.info(Sentry.logger.fmt`Order ${data.orderId} confirmed via ${data.paymentProvider}`) - + // Show order confirmation setOrderConfirmation({ orderId: data.orderId, @@ -77,6 +77,7 @@ Sentry.startSpan( ``` Where to put this in your app: + - In the `onClick` for the checkout button, or inside the submit handler of your checkout form/container component. - Auto-instrumentation will add client `fetch` spans; keep the explicit UI span for specific application context. @@ -163,10 +164,12 @@ app.post('/api/checkout', async (req: Request, res: Response) => { ``` **How the trace works together:** + - UI span starts when checkout is selected -> Server Backend starts a span to continue the track when the server `/checkout` API is called. As payment processes, a payment span is started. - Attributes and Span metrics let you track more than just the latency of the request. You can also track the store's business performance through `cart.item_count` and other `cart` attributes, and store reliability by checking error performance on `payment.provider` properties. **What to monitor with span metrics:** + - p95 span.duration of `op:ui.action` checkout by `cart.item_count` bucket. - Error rate for `op:payment` by `payment.provider`. @@ -188,28 +191,28 @@ const handleUpload = async () => { // Start Sentry span for entire upload operation await Sentry.startSpan( { - name: 'Upload media', - op: 'file.upload', + name: "Upload media", + op: "file.upload", attributes: { - 'file.size_bytes': selectedFile.size, - 'file.mime_type': selectedFile.type, - } + "file.size_bytes": selectedFile.size, + "file.mime_type": selectedFile.type, + }, }, async (span) => { const uploadStartTime = Date.now(); - + try { // Single API call to upload and start processing const uploadResponse = await fetch(`${API_BASE_URL}/api/upload`, { - method: 'POST', + method: "POST", headers: { - 'Content-Type': 'application/json', + "Content-Type": "application/json", }, body: JSON.stringify({ fileName: selectedFile.name, fileType: selectedFile.type, - fileSize: selectedFile.size - }) + fileSize: selectedFile.size, + }), }); if (!uploadResponse.ok) { @@ -217,20 +220,22 @@ const handleUpload = async () => { } const uploadData = await uploadResponse.json(); - + // Set success attributes - span.setAttribute('upload.success', true); - span.setAttribute('upload.duration_ms', Date.now() - uploadStartTime); - span.setAttribute('job.id', uploadData.jobId); - + span.setAttribute("upload.success", true); + span.setAttribute("upload.duration_ms", Date.now() - uploadStartTime); + span.setAttribute("job.id", uploadData.jobId); + // Update UI to show processing status - updateUploadStatus(uploadData.jobId, 'processing'); - + updateUploadStatus(uploadData.jobId, "processing"); } catch (error) { - span.setStatus({ code: 2, message: 'error' }); - span.setAttribute('upload.success', false); - span.setAttribute('upload.error', error instanceof Error ? error.message : 'Unknown error'); - setUploadStatus('error'); + span.setStatus({ code: 2, message: "error" }); + span.setAttribute("upload.success", false); + span.setAttribute( + "upload.error", + error instanceof Error ? error.message : "Unknown error" + ); + setUploadStatus("error"); Sentry.captureException(error); } } @@ -239,6 +244,7 @@ const handleUpload = async () => { ``` Where to put this in your app: + - In the upload button click handler or form submit handler - In drag-and-drop onDrop callback - Auto-instrumentation will capture fetch spans; the explicit span adds business context @@ -253,64 +259,69 @@ This example demonstrates proper queue instrumentation patterns. For more detail ```typescript // Import Sentry instrumentation first (required for v10) -import './instrument'; -import express from 'express'; -import * as Sentry from '@sentry/node'; +import "./instrument"; +import express from "express"; +import * as Sentry from "@sentry/node"; // POST /api/upload - Receive and validate upload, then enqueue for processing -app.post('/api/upload', async (req: Request<{}, {}, UploadRequest>, res: Response) => { - const { fileName, fileType, fileSize } = req.body; - - // Validate the upload - if (!fileName || !fileType || !fileSize) { - return res.status(400).json({ error: 'Missing required fields' }); - } +app.post( + "/api/upload", + async (req: Request<{}, {}, UploadRequest>, res: Response) => { + const { fileName, fileType, fileSize } = req.body; + + // Validate the upload + if (!fileName || !fileType || !fileSize) { + return res.status(400).json({ error: "Missing required fields" }); + } - if (fileSize > 50 * 1024 * 1024) { // 50MB limit - return res.status(400).json({ error: 'File too large (max 50MB)' }); - } + if (fileSize > 50 * 1024 * 1024) { + // 50MB limit + return res.status(400).json({ error: "File too large (max 50MB)" }); + } - // Create a job for processing - const job = createJob(fileName, fileType, fileSize); + // Create a job for processing + const job = createJob(fileName, fileType, fileSize); + + // Producer span: Enqueue media processing job + await Sentry.startSpan( + { + op: "queue.publish", + name: "queue_producer", + attributes: { + "messaging.message.id": job.id, + "messaging.destination.name": "media-processing", + "messaging.message.body.size": fileSize, + }, + }, + async () => { + // Get trace headers to pass to consumer + const { "sentry-trace": sentryTrace, baggage: sentryBaggage } = + Sentry.getTraceData(); + + // Store job with trace headers for async processing + const enrichedJob = { + ...job, + sentryTrace, + sentryBaggage, + enqueuedAt: Date.now(), + }; + await enqueueJob(enrichedJob); + + // Start async processing + setImmediate(async () => { + await processMedia(enrichedJob); + }); - // Producer span: Enqueue media processing job - await Sentry.startSpan( - { - op: 'queue.publish', - name: 'queue_producer', - attributes: { - 'messaging.message.id': job.id, - 'messaging.destination.name': 'media-processing', - 'messaging.message.body.size': fileSize, + // Respond immediately with job ID + res.json({ + jobId: job.id, + status: "accepted", + message: "Upload received and processing started", + }); } - }, - async () => { - // Get trace headers to pass to consumer - const { 'sentry-trace': sentryTrace, baggage: sentryBaggage } = Sentry.getTraceData(); - - // Store job with trace headers for async processing - const enrichedJob = { - ...job, - sentryTrace, - sentryBaggage, - enqueuedAt: Date.now(), - }; - await enqueueJob(enrichedJob); - - // Start async processing - setImmediate(async () => { - await processMedia(enrichedJob); - }); - - // Respond immediately with job ID - res.json({ - jobId: job.id, - status: 'accepted', - message: 'Upload received and processing started' - }); - } - ); -}); + ); + } +); ``` **Backend - Async Media Processing (Consumer)** @@ -325,65 +336,80 @@ export async function processMedia(job: ProcessingJob): Promise { // Parent span for the consumer transaction await Sentry.startSpan( { - name: 'media_processing_consumer', + name: "media_processing_consumer", }, async (parentSpan) => { // Consumer span: Process the queued job await Sentry.startSpan( { - op: 'queue.process', - name: 'queue_consumer', + op: "queue.process", + name: "queue_consumer", attributes: { - 'messaging.message.id': job.id, - 'messaging.destination.name': 'media-processing', - 'messaging.message.body.size': job.fileSize, - 'messaging.message.receive.latency': Date.now() - job.enqueuedAt, - 'messaging.message.retry.count': 0, - } + "messaging.message.id": job.id, + "messaging.destination.name": "media-processing", + "messaging.message.body.size": job.fileSize, + "messaging.message.receive.latency": + Date.now() - job.enqueuedAt, + "messaging.message.retry.count": 0, + }, }, async (span) => { try { const startTime = Date.now(); const operations: string[] = []; - + // Add job-specific attributes - span.setAttribute('media.size_bytes', job.fileSize); - span.setAttribute('media.mime_type', job.fileType); - span.setAttribute('media.size_bucket', getSizeBucket(job.fileSize)); - + span.setAttribute("media.size_bytes", job.fileSize); + span.setAttribute("media.mime_type", job.fileType); + span.setAttribute( + "media.size_bucket", + getSizeBucket(job.fileSize) + ); + // Simulate image optimization and thumbnail generation - if (job.fileType.startsWith('image/')) { + if (job.fileType.startsWith("image/")) { // Note: No separate spans for these operations - use attributes instead await optimizeImage(); // Simulated delay - operations.push('optimize'); - + operations.push("optimize"); + await generateThumbnail(); // Simulated delay - operations.push('thumbnail'); + operations.push("thumbnail"); } - + // Calculate results const sizeSaved = Math.floor(job.fileSize * 0.3); // 30% reduction const thumbnailCreated = Math.random() > 0.05; // 95% success rate - + // Rich attributes instead of multiple spans - span.setAttribute('processing.operations', JSON.stringify(operations)); - span.setAttribute('processing.optimization_level', 'high'); - span.setAttribute('processing.thumbnail_created', thumbnailCreated); - span.setAttribute('processing.duration_ms', Date.now() - startTime); - span.setAttribute('result.size_saved_bytes', sizeSaved); - span.setAttribute('result.size_reduction_percent', 30); - span.setAttribute('result.status', 'success'); - + span.setAttribute( + "processing.operations", + JSON.stringify(operations) + ); + span.setAttribute("processing.optimization_level", "high"); + span.setAttribute( + "processing.thumbnail_created", + thumbnailCreated + ); + span.setAttribute( + "processing.duration_ms", + Date.now() - startTime + ); + span.setAttribute("result.size_saved_bytes", sizeSaved); + span.setAttribute("result.size_reduction_percent", 30); + span.setAttribute("result.status", "success"); + // Update job status - job.status = 'completed'; - + job.status = "completed"; + // Mark parent span as successful - parentSpan.setStatus({ code: 1, message: 'ok' }); - + parentSpan.setStatus({ code: 1, message: "ok" }); } catch (error) { - span.setAttribute('result.status', 'failed'); - span.setAttribute('error.message', error instanceof Error ? error.message : 'Unknown error'); - parentSpan.setStatus({ code: 2, message: 'error' }); + span.setAttribute("result.status", "failed"); + span.setAttribute( + "error.message", + error instanceof Error ? error.message : "Unknown error" + ); + parentSpan.setStatus({ code: 2, message: "error" }); Sentry.captureException(error); } } @@ -396,6 +422,7 @@ export async function processMedia(job: ProcessingJob): Promise { ``` **How the trace works together:** + - The frontend span (`file.upload`) captures the entire user experience from file selection to server response. - The backend producer span (`queue.publish`) tracks job enqueueing with proper queue attributes. - The consumer span (`queue.process`) continues the trace using `continueTrace()` with trace headers stored in the job. @@ -404,6 +431,7 @@ export async function processMedia(job: ProcessingJob): Promise { - This pattern populates Sentry's Queues insights page for monitoring queue performance. **What to monitor with span metrics:** + - p95 upload duration by `file.size_bucket`. - Processing success rate by `media.mime_type`. - Average storage saved via `result.size_saved_bytes` where `result.status = success`. @@ -423,11 +451,11 @@ Example Repository: [NullFlix](https://github.com/getsentry/nullflix-tracing-exa ```typescript const searchResults = await Sentry.startSpan( { - op: 'function', - name: 'Search autocomplete request', + op: "function", + name: "Search autocomplete request", attributes: { - 'query.length': searchQuery.length, - 'ui.debounce_ms': DEBOUNCE_MS, + "query.length": searchQuery.length, + "ui.debounce_ms": DEBOUNCE_MS, }, }, async (span) => { @@ -437,33 +465,37 @@ const searchResults = await Sentry.startSpan( `${API_URL}/api/search?${new URLSearchParams({ q: searchQuery })}`, { signal: controller.signal, - headers: { 'Content-Type': 'application/json' }, + headers: { "Content-Type": "application/json" }, } ); if (!response.ok) { const errorData = await response.json().catch(() => ({})); - const errorMessage = errorData.error || `Search failed: ${response.status}`; + const errorMessage = + errorData.error || `Search failed: ${response.status}`; throw new Error(errorMessage); } const data: SearchResponse = await response.json(); - - span.setAttribute('results.count', data.results.length); - span.setAttribute('results.has_results', data.results.length > 0); - span.setAttribute('http.response_size', JSON.stringify(data).length); - span.setStatus({ code: 1, message: 'ok' }); - + + span.setAttribute("results.count", data.results.length); + span.setAttribute("results.has_results", data.results.length > 0); + span.setAttribute("http.response_size", JSON.stringify(data).length); + span.setStatus({ code: 1, message: "ok" }); + return data; } catch (error) { - if (error instanceof Error && error.name === 'AbortError') { - span.setAttribute('ui.aborted', true); - span.setStatus({ code: 2, message: 'cancelled' }); + if (error instanceof Error && error.name === "AbortError") { + span.setAttribute("ui.aborted", true); + span.setStatus({ code: 2, message: "cancelled" }); // Don't re-throw AbortError to avoid sending it to Sentry return { results: [] }; } - - span.setStatus({ code: 2, message: error instanceof Error ? error.message : 'unknown error' }); + + span.setStatus({ + code: 2, + message: error instanceof Error ? error.message : "unknown error", + }); throw error; } } @@ -471,61 +503,65 @@ const searchResults = await Sentry.startSpan( ``` Where to put this in your app: + - In your search input component, triggered after a debounce timeout **Backend (Node.js + Express) - instrument search with meaningful attributes:** ```typescript -app.get('/api/search', async (req: Request, res: Response) => { +app.get("/api/search", async (req: Request, res: Response) => { await Sentry.startSpan( { - name: 'Search', - op: 'search', + name: "Search", + op: "search", }, async (span) => { try { - const query = String(req.query.q || ''); + const query = String(req.query.q || ""); const queryLength = query.length; - + // Check if request was aborted - req.on('close', () => { + req.on("close", () => { if (!res.headersSent) { - span.setStatus({ code: 2, message: 'cancelled' }); - span.setAttribute('request.aborted', true); + span.setStatus({ code: 2, message: "cancelled" }); + span.setAttribute("request.aborted", true); } }); - + if (!query) { - span.setAttribute('results.count', 0); - span.setAttribute('search.engine', 'elasticsearch'); + span.setAttribute("results.count", 0); + span.setAttribute("search.engine", "elasticsearch"); return res.json({ results: [] }); } - + // Perform search const startSearch = Date.now(); const results = await searchMovies(query); const searchDuration = Date.now() - startSearch; - + // Set span attributes - span.setAttribute('search.engine', 'elasticsearch'); - span.setAttribute('search.mode', queryLength < 3 ? 'prefix' : 'fuzzy'); - span.setAttribute('results.count', results.length); - span.setAttribute('query.length', queryLength); - + span.setAttribute("search.engine", "elasticsearch"); + span.setAttribute("search.mode", queryLength < 3 ? "prefix" : "fuzzy"); + span.setAttribute("results.count", results.length); + span.setAttribute("query.length", queryLength); + // Track slow searches if (searchDuration > 500) { - span.setAttribute('performance.slow', true); - span.setAttribute('search.duration_ms', searchDuration); + span.setAttribute("performance.slow", true); + span.setAttribute("search.duration_ms", searchDuration); } - + return res.json({ results }); } catch (error: any) { - span.setStatus({ code: 2, message: error?.message || 'error' }); - span.setAttribute('error.type', (error as any)?.constructor?.name || 'Error'); - + span.setStatus({ code: 2, message: error?.message || "error" }); + span.setAttribute( + "error.type", + (error as any)?.constructor?.name || "Error" + ); + Sentry.captureException(error); if (!res.headersSent) { - return res.status(500).json({ error: 'Search failed' }); + return res.status(500).json({ error: "Search failed" }); } } } @@ -534,11 +570,13 @@ app.get('/api/search', async (req: Request, res: Response) => { ``` **How the trace works together:** + - The client span starts when the debounced search triggers, then tracks the full user-perceived latency. - The aborted requests are marked with `ui.aborted=true` and short duration, showing wasted work. - The server span shows search performance characteristics: mode (prefix vs fuzzy), results count, and slow queries. **What to monitor with span metrics:** + - p95 duration of `op:search` grouped by `query.length`. - Characteristics of slow searches via `op:search performance.slow:true`. - Compare prefix vs fuzzy via `op:search` grouped by `search.mode`. @@ -574,7 +612,7 @@ import { SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN } from '@sentry/core'; export default function CustomerSupportChat() { const [conversationHistory, setConversationHistory] = useState([]); const [sessionId, setSessionId] = useState(''); - + // Generate sessionId on client-side only to avoid hydration mismatch useEffect(() => { setSessionId(`session_${Date.now()}`); @@ -604,7 +642,7 @@ const handleSendMessage = async (userMessage: string) => { async (agentSpan) => { try { setIsLoading(true); - + // Call your backend AI agent endpoint const response = await fetch('/api/ai/chat', { method: 'POST', @@ -621,7 +659,7 @@ const handleSendMessage = async (userMessage: string) => { } const aiResponse = await response.json(); - + // Set response attributes agentSpan.setAttribute('gen_ai.response.text', aiResponse.message); agentSpan.setAttribute('gen_ai.response.id', aiResponse.responseId); @@ -629,16 +667,16 @@ const handleSendMessage = async (userMessage: string) => { agentSpan.setAttribute('gen_ai.usage.total_tokens', aiResponse.totalTokens); agentSpan.setAttribute('conversation.tools_used', aiResponse.toolsUsed?.length || 0); agentSpan.setAttribute('conversation.resolution_status', aiResponse.resolutionStatus); - + // Update UI with response setConversationHistory(prev => [ ...prev, { role: 'user', content: userMessage }, { role: 'assistant', content: aiResponse.message } ]); - + Sentry.logger.info(Sentry.logger.fmt`AI agent completed conversation turn ${conversationHistory.length + 1}`); - + } catch (error) { agentSpan.setStatus({ code: 2, message: 'internal_error' }); agentSpan.setAttribute('error.type', error instanceof Error ? error.constructor.name : 'UnknownError'); @@ -653,110 +691,140 @@ const handleSendMessage = async (userMessage: string) => { ``` Where to put this in your app: -- In the API that controls the chat handler responses in your application +- In the API that controls the chat handler responses in your application **Important:** Generate `sessionId` in `useEffect` to avoid hydration errors when using Server-Side Rendering (SSR). Using `Date.now()` or random values during component initialization will cause mismatches between server and client renders. **Backend - Custom LLM Integration with Tool Calls:** ```typescript -import { SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN } from '@sentry/core'; +import { SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN } from "@sentry/core"; // Express API route for custom AI agent -app.post('/api/ai/chat', async (req: Request, res: Response) => { +app.post("/api/ai/chat", async (req: Request, res: Response) => { const { message, sessionId, conversationHistory } = req.body; // Main agent invocation span (matches frontend) await Sentry.startSpan( { - name: 'invoke_agent Customer Support Agent', - op: 'gen_ai.invoke_agent', + name: "invoke_agent Customer Support Agent", + op: "gen_ai.invoke_agent", attributes: { - 'gen_ai.operation.name': 'invoke_agent', - 'gen_ai.agent.name': 'Customer Support Agent', - 'gen_ai.system': 'custom-llm', - 'gen_ai.request.model': 'custom-model-v2', - [SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: 'manual.ai.custom-llm', - 'conversation.session_id': sessionId, + "gen_ai.operation.name": "invoke_agent", + "gen_ai.agent.name": "Customer Support Agent", + "gen_ai.system": "custom-llm", + "gen_ai.request.model": "custom-model-v2", + [SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: "manual.ai.custom-llm", + "conversation.session_id": sessionId, }, }, async (agentSpan) => { try { const tools = [ - { name: 'search_knowledge_base', description: 'Search company knowledge base for answers' } + { + name: "search_knowledge_base", + description: "Search company knowledge base for answers", + }, ]; - - agentSpan.setAttribute('gen_ai.request.available_tools', JSON.stringify(tools)); - + + agentSpan.setAttribute( + "gen_ai.request.available_tools", + JSON.stringify(tools) + ); + let totalTokens = 0; let toolsUsed: string[] = []; - let finalResponse = ''; - + let finalResponse = ""; + // Step 1: Call custom LLM for initial reasoning const llmResponse = await Sentry.startSpan( { - name: 'chat custom-model-v2', - op: 'gen_ai.chat', + name: "chat custom-model-v2", + op: "gen_ai.request", attributes: { - 'gen_ai.operation.name': 'chat', - 'gen_ai.system': 'custom-llm', - 'gen_ai.request.model': 'custom-model-v2', - 'gen_ai.request.messages': JSON.stringify([ - { role: 'system', content: 'You are a customer support agent. Use tools when needed.' }, + "gen_ai.operation.name": "summarize", + "gen_ai.system": "custom-llm", + "gen_ai.request.model": "custom-model-v2", + "gen_ai.request.messages": JSON.stringify([ + { + role: "system", + content: + "You are a customer support agent. Use tools when needed.", + }, ...conversationHistory, - { role: 'user', content: message } + { role: "user", content: message }, ]), - 'gen_ai.request.temperature': 0.7, - 'gen_ai.request.max_tokens': 500, - [SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: 'manual.ai.custom-llm', + "gen_ai.request.temperature": 0.7, + "gen_ai.request.max_tokens": 500, + [SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: "manual.ai.custom-llm", }, }, async (llmSpan) => { const llmData = await callCustomLLM(message, conversationHistory); - + // Set LLM response attributes - llmSpan.setAttribute('gen_ai.response.text', llmData.choices[0].message.content || ''); - llmSpan.setAttribute('gen_ai.response.id', llmData.id); - llmSpan.setAttribute('gen_ai.response.model', llmData.model); - llmSpan.setAttribute('gen_ai.usage.input_tokens', llmData.usage.prompt_tokens); - llmSpan.setAttribute('gen_ai.usage.output_tokens', llmData.usage.completion_tokens); - llmSpan.setAttribute('gen_ai.usage.total_tokens', llmData.usage.total_tokens); - + llmSpan.setAttribute( + "gen_ai.response.text", + llmData.choices[0].message.content || "" + ); + llmSpan.setAttribute("gen_ai.response.id", llmData.id); + llmSpan.setAttribute("gen_ai.response.model", llmData.model); + llmSpan.setAttribute( + "gen_ai.usage.input_tokens", + llmData.usage.prompt_tokens + ); + llmSpan.setAttribute( + "gen_ai.usage.output_tokens", + llmData.usage.completion_tokens + ); + llmSpan.setAttribute( + "gen_ai.usage.total_tokens", + llmData.usage.total_tokens + ); + if (llmData.choices[0].message.tool_calls) { - llmSpan.setAttribute('gen_ai.response.tool_calls', JSON.stringify(llmData.choices[0].message.tool_calls)); + llmSpan.setAttribute( + "gen_ai.response.tool_calls", + JSON.stringify(llmData.choices[0].message.tool_calls) + ); } - + totalTokens += llmData.usage.total_tokens; return llmData; } ); - + // Step 2: Execute tool calls if present if (llmResponse.choices[0].message.tool_calls) { for (const toolCall of llmResponse.choices[0].message.tool_calls) { await Sentry.startSpan( { name: `execute_tool ${toolCall.function.name}`, - op: 'gen_ai.execute_tool', + op: "gen_ai.execute_tool", attributes: { - 'gen_ai.operation.name': 'execute_tool', - 'gen_ai.tool.name': toolCall.function.name, - 'gen_ai.tool.type': 'function', - 'gen_ai.tool.input': toolCall.function.arguments, - [SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: 'manual.ai.custom-llm', + "gen_ai.operation.name": "execute_tool", + "gen_ai.tool.name": toolCall.function.name, + "gen_ai.tool.type": "function", + "gen_ai.tool.input": toolCall.function.arguments, + [SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: "manual.ai.custom-llm", }, }, async (toolSpan) => { - const toolOutput = await searchKnowledgeBase(JSON.parse(toolCall.function.arguments).query); - - toolSpan.setAttribute('gen_ai.tool.output', toolOutput); - toolSpan.setAttribute('search.query', JSON.parse(toolCall.function.arguments).query); + const toolOutput = await searchKnowledgeBase( + JSON.parse(toolCall.function.arguments).query + ); + + toolSpan.setAttribute("gen_ai.tool.output", toolOutput); + toolSpan.setAttribute( + "search.query", + JSON.parse(toolCall.function.arguments).query + ); toolsUsed.push(toolCall.function.name); } ); } - + // Step 3: Synthesize final response from tool results const synthesis = await synthesizeResponse(llmResponse, toolsUsed); finalResponse = synthesis.message; @@ -765,16 +833,22 @@ app.post('/api/ai/chat', async (req: Request, res: Response) => { // No tools used - use original message content finalResponse = llmResponse.choices[0].message.content; } - + // Set final agent attributes - const resolutionStatus = toolsUsed.length > 0 ? 'resolved' : 'answered'; - - agentSpan.setAttribute('gen_ai.response.text', finalResponse); - agentSpan.setAttribute('gen_ai.response.id', llmResponse.id); - agentSpan.setAttribute('gen_ai.usage.total_tokens', totalTokens); - agentSpan.setAttribute('conversation.tools_used', JSON.stringify(toolsUsed)); - agentSpan.setAttribute('conversation.resolution_status', resolutionStatus); - + const resolutionStatus = toolsUsed.length > 0 ? "resolved" : "answered"; + + agentSpan.setAttribute("gen_ai.response.text", finalResponse); + agentSpan.setAttribute("gen_ai.response.id", llmResponse.id); + agentSpan.setAttribute("gen_ai.usage.total_tokens", totalTokens); + agentSpan.setAttribute( + "conversation.tools_used", + JSON.stringify(toolsUsed) + ); + agentSpan.setAttribute( + "conversation.resolution_status", + resolutionStatus + ); + res.json({ message: finalResponse, responseId: llmResponse.id, @@ -782,12 +856,14 @@ app.post('/api/ai/chat', async (req: Request, res: Response) => { toolsUsed, resolutionStatus, }); - } catch (error) { - agentSpan.setStatus({ code: 2, message: 'agent_invocation_failed' }); - agentSpan.setAttribute('error.type', error instanceof Error ? error.constructor.name : 'UnknownError'); + agentSpan.setStatus({ code: 2, message: "agent_invocation_failed" }); + agentSpan.setAttribute( + "error.type", + error instanceof Error ? error.constructor.name : "UnknownError" + ); Sentry.captureException(error); - res.status(500).json({ error: 'AI agent processing failed' }); + res.status(500).json({ error: "AI agent processing failed" }); } } ); @@ -800,32 +876,36 @@ async function searchKnowledgeBase(query: string): Promise { "Our return policy allows returns within 30 days of purchase.", "Refunds are processed within 5-7 business days after we receive the item.", "Items must be in original condition with tags attached.", - "Free return shipping is provided for defective items." + "Free return shipping is provided for defective items.", ]; - return results.join('\n'); + return results.join("\n"); } - -async function synthesizeResponse(llmResponse: any, toolsUsed: string[]): Promise { +async function synthesizeResponse( + llmResponse: any, + toolsUsed: string[] +): Promise { // Make final LLM call to synthesize tool results into response return { message: "Based on the information I found, here's your answer...", - usage: { total_tokens: 150 } + usage: { total_tokens: 150 }, }; } ``` **How the trace works together:** + - The frontend span (`gen_ai.invoke_agent`) captures the entire user interaction from message to response. - The backend agent span continues the trace with the same operation and agent name for correlation. -- The LLM spans (`gen_ai.chat`) track individual model calls with token usage and performance. +- The LLM spans (`gen_ai.request`) track individual model calls with token usage and performance. - The tool execution spans (`gen_ai.execute_tool`) monitor each tool call with input/output and timing. - Rich attributes enable monitoring of conversation quality, cost, and business outcomes. **What to monitor with span metrics:** + - p95 duration of `op:gen_ai.invoke_agent` grouped by `conversation.resolution_status`. - Token usage trends via `gen_ai.usage.total_tokens` by `gen_ai.request.model`. - Tool usage patterns via `op:gen_ai.execute_tool` grouped by `gen_ai.tool.name`. - Cost analysis via `conversation.cost_estimate_usd` aggregated by time period. - Agent effectiveness via `conversation.resolution_status` distribution. -- Error rates for each component: `op:gen_ai.chat`, `op:gen_ai.execute_tool`, `op:gen_ai.invoke_agent`. +- Error rates for each component: `op:gen_ai.request`, `op:gen_ai.execute_tool`, `op:gen_ai.invoke_agent`. diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx index 80cacf8a6a14f9..c3773626f5f6e0 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx @@ -29,202 +29,103 @@ The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instr ## Spans -### Invoke Agent Span +### AI Request span + +This span represents a request to a LLM model or service that generates a response based on the input prompt. - + + + -#### Example Invoke Agent Span: +#### Example AI Request span ```python import sentry_sdk -sentry_sdk.init(...) +messages = [{"role": "user", "content": "Tell me a joke"}] -# some example agent implementation for demonstration -my_agent = MyAgent( - name="Weather Agent", - model_provider="openai", - model="o3-mini", -) - -with sentry_sdk.start_span( - op="gen_ai.invoke_agent", - name=f"invoke_agent {my_agent.name}", -) as span: - # set data about LLM and agent +with sentry_sdk.start_span(op="gen_ai.request", name="chat o3-mini") as span: + span.set_data("gen_ai.request.model", "o3-mini") + span.set_data("gen_ai.request.messages", json.dumps(messages)) span.set_data("gen_ai.operation.name", "invoke_agent") - span.set_data("gen_ai.system", my_agent.model_provider) - span.set_data("gen_ai.request.model", my_agent.model) - span.set_data("gen_ai.agent.name", my_agent.name) - - # run the agent - result = my_agent.run() - # set agent response - # we assume result.output is a dictionary - # type of `gen_ai.response.text` needs to be a string - span.set_data("gen_ai.response.text", json.dumps([result.output])) + # Call your LLM here + result = client.chat.completions.create(model="o3-mini", messages=messages) - # set token usage - # we assume the result includes the tokens used - span.set_data("gen_ai.usage.input_tokens", result.usage.input_tokens) - span.set_data("gen_ai.usage.output_tokens", result.usage.output_tokens) + span.set_data("gen_ai.response.text", json.dumps([result.choices[0].message.content])) + # Set token usage + span.set_data("gen_ai.usage.input_tokens", result.usage.prompt_tokens) + span.set_data("gen_ai.usage.output_tokens", result.usage.completion_tokens) ``` -### AI Client Span +### Invoke Agent Span + +This span represents the execution of an AI agent, capturing the full lifecycle from receiving a task to producing a final response. - + + + -#### Example AI Client Span +#### Example of an Invoke Agent Span: ```python import sentry_sdk -sentry_sdk.init(...) - -# some example implementation for demonstration -my_ai = MyAI( - model_provider="openai", - model="o3-mini", - model_config={ - "temperature": 0.1, - "presence_penalty": 0.5, - } -) - -with sentry_sdk.start_span( - op="gen_ai.chat", - name=f"chat {my_ai.model}", -) as span: - # set data about LLM - span.set_data("gen_ai.operation.name", "chat") - span.set_data("gen_ai.system", my_ai.model_provider) - span.set_data("gen_ai.request.model", my_ai.model) - - # set up messages for LLM - max_tokens = 1024 - prompt = "Tell me a joke" - messages = [ - {"role": "user", "content": prompt}, - ] - - # set chat request data - span.set_data("gen_ai.request.message", json.dumbs(messages)) - span.set_data("gen_ai.request.max_tokens", max_tokens) - span.set_data( - "gen_ai.request.temperature", - my_ai.model_config.get("temperature"), - ) - span.set_data( - "gen_ai.request.presence_penalty", - my_ai.model_config.get("presence_penalty"), - ) - - # ask the LLM - result = my_ai.messages.create( - messages=messages, - max_tokens=max_tokens, - ) - - # set response - # we assume result.output is a dictionary - # type of `gen_ai.response.text` needs to be a string - span.set_data("gen_ai.response.text", json.dumps([result.output])) - - # set token usage - # we assume the result includes the tokens used +with sentry_sdk.start_span(op="gen_ai.invoke_agent", name="invoke_agent Weather Agent") as span: + span.set_data("gen_ai.request.model", "o3-mini") + span.set_data("gen_ai.agent.name", "Weather Agent") + + # Run the agent + final_output = my_agent.run() + + span.set_data("gen_ai.response.text", str(final_output)) + # Set token usage span.set_data("gen_ai.usage.input_tokens", result.usage.input_tokens) span.set_data("gen_ai.usage.output_tokens", result.usage.output_tokens) ``` ### Execute Tool Span - +This span represents the execution of a tool or function that was requested by an AI model, including the input arguments and resulting output. + + + + #### Example Execute Tool Span ```python import sentry_sdk -sentry_sdk.init(...) - -# some example implementation for demonstration -my_ai = MyAI( - model_provider="openai", - model="o3-mini", -) - -with sentry_sdk.start_span(op="gen_ai.chat", ...): - # .. some code .. - result = my_ai.messages.create( - messages=messages, - max_tokens=1024, - ) - # .. some code .. - -# we assume the llm tells us to call a tool in the result -if my_should_call_tool(result): - # we parse the result to know which tool to call - tool = my_get_tool_to_call(result) - - with sentry_sdk.start_span( - op="gen_ai.execute_tool", - name=f"execute_tool {tool.name}" - ) as span: - # set data about LLM and tool - span.set_data("gen_ai.system", my_ai.model_provider) - span.set_data("gen_ai.request.model", my_ai.model) - span.set_data("gen_ai.tool.type", "function") - span.set_data("gen_ai.tool.name", tool.name) - span.set_data("gen_ai.tool.description", tool.description) - - # run tool - tool_result = my_run_tool(tool) - - # set tool result - span.set_data("gen_ai.tool.output", json.dumps(tool_result)) +with sentry_sdk.start_span(op="gen_ai.execute_tool", name="execute_tool get_weather") as span: + span.set_data("gen_ai.tool.name", "get_weather") + span.set_data("gen_ai.tool.input", json.dumps({"location": "Paris"})) + + # Call the tool + result = get_weather(location="Paris") + + span.set_data("gen_ai.tool.output", json.dumps(result)) ``` ### Handoff Span - +This span marks the transition of control from one agent to another, typically when the current agent determines another agent is better suited to handle the task. + + + + -#### Example Handoff Span +#### Example of a Handoff Span ```python import sentry_sdk -sentry_sdk.init(...) - -# some example agent implementation for demonstration -my_agent = MyAgent( - name="Weather Agent", - model_provider="openai", - model="o3-mini", -) - -with sentry_sdk.start_span(op="gen_ai.invoke_agent", ...): - # .. some code .. - result = my_agent.run() - # .. some code .. - - -# we assume the llm tells us to handoff to another agent -if my_should_handoff(result): - # we parse the result to know which agent to handoff to - other_agent = my_get_agent(result) - - with sentry_sdk.start_span( - op="gen_ai.handoff", - name=f"handoff from {my_agent.name} to {other_agent.name}", - ): - # the handoff span just marks the handoff - pass - - with sentry_sdk.start_span(op="gen_ai.invoke_agent", ...): - # .. some code .. - other_agent.run() - # .. some code .. +with sentry_sdk.start_span(op="gen_ai.handoff", name="handoff from Weather Agent to Travel Agent"): + pass # Handoff span just marks the transition + +with sentry_sdk.start_span(op="gen_ai.invoke_agent", name="invoke_agent Travel Agent"): + # Run the target agent here + pass ``` ## Common Span Attributes diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/index.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/index.mdx index 3005ea387e2f10..509d7af43d2ec6 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/index.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/index.mdx @@ -152,6 +152,7 @@ def chew(): def eat_slice(slice): chew() ``` + See the [@sentry_sdk.trace decoration section](#sentry_sdktrace-decorator) below for more details. ### Manually @@ -182,7 +183,6 @@ The parameters of `start_span()` and `start_child()` are the same. See the [API When you create your span manually, make sure to call `span.finish()` after the block of code you want to wrap in a span to finish the span. If you do not finish the span it will not be sent to Sentry. - ## @sentry_sdk.trace decorator You can set `op`, `name` and `attributes` parameters in the `@sentry_sdk.trace` decorator to customize your spans. Attribute values can only be primitive types (like `int`, `float`, `bool`, `str`) or a list of those types without mixing types. @@ -211,6 +211,7 @@ The parameters `op`, `name` and `attributes` were added to the `@sentry_sdk.trac ``` The code above will customize the `my_function` spans like this: + ```mermaid gantt dateFormat DD @@ -245,6 +246,7 @@ Available templates are `AI_AGENT`, `AI_TOOL`, and `AI_CHAT`. ``` This will treat `my_function` as an AI agent and will create the following span tree that is compatible with the [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans/) and the Sentry conventions for [AI Agents instrumentation](/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module/#spans). Depending on the template, there will also be a couple of attributes set by default, but those are omitted in the graph below for readability reasons: + ```mermaid gantt dateFormat DD @@ -257,8 +259,9 @@ gantt ``` For the span attributes that are set for the different available templates, see the AI Agents instrumentation documentation: + - `SPANTEMPLATE.AI_AGENT` -> [Invoke Agent Span](/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module/#invoke-agent-span) -- `SPANTEMPLATE.AI_CHAT` -> [AI Client Span](/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module/#ai-client-span) +- `SPANTEMPLATE.AI_CHAT` -> [AI Request span](/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module/#ai-client-span) - `SPANTEMPLATE.AI_TOOL` -> [Execute Tool Span](/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module/#execute-tool-span) Currently it is not possible to define custom span templates. @@ -380,6 +383,7 @@ sentry_sdk.init( ## Update Current Span Shortcut You can update the data of the currently running span using the `sentry_sdk.update_current_span()` function. You can set `op`, `name` and `attributes` to update your span. Attribute values can only be primitive types (like `int`, `float`, `bool`, `str`) or a list of those types without mixing types. + ```python {diff} import sentry_sdk @@ -401,6 +405,7 @@ You can update the data of the currently running span using the `sentry_sdk.upda ``` The code above will update the `my_function` (now `my_op`) spans with custom data like this: + ```mermaid gantt dateFormat DD diff --git a/includes/tracing/ai-agents-module/ai-client-span.mdx b/includes/tracing/ai-agents-module/ai-client-span.mdx index a24ea6b60c9472..ae33dfbc671e1e 100644 --- a/includes/tracing/ai-agents-module/ai-client-span.mdx +++ b/includes/tracing/ai-agents-module/ai-client-span.mdx @@ -1,10 +1,8 @@ -This span represents a request to an AI model or service that generates a response or requests a tool call based on the input prompt. - The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. -- The span `op` MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.chat"`) +- The span `op` MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.request"`) - The span `name` SHOULD be `{gen_ai.operation.name} {gen_ai.request.model}"`. (e.g. `"chat o3-mini"`) - All [Common Span Attributes](#common-span-attributes) SHOULD be set (all `required` common attributes MUST be set). diff --git a/includes/tracing/ai-agents-module/common-span-attributes.mdx b/includes/tracing/ai-agents-module/common-span-attributes.mdx index 50acaca662334f..d27be0b199887d 100644 --- a/includes/tracing/ai-agents-module/common-span-attributes.mdx +++ b/includes/tracing/ai-agents-module/common-span-attributes.mdx @@ -1,39 +1,7 @@ Some attributes are common to all AI Agents spans: -| Data Attribute | Type | Requirement Level | Description | Example | -| :---------------------- | :----- | :---------------- | :--------------------------------------------------------------------------------------- | :---------------- | -| `gen_ai.system` | string | required | The Generative AI product as identified by the client or server instrumentation. **[0]** | `"openai"` | -| `gen_ai.request.model` | string | required | The name of the AI model a request is being made to. | `"o3-mini"` | -| `gen_ai.operation.name` | string | optional | The name of the operation being performed. **[1]** | `"chat"` | -| `gen_ai.agent.name` | string | optional | The name of the agent this span belongs to. | `"Weather Agent"` | - -**[0]** Well-defined values for data attribute `gen_ai.system`: - -| Value | Description | -| :------------------ | :-------------------------------- | -| `"anthropic"` | Anthropic | -| `"aws.bedrock"` | AWS Bedrock | -| `"az.ai.inference"` | Azure AI Inference | -| `"az.ai.openai"` | Azure OpenAI | -| `"cohere"` | Cohere | -| `"deepseek"` | DeepSeek | -| `"gcp.gemini"` | Gemini | -| `"gcp.gen_ai"` | Any Google generative AI endpoint | -| `"gcp.vertex_ai"` | Vertex AI | -| `"groq"` | Groq | -| `"ibm.watsonx.ai"` | IBM Watsonx AI | -| `"mistral_ai"` | Mistral AI | -| `"openai"` | OpenAI | -| `"perplexity"` | Perplexity | -| `"xai"` | xAI | - -**[1]** Well-defined values for data attribute `gen_ai.operation.name`: - -| Value | Description | -| :------------------- | :---------------------------------------------------------------------- | -| `"chat"` | Chat completion operation such as OpenAI Chat API | -| `"create_agent"` | Create GenAI agent | -| `"embeddings"` | Embeddings operation such as OpenAI Create embeddings API | -| `"execute_tool"` | Execute a tool | -| `"generate_content"` | Multimodal content generation operation such as Gemini Generate Content | -| `"invoke_agent"` | Invoke GenAI agent | +| Data Attribute | Type | Requirement Level | Description | Example | +| :---------------------- | :----- | :---------------- | :--------------------------------------------------- | :---------------- | +| `gen_ai.request.model` | string | required | The name of the AI model a request is being made to. | `"o3-mini"` | +| `gen_ai.operation.name` | string | optional | The name of the operation being performed. | `"summarize"` | +| `gen_ai.agent.name` | string | optional | The name of the agent this span belongs to. | `"Weather Agent"` |