Skip to content

Commit e505a94

Browse files
test(node): Fix flaky Vercel AI integration test (#21076)
The `creates ai related spans with sendDefaultPii: false` test in the Vercel AI integration suite was looking up `firstGenerateContentSpan` via `find()` by name + `vercel.ai.operationId` + `GEN_AI_INPUT_MESSAGES_ATTRIBUTE === undefined`. With `sendDefaultPii: false`, both the first and the third `generate_content mock-model-id` spans have undefined input messages (the first because telemetry isn't explicitly enabled, the third — the tool-call generation — for the same reason). Depending on the order spans arrive in the container, `find()` could match the third span (which has `inputTokens: 15`) and the subsequent `expect(...INPUT_TOKENS).toBe(10)` would fail. Tightens the find query to also require `GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE === 10`, mirroring the disambiguation already used for `firstInvokeAgentSpan` in the same test. The same pattern is also present in `v5/test.ts` and `v6/test.ts` — fixed there as well. Fixes #20997 Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 2069779 commit e505a94

3 files changed

Lines changed: 6 additions & 3 deletions

File tree

dev-packages/node-integration-tests/suites/tracing/vercelai/test.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,8 @@ describe('Vercel AI integration', () => {
5858
span =>
5959
span.name === 'generate_content mock-model-id' &&
6060
span.attributes['vercel.ai.operationId'].value === 'ai.generateText.doGenerate' &&
61-
span.attributes[GEN_AI_INPUT_MESSAGES_ATTRIBUTE] === undefined,
61+
span.attributes[GEN_AI_INPUT_MESSAGES_ATTRIBUTE] === undefined &&
62+
span.attributes[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE].value === 10,
6263
);
6364
expect(firstGenerateContentSpan).toBeDefined();
6465
expect(firstGenerateContentSpan!.name).toBe('generate_content mock-model-id');

dev-packages/node-integration-tests/suites/tracing/vercelai/v5/test.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,8 @@ describe('Vercel AI integration (V5)', () => {
5858
span =>
5959
span.name === 'generate_content mock-model-id' &&
6060
span.attributes['vercel.ai.operationId'].value === 'ai.generateText.doGenerate' &&
61-
span.attributes[GEN_AI_INPUT_MESSAGES_ATTRIBUTE] === undefined,
61+
span.attributes[GEN_AI_INPUT_MESSAGES_ATTRIBUTE] === undefined &&
62+
span.attributes[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE].value === 10,
6263
);
6364
expect(firstGenerateContentSpan).toBeDefined();
6465
expect(firstGenerateContentSpan!.name).toBe('generate_content mock-model-id');

dev-packages/node-integration-tests/suites/tracing/vercelai/v6/test.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,8 @@ describe('Vercel AI integration (V6)', () => {
5959
span =>
6060
span.name === 'generate_content mock-model-id' &&
6161
span.attributes['vercel.ai.operationId'].value === 'ai.generateText.doGenerate' &&
62-
span.attributes[GEN_AI_INPUT_MESSAGES_ATTRIBUTE] === undefined,
62+
span.attributes[GEN_AI_INPUT_MESSAGES_ATTRIBUTE] === undefined &&
63+
span.attributes[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE].value === 10,
6364
);
6465
expect(firstGenerateContentSpan).toBeDefined();
6566
expect(firstGenerateContentSpan!.name).toBe('generate_content mock-model-id');

0 commit comments

Comments
 (0)