Skip to content

feat(otel): add useLinksInsteadOfParent option to HatchetInstrumentor#3804

Open
purva-8 wants to merge 3 commits intohatchet-dev:mainfrom
purva-8:feat/otel-span-links-fire-and-forget
Open

feat(otel): add useLinksInsteadOfParent option to HatchetInstrumentor#3804
purva-8 wants to merge 3 commits intohatchet-dev:mainfrom
purva-8:feat/otel-span-links-fire-and-forget

Conversation

@purva-8
Copy link
Copy Markdown

@purva-8 purva-8 commented May 2, 2026

Closes #3641

Problem

HatchetInstrumentor currently forces every spawned run into a parent→child OTel relationship. For fire-and-forget patterns (runNoWait fan-outs, long-running children, rate-limited queues) this is wrong: the parent trace stays open until the slowest child finishes, which inflates p50/p95 metrics and pollutes the waterfall view.

OTel's span links are the right primitive here — the child starts a new root trace with a navigable link back to the spawning span, so the parent trace closes as soon as the spawn loop returns.

Solution

Add a useLinksInsteadOfParent predicate to HatchetInstrumentationConfig:

new HatchetInstrumentor({
  // Return true to use a span link instead of parent context for this task.
  useLinksInsteadOfParent: (actionId) => actionId.startsWith('fan-out-worker:'),
});

When the predicate returns true for a given actionId:

  • handleStartStepRun extracts the SpanContext from the incoming additionalMetadata (same as before)
  • Calls startActiveSpan(name, { kind, attributes, links: [{ context: parentSpanCtx }] }, fn)no parent context argument, so the child span becomes a root in a new trace with a link pointing back to the spawning span
  • If the extracted span context is invalid (e.g. no traceparent in metadata), the links array is empty and the span still starts cleanly as a root

When the predicate returns false (or is not provided):

  • Existing startActiveSpan(name, opts, parentContext, fn) path is used - zero behaviour change

Default: undefined → all spans use parent-child semantics, fully backwards compatible.

Changes

  • sdks/typescript/src/opentelemetry/instrumentor.ts - new config option + updated _patchHandleStartStepRun
  • sdks/typescript/src/opentelemetry/instrumentor.test.ts - 5 unit tests (default, predicate=false, predicate=true, actionId forwarding, invalid-context fallback)

Test plan

  • pnpm test:unit - all 195 tests pass (2 pre-existing skips)
  • tsc --noEmit - no type errors
  • Manual verification with a real Hatchet worker + an OTel backend (Jaeger / Honeycomb) to confirm child spans appear as separate root traces with a link

Fire-and-forget child runs (runNoWait fan-outs) should not extend their
parent trace's duration until the slowest child finishes. OTel span links
are the correct primitive: the child starts a new root trace with a
navigable link back to the spawning span.

Add a useLinksInsteadOfParent predicate to HatchetInstrumentationConfig.
When it returns true for a given actionId, handleStartStepRun starts the
step span without a parent context and instead attaches the extracted
SpanContext as a link. Default is undefined, preserving all existing
parent-child behaviour exactly.

Closes hatchet-dev#3641

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 2, 2026 14:02
@vercel
Copy link
Copy Markdown

vercel Bot commented May 2, 2026

@purva-8 is attempting to deploy a commit to the Hatchet Team on Vercel.

A member of the Team first needs to authorize it.

@promptless-for-oss
Copy link
Copy Markdown

Promptless prepared a documentation update related to this change.

Added the new useLinksInsteadOfParent option to the TypeScript OpenTelemetry configuration table, documenting how users can use span links instead of parent-child relationships for fire-and-forget patterns.

Review: Document useLinksInsteadOfParent option

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new OpenTelemetry configuration option to control whether hatchet.start_step_run spans should be parented to the spawning span (current behavior) or started as a new root trace with a span link back (for fire-and-forget fan-out patterns), addressing #3641.

Changes:

  • Introduces useLinksInsteadOfParent(actionId) => boolean on HatchetInstrumentor config.
  • Updates handleStartStepRun instrumentation to create span links (instead of a parent context) when the predicate returns true.
  • Adds Jest unit tests covering default behavior, predicate true/false, actionId forwarding, and invalid context fallback.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
sdks/typescript/src/opentelemetry/instrumentor.ts Adds config predicate and switches start_step_run to link-based trace semantics when enabled.
sdks/typescript/src/opentelemetry/instrumentor.test.ts New unit tests validating link vs parent-child behavior for start_step_run.

Comment thread sdks/typescript/src/opentelemetry/instrumentor.ts
Comment thread sdks/typescript/src/opentelemetry/instrumentor.test.ts
Comment thread sdks/typescript/src/opentelemetry/instrumentor.test.ts
purva-8 and others added 2 commits May 2, 2026 19:38
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT] OTel: option to use span links instead of parent context for fire-and-forget child runs

3 participants