Skip to content

feat(observability): distributed tracing with OpenTelemetry #171

Description

@chitrank2050

Problem

When a usage event takes 5 seconds to process, where's the bottleneck? Is it Postgres? Kafka? Redis? The validation logic? Without tracing, debugging distributed systems is guesswork.

Proposed solution

Integrate OpenTelemetry SDK:

  1. Auto-instrument: HTTP (incoming + outgoing), Prisma/pg, Redis (ioredis), kafkajs
  2. Manual spans for business logic: outbox publish, validation checks, aggregation upsert, invoice generation
  3. Export to Jaeger (local dev) or OTLP collector (production)
  4. Trace context propagation: HTTP → Kafka message headers → consumer spans
  5. Correlation ID from request middleware linked to trace ID

API design (if applicable)

No API change. Traces exported to collector backend.

Alternatives considered

  • Datadog APM — rejected as the only option. Vendor lock-in. OpenTelemetry is vendor-neutral — export to Datadog, Jaeger, Grafana Tempo, or any OTLP-compatible backend.
  • Manual logging with timestamps — rejected. Not scalable. Can't visualize call graphs or identify parallelism.

Additional context

Add Jaeger to docker-compose.yml for local development. UI at http://localhost:16686.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions