Skip to content

COMPONENTS.md

Latest commit

 

History

History
546 lines (433 loc) · 27.8 KB

COMPONENTS.md

File metadata and controls

546 lines (433 loc) · 27.8 KB

TelemetryFlow Collector - Component Reference

  • Version: 1.1.8
  • OTEL Version: 0.147.0
  • Last Updated: March 2026

This document provides a comprehensive reference of all OpenTelemetry Collector components included in the TelemetryFlow Collector distribution.

Component Overview

graph TB
    subgraph Components["TelemetryFlow Collector Components"]
        subgraph EXT["Extensions (12)"]
            E1[Health Check]
            E2[Authentication]
            E3[Debugging]
        end

        subgraph RCV["Receivers (55+)"]
            R1[OTLP]
            R2[Prometheus]
            R3[Logs]
            R4[Infrastructure]
        end

        subgraph PRC["Processors (22+)"]
            P1[Batch]
            P2[Transform]
            P3[Filter]
            P4[Sampling]
        end

        subgraph EXP["Exporters (45+)"]
            X1[OTLP]
            X2[Prometheus]
            X3[Cloud]
            X4[APM]
        end

        subgraph CON["Connectors (8)"]
            C1[spanmetrics]
            C2[servicegraph]
            C3[routing]
        end
    end

    RCV --> PRC
    PRC --> EXP
    PRC --> CON
    CON --> EXP

    style EXT fill:#E1BEE7,stroke:#7B1FA2
    style RCV fill:#BBDEFB,stroke:#1976D2
    style PRC fill:#C8E6C9,stroke:#388E3C
    style EXP fill:#FFE0B2,stroke:#F57C00
    style CON fill:#B2EBF2,stroke:#0097A7
Loading

Component Data Flow

flowchart LR
    subgraph Sources["Data Sources"]
        S1[Applications]
        S2[Infrastructure]
        S3[Cloud Services]
    end

    subgraph Collector["TelemetryFlow Collector"]
        R[Receivers]
        P[Processors]
        E[Exporters]
        C[Connectors]
    end

    subgraph Backends["Observability Backends"]
        B1[Prometheus]
        B2[Jaeger/Tempo]
        B3[Loki]
        B4[Cloud APM]
    end

    S1 --> R
    S2 --> R
    S3 --> R
    R --> P
    P --> E
    P --> C
    C --> E
    E --> B1
    E --> B2
    E --> B3
    E --> B4

    style Collector fill:#E3F2FD,stroke:#1565C0
Loading
Category Count Purpose
Extensions 12 Authentication, health checks, debugging
Receivers 55+ Ingest telemetry data
Processors 22+ Transform and enrich data
Exporters 45+ Send data to backends
Connectors 8 Bridge pipelines, derive metrics

Extensions

Extensions provide auxiliary capabilities like health checks, authentication, and debugging.

Core Extensions

Extension Description Documentation
zpages zPages debugging interface Link
health_check HTTP health check endpoint Link
pprof Go pprof profiling endpoint Link
file_storage Persistent storage for queuing Link

Authentication Extensions

Extension Description Documentation
basicauth HTTP Basic authentication Link
bearertoken Bearer token authentication Link
headers_setter Add custom headers Link
oauth2client OAuth 2.0 client credentials Link
oidc OpenID Connect authentication Link
sigv4auth AWS Signature Version 4 Link

Receivers

Receivers collect telemetry data from various sources.

OTLP Receiver (Core)

The primary receiver for OpenTelemetry Protocol data. Supports all three signal types (traces, metrics, logs) over both gRPC and HTTP protocols.

Capabilities:

Feature gRPC (4317) HTTP (4318)
Protocol Buffers Yes Yes
JSON Encoding No Yes
Streaming Bidirectional Request/Response
TLS/mTLS Yes Yes
Compression gzip gzip

Basic Configuration:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
      http:
        endpoint: "0.0.0.0:4318"

Advanced Configuration with TLS:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
        max_recv_msg_size_mib: 4
        max_concurrent_streams: 100
        read_buffer_size: 524288
        write_buffer_size: 524288
        tls:
          cert_file: /etc/tfo-collector/certs/server.crt
          key_file: /etc/tfo-collector/certs/server.key
          client_ca_file: /etc/tfo-collector/certs/ca.crt
        keepalive:
          server_parameters:
            max_connection_idle: 15s
            max_connection_age: 30s
            max_connection_age_grace: 5s
            time: 10s
            timeout: 5s
      http:
        endpoint: "0.0.0.0:4318"
        max_request_body_size: 10485760
        cors:
          allowed_origins: ["*"]
          allowed_headers: ["*"]
          max_age: 7200
        tls:
          cert_file: /etc/tfo-collector/certs/server.crt
          key_file: /etc/tfo-collector/certs/server.key

HTTP API Endpoints:

TFO-Collector supports both standard OTLP v1 and TelemetryFlow Platform v2 endpoints:

Version Endpoint Signal Content-Type
v1 (Standard OTLP) /v1/traces Traces application/x-protobuf, application/json
v1 (Standard OTLP) /v1/metrics Metrics application/x-protobuf, application/json
v1 (Standard OTLP) /v1/logs Logs application/x-protobuf, application/json
v2 (TelemetryFlow) /v2/traces Traces application/x-protobuf, application/json
v2 (TelemetryFlow) /v2/metrics Metrics application/x-protobuf, application/json
v2 (TelemetryFlow) /v2/logs Logs application/x-protobuf, application/json

Note: The v1 endpoints follow the standard OpenTelemetry specification. The v2 endpoints are TelemetryFlow Platform-specific for enhanced features. Both versions use the same handlers and are functionally equivalent.

Trace Receivers (Legacy Formats)

Receiver Protocol Port Documentation
jaeger Thrift/gRPC 14250, 14268, 6831, 6832 Link
zipkin HTTP 9411 Link
opencensus gRPC 55678 Link
skywalking gRPC/HTTP 11800, 12800 Link

Metrics Receivers

Receiver Description Documentation
hostmetrics CPU, memory, disk, network metrics Link
prometheus Scrape Prometheus endpoints Link
statsd StatsD metrics Link
carbon Graphite Carbon metrics Link
collectd collectd metrics Link
influxdb InfluxDB line protocol Link

Log Receivers

Receiver Description Documentation
filelog Tail log files Link
journald Linux systemd journal Link
syslog RFC 3164/5424 syslog Link
tcplog TCP log ingestion Link
udplog UDP log ingestion Link
windowseventlog Windows Event Log Link
fluentforward Fluentd/Fluent Bit forward Link

Infrastructure Receivers

Receiver Description Documentation
docker_stats Docker container metrics Link
podman Podman container metrics Link
k8s_cluster Kubernetes cluster metrics Link
k8s_events Kubernetes events Link
k8sobjects Kubernetes objects Link
kubeletstats Kubelet stats Link

Database Receivers

Receiver Description Documentation
mysql MySQL metrics Link
postgresql PostgreSQL metrics Link
mongodb MongoDB metrics Link
redis Redis metrics Link
elasticsearch Elasticsearch metrics Link
sqlquery Custom SQL queries Link
oracledb Oracle Database metrics Link

Message Queue Receivers

Receiver Description Documentation
kafka Kafka consumer Link
kafkametrics Kafka broker metrics Link
rabbitmq RabbitMQ metrics Link
pulsar Apache Pulsar Link

Web Server Receivers

Receiver Description Documentation
nginx Nginx stub status Link
apache Apache mod_status Link
haproxy HAProxy stats Link
iis IIS metrics (Windows) Link

Cloud Receivers

Receiver Description Documentation
awscloudwatch AWS CloudWatch metrics Link
awscontainerinsight AWS Container Insights Link
awsecscontainermetrics AWS ECS metrics Link
awsxray AWS X-Ray traces Link
azureeventhub Azure Event Hub Link
azuremonitor Azure Monitor metrics Link
googlecloudpubsub Google Cloud Pub/Sub Link

Processors

Processors transform, filter, and enrich telemetry data.

Core Processors

Processor Description Documentation
batch Batch data for efficient export Link
memory_limiter Prevent OOM conditions Link

Attribute Processors

Processor Description Documentation
attributes Modify span/metric/log attributes Link
resource Modify resource attributes Link
resourcedetection Auto-detect cloud/host resources Link
k8sattributes Add Kubernetes metadata Link

Transform Processors

Processor Description Documentation
transform OTTL-based transformation Link
metricstransform Rename, aggregate metrics Link
span Rename spans, extract attributes Link
logstransform Transform logs Link

Filtering Processors

Processor Description Documentation
filter Filter telemetry by conditions Link
routing Route to different exporters Link
groupbyattrs Group by attributes Link
groupbytrace Wait for complete traces Link

Sampling Processors

Processor Description Documentation
tail_sampling Intelligent trace sampling Link
probabilistic_sampler Probabilistic sampling Link

Metrics Processors

Processor Description Documentation
cumulativetodelta Convert cumulative to delta Link
deltatocumulative Convert delta to cumulative Link
metricsgeneration Derive new metrics Link
interval Aggregate over time intervals Link

Security Processors

Processor Description Documentation
redaction Remove sensitive data (PII) Link

Exporters

Exporters send telemetry data to various backends.

Core OTLP Exporters

Exporter Description Documentation
otlp OTLP gRPC exporter Link
otlphttp OTLP HTTP exporter Link
debug Debug output (console) Link

OTLP gRPC Exporter Configuration:

exporters:
  otlp:
    endpoint: "otel-backend:4317"
    tls:
      insecure: false
      cert_file: /etc/tfo-collector/certs/client.crt
      key_file: /etc/tfo-collector/certs/client.key
      ca_file: /etc/tfo-collector/certs/ca.crt
    headers:
      Authorization: "Bearer ${API_TOKEN}"
    compression: gzip
    timeout: 30s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 5000

OTLP HTTP Exporter Configuration:

exporters:
  otlphttp:
    endpoint: "https://otel-backend:4318"
    tls:
      insecure: false
      cert_file: /etc/tfo-collector/certs/client.crt
      key_file: /etc/tfo-collector/certs/client.key
    headers:
      Authorization: "Bearer ${API_TOKEN}"
    compression: gzip
    timeout: 30s
    encoding: proto  # or "json"

OTLP Exporter Features:

Feature gRPC HTTP
Protocol Buffers Yes Yes
JSON Encoding No Yes
TLS/mTLS Yes Yes
Compression gzip gzip
Retry on Failure Yes Yes
Sending Queue Yes Yes
Custom Headers Yes Yes

Metrics Exporters

Exporter Description Documentation
prometheus Prometheus scrape endpoint Link
prometheusremotewrite Prometheus remote write Link
carbon Graphite Carbon Link
influxdb InfluxDB Link

Logs Exporters

Exporter Description Documentation
loki Grafana Loki Link
file Local file output Link

Database Exporters

Exporter Description Documentation
elasticsearch Elasticsearch Link
clickhouse ClickHouse analytics Link
cassandra Apache Cassandra Link

Message Queue Exporters

Exporter Description Documentation
kafka Apache Kafka Link
pulsar Apache Pulsar Link

Cloud Exporters

Exporter Description Documentation
awsemf AWS CloudWatch EMF Link
awsxray AWS X-Ray Link
awscloudwatchlogs AWS CloudWatch Logs Link
awskinesis AWS Kinesis Link
awss3 AWS S3 Link
googlecloud Google Cloud Monitoring Link
googlecloudpubsub Google Cloud Pub/Sub Link
azuremonitor Azure Monitor Link
azuredataexplorer Azure Data Explorer Link

APM Platform Exporters

Exporter Description Documentation
datadog Datadog APM Link
splunkhec Splunk HEC Link
signalfx Splunk SignalFx Link
dynatrace Dynatrace Link
coralogix Coralogix Link
logzio Logz.io Link
sumologic Sumo Logic Link
sentry Sentry Link

Legacy Trace Exporters

Exporter Description Documentation
zipkin Zipkin backend Link
opencensus OpenCensus backend Link
skywalking SkyWalking backend Link

Note: Jaeger exporter is deprecated. Use OTLP exporter to send traces to Jaeger instead.

Connectors

Connectors bridge pipelines and derive new telemetry.

Core Connectors

Connector Description Documentation
forward Forward data between pipelines Link
count Count signals Link

Exemplars & Service Graph Connectors

Connector Description Documentation
spanmetrics Derive metrics from traces with exemplars Link
servicegraph Build service dependency graphs Link
exceptions Extract exceptions from traces Link

Routing Connectors

Connector Description Documentation
routing Route data to different pipelines Link
failover Failover between pipelines Link

Adding New Components

To add a component not listed here:

  1. Check if it exists in OTEL Collector Contrib
  2. Add it to manifest.yaml: 
    receivers:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/newreceiver v0.114.0
  3. Rebuild with OCB: builder --config manifest.yaml
  4. Configure in your YAML config file

Related Documentation