Skip to content

bug: OTLP trace export fails to start with gzip compression or an https:// collector endpoint #3899

Description

@krishicks

User Story

As a gateway operator sending traces to an OTLP collector, I want the documented exporter settings and TLS-protected collector endpoints to work, so that enabling compression or pointing at an https:// collector does not silently turn off tracing.

Problem Statement

The workspace builds opentelemetry-otlp 0.32 with only the grpc-tonic and trace features (Cargo.toml:73). Neither gzip-tonic nor any TLS feature (tls-ring, tls-aws-lc, tls-provider-agnostic) is enabled. As a result, building the span exporter fails when:

  • OTEL_EXPORTER_OTLP_COMPRESSION=gzip is set, even though docs/how-it-works/gateways/configuration.mdx:373 lists OTEL_EXPORTER_OTLP_COMPRESSION as supported exporter transport tuning.
  • [openshell.gateway.otlp].endpoint uses https://, which is the usual way to reach a collector outside the cluster or behind TLS.

When the exporter fails to build, the gateway logs an error and continues without trace export (crates/openshell-server/src/cli.rs:673-677). The exporter is built in the shared openshell-otel crate (crates/openshell-otel/src/lib.rs:197-200), which compute drivers also use for their own OTLP export.

Impact / Why This Matters

Operators who follow the documented tuning, or who point the gateway at a TLS collector, get no traces at all. The only signal is a single error line at startup; the gateway otherwise runs normally, so the loss is easy to miss. The workaround is to use a plaintext http:// endpoint without compression, which is not acceptable when the collector is reached over an untrusted network or only exposes a TLS receiver.

Acceptance Criteria

  • With OTEL_EXPORTER_OTLP_COMPRESSION=gzip, the gateway starts OTLP export and sends compressed spans.
  • With an https:// endpoint, the gateway starts OTLP export over TLS, verifying the collector against system or configured roots.
  • Compute drivers that export through openshell-otel accept the same settings.
  • The OTLP section of the gateway configuration reference matches the supported compression values and endpoint schemes; anything unsupported (for example zstd) is either enabled or documented as unsupported.

Reproduction Steps

  1. Write a gateway config:
    [openshell]
    version = 2
    
    [openshell.gateway.otlp]
    endpoint = "http://127.0.0.1:4317"
  2. Start the gateway with OTEL_EXPORTER_OTLP_COMPRESSION=gzip:
    OPENSHELL_GATEWAY_CONFIG=gateway.toml OTEL_EXPORTER_OTLP_COMPRESSION=gzip openshell-gateway --port 0 --disable-tls --db-url sqlite:gw.db
  3. Observe the OTLP startup error and that export is disabled.
  4. Change the endpoint to https://127.0.0.1:4317, start the gateway without the compression variable, and observe the same outcome.

Without either setting, the gateway logs OTLP exporting enabled endpoint=http://127.0.0.1:4317.

Environment

  • OpenShell: local dev build of main (opentelemetry-otlp features unchanged through 33a8eac)
  • OS: macOS (Darwin 25.6.0)
  • Runtime, deployment, or integration: openshell-gateway run directly with the Podman compute driver; applies to any deployment using [openshell.gateway.otlp]

Logs

ERROR openshell_server::cli: OTLP exporting is configured but could not be started; continuing without it error=failed to build the OTLP span exporter: feature 'gzip-tonic' is required to use the compression algorithm 'gzip'
ERROR openshell_server::cli: OTLP exporting is configured but could not be started; continuing without it error=failed to build the OTLP span exporter: endpoint: endpoint 'https://127.0.0.1:4317' uses HTTPS but no TLS feature is enabled; enable one of the `tls-ring`, `tls-aws-lc`, or `tls-provider-agnostic` features on `opentelemetry-otlp`

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:gatewayGateway server and control-plane worktopic:observabilityLogging, metrics, and observability work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions