Skip to content
73 changes: 72 additions & 1 deletion specification/appendix-d-observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,77 @@ The following describes how keys in [flag metadata](types.md#flag-metadata) are
| ---------------------------- | ----------------------- | ----------------- | -------- | ------------------------------------------------------------------------------------------------ |
| `feature_flag.provider.name` | `name` | `Recommended` | `string` | The name of the provider as defined in the `provider metadata`, available in the `hook context`. |

## Telemetry Hook Implementation Guide

This section provides guidance for implementing observability hooks that emit OpenTelemetry signals during feature flag evaluations. The recommendations ensure consistency across SDK implementations while allowing for language-specific idioms.

### Signal Emission Patterns

Telemetry hooks can emit OpenTelemetry signals in three distinct ways:

| Pattern | Advantages | Disadvantages |
| ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Span Events**</br>![recommended](https://img.shields.io/badge/recommended-green) | - Leverages existing trace configuration and tooling</br> - Minimal overhead, no additional spans created</br> - Maintains trace context relationships</br> - Simpler than creating spans | - Requires an active span to function</br> - Must gracefully handle absence of active span</br> - Limited to span lifetime and context. |
| **Event Logging** | - Works independently without active spans</br> - Aligns with OpenTelemetry's emerging direction</br> - Suitable for environments without tracing</br> - Simpler implementation model | - Requires an event exporter to be configured</br> - Processed and stored separately from spans</br> - Event logging standards still evolving |
| **Standalone Spans** | - Distributed traces contain every evaluation</br> - Detailed timing information</br> - Full span lifecycle control | - Creates one span per evaluation</br> - May clutter trace visualizations</br> - Increased overhead and resource usage</br> - Potential performance impact at scale</br> - More complex implementation |

> [!NOTE]
> While span events are recommended for their low overhead and ease of use, OpenTelemetry is trending toward using log-based events instead of span events. Please refer to the [OpenTelemetry Span Event Deprecation Plan][otel-span-event-deprecation-plan] for more details.

### Hook Lifecycle Implementation

#### Before Stage

The `before` hook stage is primarily used by standalone span hooks to create and store spans. When creating spans, it's recommended to use the name `feature_flag.evaluation` and store them in hook data using a consistent, documented key for easy retrieval in later stages.

#### Error Stage

The `error` hook stage records exception information unless explicitly configured to exclude it. Implementations typically use [OpenTelemetry's standard exception][otel-record-error] recording semantics (`recordException` for spans, exception log events for event logging). Configuration options like `excludeExceptions` allow users to control this behavior based on their needs.

#### Finally Stage

The `finally` hook stage is where telemetry signals are emitted with complete evaluation details. This stage should include all required and conditionally required attributes as defined in the attribute mapping tables above. It's also responsible for proper resource cleanup (like ending spans or closing connections).

### Attribute Transformations

When building telemetry attributes, implementations should extract and map well-known fields from flag metadata to their corresponding event record attributes as defined in the Flag Metadata table above. Remember to transform enumeration values (like error codes and reasons) from OpenFeature's uppercase format to OpenTelemetry's lowercase snake_case convention.

### Value Handling and Privacy

Flag values can contain large or sensitive data, so implementations should provide configuration to control whether values are included in telemetry signals. It's the users' responsibility to manage this configuration. When values are included, they need to be serialized appropriately for OpenTelemetry.

Consider providing mechanisms to redact or obfuscate sensitive flag values, along with size limits to prevent telemetry bloat. This helps balance observability needs with privacy and performance concerns.

### Configuration Options

For consistency across implementations, consider supporting a common set of configuration options:

- `attributeMapper` (function): Custom function to add additional attributes to the signal
- `excludeAttributes` (list): List of attribute keys to exclude from the signal
- `excludeExceptions` (boolean): Whether to omit [exception details][otel-exception-details] from error signals
- `eventMutator` (function): Custom function to modify event attributes before sending

### Error Handling

Hooks should be designed to never throw exceptions that interrupt flag evaluation. Any internal errors can be logged at appropriate levels (debug/trace) without affecting application execution.

### Implementation Patterns

#### Common Base Class

In object-oriented languages, you might find it helpful to create a base hook class containing common functionality shared across all telemetry hook types. This typically includes:

- Shared configuration options
- Attribute building and transformation methods
- Enumeration format conversion
- Metadata extraction logic
- Logger instances for internal debugging

This pattern can reduce code duplication and ensure consistency across different hook implementations, though it's not required.

## History

Feature flags in the OpenTelemetry semantic conventions are currently in development and are marked as experimental.
Feature flags in the OpenTelemetry semantic conventions are currently in development and are marked as a release candidate.
The following table describes the history of changes to the OpenTelemetry feature flag event records as it progresses towards a stable release.

| Original Field Name | New Field Name | Semantic Convention Release |
Expand All @@ -71,3 +139,6 @@ The following table describes the history of changes to the OpenTelemetry featur
[^3]: Include `error.type` and `error.message`, if and only if an error occurred during a flag evaluation.

[otel-ff-events]: https://opentelemetry.io/docs/specs/semconv/feature-flags/feature-flags-logs/
[otel-span-event-deprecation-plan]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4430-span-event-api-deprecation-plan.md
[otel-record-error]: https://opentelemetry.io/docs/specs/semconv/general/recording-errors/
[otel-exception-details]: https://opentelemetry.io/docs/specs/semconv/exceptions/
Loading