What happens
Eval measurements write a portable local artifact (eval-measurements.jsonl) after each managed agent run, but remote score export is still planned in ADR 0087. When OTEL_EXPORTER_OTLP_* is set, agent spans already export live (ADR 0050); scores do not. Orgs that use MLflow, Phoenix, Jaeger, or another OTLP backend cannot see fitness / measurement results on the same remote trace without a vendor-specific Assessments API in core (which we rejected).
What should happen
When OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is set (same env as agent traces):
- Newly written measurement rows also export over OTLP as GenAI evaluation events (
gen_ai.evaluation.result) on the same trace_id, remote-parented to the scored span.
- Local
eval-measurements.jsonl remains the derived source of truth beside telemetry; OTLP export must not rewrite run-telemetry.jsonl.
- Export is fail-open: OTLP failures warn and never fail the measure / agent job.
- No vendor score adapters (
MLFLOW_*, Assessments REST, etc.) in core or managed workflows.
Vendor-specific UI surfaces (e.g. MLflow Assessments panels) can be a separate consumer that reads the OTLP evaluation event or the JSONL — out of scope for fullsend core.
Context
- Decision: ADR 0087
- Guide: Eval Measurements (still marks remote scores as planned)
- Related: ADR 0050 distributed tracing; dogfood already exports agent spans to MLflow via
OTEL_EXPORTER_OTLP_TRACES_*
What happens
Eval measurements write a portable local artifact (
eval-measurements.jsonl) after each managed agent run, but remote score export is still planned in ADR 0087. WhenOTEL_EXPORTER_OTLP_*is set, agent spans already export live (ADR 0050); scores do not. Orgs that use MLflow, Phoenix, Jaeger, or another OTLP backend cannot see fitness / measurement results on the same remote trace without a vendor-specific Assessments API in core (which we rejected).What should happen
When
OTEL_EXPORTER_OTLP_ENDPOINTorOTEL_EXPORTER_OTLP_TRACES_ENDPOINTis set (same env as agent traces):gen_ai.evaluation.result) on the sametrace_id, remote-parented to the scored span.eval-measurements.jsonlremains the derived source of truth beside telemetry; OTLP export must not rewriterun-telemetry.jsonl.MLFLOW_*, Assessments REST, etc.) in core or managed workflows.Vendor-specific UI surfaces (e.g. MLflow Assessments panels) can be a separate consumer that reads the OTLP evaluation event or the JSONL — out of scope for fullsend core.
Context
OTEL_EXPORTER_OTLP_TRACES_*