Skip to content

refactor(trace/semantics): lift conditional gRPC status fallback into mappings.json#50397

Draft
edwardhu-datadog wants to merge 6 commits intofeat/ehu/OTEL-2718-otlp-grpc-status-trace-metricsfrom
feat/ehu/OTEL-2718-semantics-conditional-grpc-status
Draft

refactor(trace/semantics): lift conditional gRPC status fallback into mappings.json#50397
edwardhu-datadog wants to merge 6 commits intofeat/ehu/OTEL-2718-otlp-grpc-status-trace-metricsfrom
feat/ehu/OTEL-2718-semantics-conditional-grpc-status

Conversation

@edwardhu-datadog
Copy link
Copy Markdown

@edwardhu-datadog edwardhu-datadog commented May 5, 2026

Summary

Stacked on top of #50272 (please review that one first).

Moves the gRPC-specific conditional logic out of transform.getOTelGRPCStatusCode and into the semantic registry as when clauses on TagInfo, addressing the reviewer's request in #50272 (comment) and aligning with the dd-source trace-semantics precedent.

After this PR, the rpc.response.status_codeConceptGRPCStatusCode mapping (gated on rpc.system.name=grpc, or rpc.system=grpc when rpc.system.name is absent) is expressed declaratively in mappings.json. The transform package no longer carries gRPC-specific code; LookupString(reg, spanAccessor, ConceptGRPCStatusCode) does the routing.

Design notes

  • Raw-attribute reads in conditions. Conditions read by exact attribute key — no transitive concept resolution. To gate on a renamed attribute (e.g. rpc.systemrpc.system.name), list one fallback row per condition attribute. Matches the dd-source pattern and avoids a resolving map[Concept]struct{} cycle-detection thread through the lookup machinery.
  • Why two rpc.system.name=grpc rows + two rpc.system=grpc (with rpc.system.name absent) rows. dd-source's precedent uses only rpc.system=grpc because it predates OTel semconv v1.39.0 (no references to rpc.system.name or rpc.response.status_code). Our code needs both because the originating commit (dc66769) explicitly handles SDKs that have migrated rpc.response.status_code without updating rpc.system to rpc.system.name. New-semconv rows are listed first so they win precedence; the legacy rows additionally require rpc.system.name to be absent so the new key remains authoritative when both are present (matching the pre-refactor getOTelGRPCStatusCode precedence).
  • Condition schema. {attribute, present, eq}. The dd-source precedent uses valueIs for single-value match and has no array form; In would be unused in production today, so it's not in the schema.

Verifying the "version": "1.39.0" field

rpc.response.status_code and rpc.system.name were both introduced in OTel semantic-conventions v1.39.0 (published 2026-01-12). Verified against the upstream repo:

Attribute Introduced in commit First semconv release Source
rpc.response.status_code cd5b45c6 (2025-12-04) — "Deprecate per-rpc status code attributes in favor of rpc.response.status_code" v1.39.0 (2026-01-12) .chloggen/rpc-status-code.yaml
rpc.system.name 18db3e44 (2025-12-09) — "RPC: align with naming best practices" (renames rpc.systemrpc.system.name) v1.39.0 (2026-01-12) .chloggen/rpc-system-name.yaml

Cross-checked: model/registry/rpc.yaml at v1.24.0 defines rpc.connect_rpc.error_code, rpc.grpc.status_code, rpc.method, rpc.service, rpc.system etc., but not rpc.response.status_code or rpc.system.name. model/rpc/registry.yaml at v1.39.0 defines both.

What changed

File Change
pkg/trace/semantics/lookup.go Add conditionMatches/conditionsMatch helpers; insert if !conditionsMatch(...) { continue } in each of Lookup/LookupFloat64/LookupInt64 (3 lines each, otherwise unchanged from main).
pkg/trace/semantics/semantics.go Add Condition type and When []Condition on TagInfo.
pkg/trace/semantics/mappings.json Add rpc.system.name to the rpc.system fallback chain. Add four conditional rpc.response.status_code rows under rpc.grpc.status_code (string/int64 × rpc.system.name/rpc.system+rpc.system.name absent).
pkg/trace/semantics/lookup_test.go New TestConditionalLookup — 8 sub-tests built on NewEmbeddedRegistry() (production rules), no synthetic concepts.
pkg/trace/transform/transform.go Delete getOTelGRPCStatusCode; call LookupString(..., ConceptGRPCStatusCode) directly.

Net diff (against #50272's branch): +143 / -25 across 5 files.

Test plan

  • go test ./pkg/trace/semantics/... -run TestConditionalLookup -v -count=1 — 8 sub-tests pass.
  • go test ./pkg/trace/semantics/... -count=1 — passes.
  • go test ./pkg/trace/transform/... -count=1 — passes; no behavioral change.
  • go test ./pkg/trace/otel/stats/... -count=1 — gRPC OTLP fixture continues to populate rpc.grpc.status_code in span Meta.
  • CI — let GitLab/CircleCI confirm broader integration tests.

Notes for reviewer

🤖 Generated with Claude Code

edwardhu-datadog and others added 5 commits May 5, 2026 15:32
… mappings.json

Move the rpc.system.name/rpc.system → rpc.response.status_code special case
out of transform.getOTelGRPCStatusCode and into the semantic registry as
`when` clauses on TagInfo. This removes the gRPC-specific helper from the
transform package and lets future conditional fallbacks be expressed
declaratively in mappings.json.

Conditions read attributes by exact key (no transitive concept resolution)
— to gate on a renamed attribute, list one fallback row per condition
attribute. This matches the dd-source trace-semantics precedent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the float64-precision sub-test from TestLookupFloat64; it tests a
  structural property unrelated to conditional fallback that this PR
  doesn't touch.
- Drop the conditional.eq cases; the gRPC test cases already exercise Eq
  accept/reject on a realistic concept.
- Drop the conditional.present/conditional.absent cases; Present is
  unused in mappings.json today and the AND test covers Present:true
  implicitly.
- Rename testBoolPtr to boolPtr (the _test.go suffix already conveys it
  is test-only).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nal check inline

Drop the lookupWithConditions/readTag/readFloat64Tag/readInt64Tag
extraction. The conditional fallback feature only needs a single
`if !conditionsMatch(...) { continue }` line at the top of each loop;
extracting per-type readers and a generic helper was orthogonal
churn that obscured the diff.

Net diff vs main: +37 lines, 0 deletions to existing code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-refactor getOTelGRPCStatusCode used GetOTelAttrFromEitherMap with
keys ("rpc.system.name", "rpc.system"), which returns the first non-empty
key — so when an SDK sets rpc.system.name=jsonrpc and (incorrectly) also
rpc.system=grpc, the new key wins and rpc.response.status_code is rejected.

Without this fix the two independent registry rows would each evaluate
against their own attribute, and the legacy rpc.system=grpc row would
match — accepting a status code that pre-PR rejected.

Gate the legacy rows on rpc.system.name being absent so the new semconv
key remains authoritative when both are present.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- TestConditionalLookup now uses NewEmbeddedRegistry() against the real
  mappings.json instead of a hand-rolled fake. Removes ~50 lines of
  Provider/Type struct-literal boilerplate and the synthetic
  conditional.and concept (the rpc.system+rpc.system.name two-condition
  AND in production gives implicit AND coverage). boolPtr helper is no
  longer needed.
- Collapse the duplicated `c.Present != nil` check at the tail of
  conditionMatches into a single return.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@datadog-datadog-prod-us1-2
Copy link
Copy Markdown
Contributor

datadog-datadog-prod-us1-2 Bot commented May 5, 2026

🎯 Code Coverage (details)
Patch Coverage: 72.22%
Overall Coverage: 50.27% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 51cd1fe | Docs | Datadog PR Page | Give us feedback!

…C keys

rpc.response.status_code and rpc.system.name were both first added to
OTel semantic-conventions in v1.39.0 (Jan 2026), not v1.24.0 (Dec 2023).

Verified against the upstream repo:
  - rpc.response.status_code: commit cd5b45c6 (2025-12-04), .chloggen/rpc-status-code.yaml
    "Deprecate per-rpc status code attributes in favor of rpc.response.status_code"
  - rpc.system.name: commit 18db3e44 (2025-12-09), .chloggen/rpc-system-name.yaml
    "Align RPC conventions with naming guidelines. Renames rpc.system to rpc.system.name."

Both commits land in v1.39.0 (published 2026-01-12). model/registry/rpc.yaml
at v1.24.0 has neither attribute.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 5, 2026

Files inventory check summary

File checks results against ancestor 874cceb7:

Results for datadog-agent_7.80.0~devel.git.456.51cd1fe.pipeline.111602222-1_amd64.deb:

No change detected

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 5, 2026

Static quality checks

✅ Please find below the results from static quality gates
Comparison made with ancestor 874cceb
📊 Static Quality Gates Dashboard
🔗 SQG Job

Successful checks

Info

Quality gate Change Size (prev → curr → max)
agent_deb_amd64 +8.64 KiB (0.00% increase) 740.932 → 740.941 → 750.310
agent_deb_amd64_fips +12.64 KiB (0.00% increase) 699.120 → 699.133 → 702.690
agent_heroku_amd64 +8.81 KiB (0.00% increase) 309.076 → 309.085 → 313.960
agent_rpm_amd64 +8.64 KiB (0.00% increase) 740.916 → 740.924 → 750.280
agent_rpm_amd64_fips +12.64 KiB (0.00% increase) 699.104 → 699.116 → 702.670
agent_rpm_arm64 +9.07 KiB (0.00% increase) 718.990 → 718.999 → 724.050
agent_rpm_arm64_fips +13.07 KiB (0.00% increase) 680.269 → 680.282 → 684.460
agent_suse_amd64 +8.64 KiB (0.00% increase) 740.916 → 740.924 → 750.280
agent_suse_amd64_fips +12.64 KiB (0.00% increase) 699.104 → 699.116 → 702.670
agent_suse_arm64 +9.07 KiB (0.00% increase) 718.990 → 718.999 → 724.050
agent_suse_arm64_fips +13.07 KiB (0.00% increase) 680.269 → 680.282 → 684.460
docker_agent_amd64 +10.28 KiB (0.00% increase) 801.308 → 801.318 → 805.870
docker_agent_arm64 +10.73 KiB (0.00% increase) 804.211 → 804.221 → 809.730
docker_agent_jmx_amd64 +10.32 KiB (0.00% increase) 992.228 → 992.238 → 996.590
docker_agent_jmx_arm64 +10.73 KiB (0.00% increase) 983.909 → 983.920 → 989.410
docker_host_profiler_amd64 +22.1 KiB (0.01% increase) 301.103 → 301.125 → 315.800
docker_host_profiler_arm64 +10.61 KiB (0.00% increase) 312.616 → 312.626 → 327.400
15 successful checks with minimal change (< 2 KiB)
Quality gate Current Size
docker_cluster_agent_amd64 206.603 MiB
docker_cluster_agent_arm64 220.635 MiB
docker_cws_instrumentation_amd64 7.142 MiB
docker_cws_instrumentation_arm64 6.689 MiB
docker_dogstatsd_amd64 39.370 MiB
docker_dogstatsd_arm64 37.565 MiB
dogstatsd_deb_amd64 30.024 MiB
dogstatsd_deb_arm64 28.169 MiB
dogstatsd_rpm_amd64 30.024 MiB
dogstatsd_suse_amd64 30.024 MiB
iot_agent_deb_amd64 44.454 MiB
iot_agent_deb_arm64 41.439 MiB
iot_agent_deb_armhf 42.175 MiB
iot_agent_rpm_amd64 44.455 MiB
iot_agent_suse_amd64 44.455 MiB
On-wire sizes (compressed)
Quality gate Change Size (prev → curr → max)
agent_deb_amd64 +7.55 KiB (0.00% increase) 175.278 → 175.286 → 179.160
agent_deb_amd64_fips -39.29 KiB (0.02% reduction) 167.005 → 166.966 → 174.440
agent_heroku_amd64 neutral 74.955 MiB → 80.310
agent_rpm_amd64 -34.2 KiB (0.02% reduction) 177.293 → 177.260 → 182.080
agent_rpm_amd64_fips -17.16 KiB (0.01% reduction) 168.390 → 168.373 → 174.140
agent_rpm_arm64 -21.89 KiB (0.01% reduction) 159.373 → 159.351 → 163.610
agent_rpm_arm64_fips +15.31 KiB (0.01% increase) 151.680 → 151.695 → 156.850
agent_suse_amd64 -34.2 KiB (0.02% reduction) 177.293 → 177.260 → 182.080
agent_suse_amd64_fips -17.16 KiB (0.01% reduction) 168.390 → 168.373 → 174.140
agent_suse_arm64 -21.89 KiB (0.01% reduction) 159.373 → 159.351 → 163.610
agent_suse_arm64_fips +15.31 KiB (0.01% increase) 151.680 → 151.695 → 156.850
docker_agent_amd64 neutral 267.675 MiB → 272.990
docker_agent_arm64 +12.7 KiB (0.00% increase) 254.706 → 254.719 → 261.470
docker_agent_jmx_amd64 +25.87 KiB (0.01% increase) 336.312 → 336.338 → 341.610
docker_agent_jmx_arm64 neutral 319.360 MiB → 326.050
docker_cluster_agent_amd64 +7.61 KiB (0.01% increase) 72.417 → 72.424 → 73.460
docker_cluster_agent_arm64 +2.13 KiB (0.00% increase) 67.874 → 67.876 → 68.680
docker_cws_instrumentation_amd64 neutral 2.999 MiB → 3.330
docker_cws_instrumentation_arm64 neutral 2.729 MiB → 3.090
docker_host_profiler_amd64 +24.89 KiB (0.02% increase) 110.742 → 110.766 → 125.600
docker_host_profiler_arm64 +17.48 KiB (0.02% increase) 105.070 → 105.087 → 120.000
docker_dogstatsd_amd64 neutral 15.238 MiB → 15.870
docker_dogstatsd_arm64 neutral 14.555 MiB → 14.890
dogstatsd_deb_amd64 neutral 7.943 MiB → 8.830
dogstatsd_deb_arm64 neutral 6.826 MiB → 7.750
dogstatsd_rpm_amd64 neutral 7.953 MiB → 8.840
dogstatsd_suse_amd64 neutral 7.953 MiB → 8.840
iot_agent_deb_amd64 +2.35 KiB (0.02% increase) 11.701 → 11.703 → 13.210
iot_agent_deb_arm64 neutral 9.996 MiB → 11.620
iot_agent_deb_armhf neutral 10.205 MiB → 11.780
iot_agent_rpm_amd64 neutral 11.720 MiB → 13.230
iot_agent_suse_amd64 neutral 11.720 MiB → 13.230

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant