Skip to content

fix(telemetry): rebuild remote metric scalar attributes - #406

Merged
huronat merged 12 commits into
mainfrom
fix/collector-metric-scalars
Sep 28, 2026
Merged

huronat merged 12 commits into
mainfrom
fix/collector-metric-scalars

Conversation

@huronat

@huronat huronat commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Collector 0.119 scalar lookups read the first protobuf attribute with a given key, but keep_keys retains later duplicates. Raw OTLP tests reproduce duplicate strings and nested payloads surviving the existing remote metric allowlist.

Rebuild remote resource and point maps after the existing exact per-metric pruning. Each allowed resource retains one string; each allowed point key retains its first scalar value. Clear scratch state per resource/point, omit missing or non-scalar values, and preserve collector-authored identity. Local Prometheus processing, metric names, values, histogram buckets, timestamps, temporality and producer domain policies remain unchanged.

The Docker regression runs the actual rendered 0.119 pipeline through both receivers. It covers allowed-first/hostile-second duplicates, reversed service claims, nested-first values, resource/point conflicts and missing optional fields across successive points. Both metric cases and both existing trace cases pass. Ruff, workflow syntax and diff checks pass. CI runs the runtime cases. A sibling sweep identified candidate key-pruning statements; the runtime tests establish the specific resource and datapoint defect fixed here.

This PR changes only metric processing, its contract/documentation and its regression/CI coverage. It reuses the pinned collector helpers from #402.

Summary by CodeRabbit

  • Bug Fixes

    • Remote metric exports now retain only unique, scalar values for approved resource and datapoint attributes, excluding duplicate, missing, and non-scalar values. When duplicate keys occur, the first lookup value is retained.
    • Metric values and aggregation data remain intact, and local Prometheus processing is unchanged.
  • Documentation

    • Clarified remote metric attribute handling rules, including which values are retained and how they behave across collector pipelines.

@huronat
huronat requested a review from a team as a code owner September 28, 2026 15:24
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 90ff6ed8-93af-4e3f-bfe3-c4f6f6473567

📥 Commits

Reviewing files that changed from the base of the PR and between 0dea2c2 and b8c3b03.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The collector now rebuilds allowlisted Better Stack metric attributes from scalar values. A Docker-backed regression checks both receiver pipelines, and CI runs matching collector privacy tests.

Changes

Remote metric privacy

Layer / File(s) Summary
Metric reconstruction and pipeline wiring
telemetry/contract.yaml, deploy/helm/sie-cluster/templates/_otel-collector-config.tpl, telemetry/README.md
The contract describes scalar reconstruction after datapoint-attribute validation. Both Better Stack metric pipelines apply the transform after contract filtering; the gateway pipeline then applies queue identity. The README documents retained values and states that local Prometheus processing remains unchanged.
Regression coverage and CI
tools/ci/tests/test_collector_metric_privacy.py, .github/workflows/ci.yml
The Docker-backed test checks duplicate and non-scalar attributes, remote and local output, and preserved metric data for both receiver pipelines. CI runs all matching collector privacy tests.

Sequence Diagram(s)

sequenceDiagram
  participant OtlpReceiver
  participant Collector
  participant BetterStackExporter
  OtlpReceiver->>Collector: Send metrics
  Collector->>Collector: Apply transform/contract_metrics
  Collector->>Collector: Apply transform/remote_metric_scalars
  opt Gateway pipeline
    Collector->>Collector: Apply transform/remote_queue_identity
  end
  Collector->>BetterStackExporter: Export processed metrics
Loading

Suggested reviewers: mamayer19

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to b8c3b

The metric privacy regression is selected by the pinned collector privacy CI step. No actionable merge risk is established by the supplied evidence.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 1 files. (1 skipped: 1 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: rebuilding remote metric scalar attributes for telemetry.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 1 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@huronat

huronat commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 28, 2026
Base automatically changed from fix/safe-batch-fanin to main September 28, 2026 15:44
@huronat
huronat dismissed coderabbitai[bot]’s stale review September 28, 2026 15:44

The base branch was changed.

# Conflicts:
#	.github/workflows/ci.yml
coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 28, 2026
@huronat
huronat merged commit 91079ac into main Sep 28, 2026
41 checks passed
@huronat
huronat deleted the fix/collector-metric-scalars branch September 28, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant