Skip to content

[BugFix] Emit valid Prometheus # TYPE lines for BE/CN JVM metrics (backport #75240)#75369

Merged
wanpengfei-git merged 1 commit into
branch-4.1from
mergify/bp/branch-4.1/pr-75240
Jun 25, 2026
Merged

[BugFix] Emit valid Prometheus # TYPE lines for BE/CN JVM metrics (backport #75240)#75369
wanpengfei-git merged 1 commit into
branch-4.1from
mergify/bp/branch-4.1/pr-75240

Conversation

@mergify

@mergify mergify Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

enable_jvm_metrics = true makes the BE/CN /metrics endpoint emit invalid
Prometheus text. Each JVM gauge is registered with its label baked into the
metric-name string, e.g.:

register_metric("jvm_heap_size_bytes{type=\"committed\"}", &gauge);

so the exposition prints that string verbatim after # TYPE :

# TYPE starrocks_be_jvm_heap_size_bytes{type="committed"} gauge   <-- illegal
starrocks_be_jvm_heap_size_bytes{type="committed"} 905969664

A # TYPE metric name must not contain a label set. Prometheus aborts the
entire scrape on the first text-format error, so flipping this flag on
takes the node's whole metric set offline (up == 0) — not just the JVM
series. promtool check metrics rejects it with invalid metric name in comment. Reproduced on 4.0.10/4.0.11 and main; the feature shipped in 4.0.0
(#62210).

What I'm doing:

Register each JVM gauge under a bare metric name plus a real type label via
MetricLabels, mirroring the existing stream_load_metrics pattern. The
metrics framework then emits one valid # TYPE <name> line followed by the
labelled samples:

# TYPE starrocks_be_jvm_heap_size_bytes gauge
starrocks_be_jvm_heap_size_bytes{type="committed"} 905969664
starrocks_be_jvm_heap_size_bytes{type="max"} 21474836480
starrocks_be_jvm_heap_size_bytes{type="used"} 256609448

The sample lines (name + labels) are byte-identical to before, so the
time-series identity is unchanged and existing dashboards/alerts keep working;
only the previously-malformed # TYPE comment is corrected. No metric is
renamed, so no docs change is required.

jvm_metrics_test is updated to look metrics up by name + label, plus a
regression guard asserting the old braced names no longer resolve (which would
mean the label was still baked into the name).

Fixes #75159

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5

This is an automatic backport of pull request #75240 done by [Mergify](https://mergify.com).

…5240)

Signed-off-by: Srihith Garlapati <srihith.garlapati@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit ba3b1f1)
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) June 25, 2026 17:29
@wanpengfei-git wanpengfei-git merged commit d4d4fff into branch-4.1 Jun 25, 2026
39 of 40 checks passed
@wanpengfei-git wanpengfei-git deleted the mergify/bp/branch-4.1/pr-75240 branch June 25, 2026 17:55
@mergify

mergify Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants