otel.instrumentation.reactor.enabled dramatacilly increases cpu usage

[Tiberius202]

Describe the bug

The "otel.instrumentation.reactor.enabled" flag dramatically increases the cpu usage of the java app.

Steps to reproduce

Run performance tests on Java program with Java agent with
otel.instrumentation.common.default-enabled=false
and
otel.instrumentation.reactor.enabled=true

Then run performance tests with
otel.instrumentation.reactor.enabled=true

Expected behavior

We expect cpu utilization in our testing to be around 35% with and without OTEL auto instrumentation.

Actual behavior

When we run OTEL instrumentation with this flag enabled (which is enabled by the defaults), we see CPU utilization is around 60%. We see a doubling of cpu utilization in all of our tests which include this instrumentation flag.

Javaagent or library instrumentation version

2.1.0

Environment

JDK:
openjdk 17.0.14 2025-01-21 LTS
OpenJDK Runtime Environment Corretto-17.0.14.7.1 (build 17.0.14+7-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.14.7.1 (build 17.0.14+7-LTS, mixed mode)
OS:
Alpine Linux v3.21.2
Spring Boot 3.3.7
OTEL 2.1.0 Java Agent
ECS Fargate single task with 8vcpu
Client load of 500 TPS with Hercules and JMeter

OTEL Settings:
otel.instrumentation.spring-boot-actuator-autoconfigure.enabled=true
otel.exporter.otlp.metrics.temporality.preference=DELTA
otel.javaagent.logging=application

Additional context

Thanks to [email protected] for finding this. He may have some more information and testing results.
cc: @akats7

[laurit]

A minimal sample that reproduces the issue could be helpful.

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed automatically if there is no response from the author within 7 additional days from this comment.