Fix APM Disablement V1 (ASM Standalone)

[vpellan]

What does this PR do?

This PR fixes multiple bugs on APM Disablement V1 (aka ASM Standalone) when it was activated:

• When HTTP frameworks (Net::HTTP, RestClient, Excon, Faraday...) were instrumented, the sampling_priority was set to MANUAL_REJECT if tags contained _dd.p.appsec (which was used to check for upstream propagated appsec events), then not sampled later due to a change in the sampler in this PR This was fixed by making this check during the building step of a trace from a digest (which also means that it now works even if an HTTP framework is not instrumented)
• Distributed tracing was skipped only for Net::HTTP (in case of upstream or local security event). It now skips it on every HTTP frameworks supported, but also Sidekiq and GRPC.
• The APM Tracing disabled tag (_dd.apm.enabled=0) was not added when appsec was disabled. It is now fixed and the code related to it was moved to Tracing
• Added tests that actually tests APM disablement heartbeat traces (ASM Standalone too) and cleaned related tests

Motivation:

When I tried to activate SCA Standalone system-tests, I discovered that disabling AppSec and disabling APM Tracing would result in heartbeat traces not sending dd.apm.enabled=0 tag. During my investigation, I discovered more bugs (listed above).
This is also a step towards APM Disablement V2. For now, the variables are still called after appsec standalone, but this will change to APM disablement in the future and lose all reference to asm standalone, as it is in fact an APM feature.

Change log entry

None.

Additional Notes:

This PR should be followed soon by APM Disablement V2, which will replace the env var by DD_APM_TRACING_ENABLED, and every config/variable referencing asm standalone to apm disablement.

I understand that there might be some concerns about AppSec stuff being in APM territory but this is mostly a matter of naming that will be fixed when APM Disablement V2 will be released. In the V2, I will replace any instance of Datadog.configuration.appsec.standalone.enabled by Datadog.configuration.tracing.apm.disabled, Datadog::AppSec::Ext::TAG_DISTRIBUTED_APPSEC_EVENT by something like Datadog::Tracing::Ext::TAG_DISTRIBUTED_TRACE_SOURCE (_dd.p.ts, which will be a bitmask). The only exception might be in the propagate_sampling_priority? method, in tracer.rb, where we check if AppSec is activated.

How to test the change?

CI and system-tests

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 99.72603% with 1 line in your changes missing coverage. Please review.

Project coverage is 97.76%. Comparing base (4f13111) to head (839e483).
Report is 54 commits behind head on master.

Files with missing lines	Patch %	Lines
.../datadog/tracing/distributed/propagation_policy.rb	92.30%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4479      +/-   ##
==========================================
+ Coverage   97.73%   97.76%   +0.02%     
==========================================
  Files        1386     1392       +6     
  Lines       84166    84891     +725     
  Branches     4257     4277      +20     
==========================================
+ Hits        82262    82995     +733     
+ Misses       1904     1896       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[datadog-datadog-prod-us1]

Datadog Report

Branch report: vpellan/fix-sca-standalone
Commit report: 839e483
Test service: dd-trace-rb

✅ 0 Failed, 20782 Passed, 1370 Skipped, 3m 13.89s Total Time

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-03-26 10:50:51

Comparing candidate commit 839e483 in PR branch vpellan/fix-sca-standalone with baseline commit 4f13111 in branch master.

Found 1 performance improvements and 1 performance regressions! Performance is the same for 29 metrics, 2 unstable metrics.

scenario:method instrumentation

• 🟥 throughput [-10760.935op/s; -10287.772op/s] or [-6.043%; -5.777%]

scenario:tracing - Propagation - Datadog

• 🟩 throughput [+2447.321op/s; +2524.772op/s] or [+8.236%; +8.497%]

[TonyCTHsu]

FYI: Currently, there is a karafka integration under review which contains distributed tracing mechanism

[vpellan]

FYI: Currently, there is a karafka integration under review which contains distributed tracing mechanism

What is the process for this kind of situation ? If I merge to master first, should I myself add a commit on the Karafka integration to add APM Disablement capabilities ? (Of course if the other way around happens I could just rebase and add it here)

[TonyCTHsu]

What is the process for this kind of situation ?

I would suggest not to wait on it. However, it would be much welcomed to encapsulate the logic so that the pattern can be adopted easily.

[Strech]

LGTM 👍🏼

Great job, lots of effort in it and thanks for keep pushing! I left few non-blocking suggestions and IMHO it's good to go 🚀

I'll review it again with the the rate limiting, instead of disablement in mind.