Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(profiling): add runtime_platform tag automatically #954

Merged
merged 10 commits into from
Mar 22, 2025
Merged

feat(profiling): add runtime_platform tag automatically #954

merged 10 commits into from
Mar 22, 2025

Conversation

ivoanjo
Copy link
Member

@ivoanjo ivoanjo commented Mar 20, 2025 •
edited by morrisonlevi
Loading

What does this PR do?

This PR adds the runtime_platform tag as per Profiler common tags. It uses the target triple, which is a bit different from what Ruby did before, but close enough.

Motivation

Up until now, only Ruby was reporting this tag, but in incident 35390 it would've been useful to have this information, and it seems easy enough to add.

Additional Notes

We should probably talk to the Datadog platform team to figure out if there's a "standard" or "approved" tag name, but this one is already in use so let's not block on that. We can migrate later as needed.

How to test the change?

I've added test coverage for this feature. I've also tested it with the Ruby profiler.

@ivoanjo
[NO-TICKET] Add runtime_platform tag automatically to reported profiles
f5e729f 
**What does this PR do?**

This PR adds the `runtime_platform` tag as per
https://docs.google.com/spreadsheets/d/1LOGMf4c4Avbtn36uZ2SWvhIGKRPLM1BoWkUP4JYj7hA/edit?gid=0#gid=0
in the same format "architecture-os[-musl]".

**Motivation:**

Up until now, only Ruby was reporting this tag, but in incident 35390
it would've been useful to have this information, and it seems
easy enough to add.

**Additional Notes:**

N/A

**How to test the change?**

I've added test coverage for this feature. I've also tested it
with the Ruby profiler.
@ivoanjo ivoanjo requested review from a team as code owners March 20, 2025 14:26
@github-actions github-actions bot added profiling Relates to the profiling* modules. common labels Mar 20, 2025
@morrisonlevi morrisonlevi changed the title [NO-TICKET] Add runtime_platform tag automatically to reported profiles feat: add runtime_platform tag automatically to profiles Mar 20, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 20, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-03-21 18:53:47

Comparing candidate commit f44a94d in PR branch ivoanjo/automatically-add-runtime-platform-profiler with baseline commit dd5ca94 in branch main.

Found 2 performance improvements and 5 performance regressions! Performance is the same for 45 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/x371413321323331

  • 🟥 execution_time [+513.459ns; +534.653ns] or [+8.504%; +8.855%]
  • 🟥 throughput [-13450974.031op/s; -12960508.818op/s] or [-8.122%; -7.826%]

scenario:credit_card/is_card_number_no_luhn/ 3782-8224-6310-005

  • 🟩 execution_time [-2.774µs; -2.669µs] or [-4.164%; -4.006%]
  • 🟩 throughput [+626907.352op/s; +651122.334op/s] or [+4.176%; +4.338%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟥 execution_time [+524.640ns; +545.391ns] or [+8.691%; +9.035%]
  • 🟥 throughput [-13706935.305op/s; -13227006.772op/s] or [-8.274%; -7.985%]

scenario:tags/replace_trace_tags

  • 🟥 execution_time [+156.570ns; +165.427ns] or [+6.631%; +7.006%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.499µs 2.522µs ± 0.024µs 2.514µs ± 0.008µs 2.525µs 2.596µs 2.603µs 2.610µs 3.83% 2.332 4.848 0.96% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.519µs; 2.525µs] or [-0.134%; +0.134%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 245.549ns 253.746ns ± 9.856ns 248.876ns ± 1.549ns 255.286ns 278.201ns 282.327ns 284.386ns 14.27% 1.707 1.717 3.87% 0.697ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [252.380ns; 255.112ns] or [-0.538%; +0.538%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.204µs 24.651µs ± 10.218µs 17.456µs ± 0.157µs 33.417µs 42.572µs 44.194µs 84.100µs 381.79% 1.679 4.999 41.35% 0.723µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.234µs; 26.067µs] or [-5.745%; +5.745%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.179µs 3.176µs ± 1.424µs 2.981µs ± 0.029µs 3.011µs 3.620µs 13.782µs 14.820µs 397.06% 7.392 55.612 44.73% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.979µs; 3.373µs] or [-6.215%; +6.215%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.916µs 4.982µs ± 0.038µs 4.961µs ± 0.029µs 5.016µs 5.043µs 5.045µs 5.048µs 1.74% 0.232 -1.503 0.77% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [4.976µs; 4.987µs] or [-0.107%; +0.107%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.055ms 6.070ms ± 0.007ms 6.069ms ± 0.003ms 6.072ms 6.079ms 6.082ms 6.136ms 1.11% 4.155 37.032 0.12% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.069ms; 6.071ms] or [-0.016%; +0.016%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 208.489µs 209.033µs ± 0.189µs 209.037µs ± 0.127µs 209.163µs 209.323µs 209.425µs 209.579µs 0.26% -0.120 0.014 0.09% 0.013µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4771478.645op/s 4783937.254op/s ± 4327.019op/s 4783849.088op/s ± 2903.387op/s 4786634.919op/s 4791259.784op/s 4793954.702op/s 4796404.615op/s 0.26% 0.126 0.014 0.09% 305.966op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.212µs 18.299µs ± 0.104µs 18.291µs ± 0.024µs 18.316µs 18.365µs 18.394µs 19.662µs 7.49% 11.296 145.549 0.57% 0.007µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 50859308.621op/s 54648617.419op/s ± 292352.444op/s 54670402.944op/s ± 72427.563op/s 54739862.606op/s 54881423.619op/s 54893922.853op/s 54908851.307op/s 0.44% -10.953 139.615 0.53% 20672.440op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.669µs 10.732µs ± 0.023µs 10.733µs ± 0.017µs 10.749µs 10.766µs 10.780µs 10.794µs 0.56% -0.180 -0.261 0.22% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 92648287.147op/s 93178934.850op/s ± 201094.258op/s 93166489.396op/s ± 146824.866op/s 93317079.413op/s 93530694.131op/s 93660788.561op/s 93730375.203op/s 0.61% 0.191 -0.255 0.22% 14219.511op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [209.007µs; 209.059µs] or [-0.013%; +0.013%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4783337.571op/s; 4784536.938op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.285µs; 18.314µs] or [-0.079%; +0.079%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54608100.182op/s; 54689134.656op/s] or [-0.074%; +0.074%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.729µs; 10.735µs] or [-0.030%; +0.030%] None None None
normalization/normalize_name/normalize_name/good throughput [93151065.120op/s; 93206804.580op/s] or [-0.030%; +0.030%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 66.118µs 66.326µs ± 0.187µs 66.307µs ± 0.057µs 66.367µs 66.468µs 66.561µs 68.701µs 3.61% 10.311 128.176 0.28% 0.013µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [66.300µs; 66.352µs] or [-0.039%; +0.039%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 504.384µs 505.809µs ± 1.625µs 505.641µs ± 0.241µs 505.875µs 506.291µs 512.869µs 520.888µs 3.02% 8.106 69.695 0.32% 0.115µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1919800.341op/s 1977051.825op/s ± 6190.085op/s 1977687.292op/s ± 944.039op/s 1978644.109op/s 1980247.134op/s 1981359.583op/s 1982615.546op/s 0.25% -8.048 68.910 0.31% 437.705op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 452.131µs 453.265µs ± 0.364µs 453.240µs ± 0.224µs 453.499µs 453.846µs 454.024µs 454.546µs 0.29% 0.135 0.482 0.08% 0.026µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2199999.538op/s 2206218.588op/s ± 1773.130op/s 2206336.721op/s ± 1092.374op/s 2207351.784op/s 2208916.902op/s 2210359.926op/s 2211749.833op/s 0.25% -0.129 0.480 0.08% 125.379op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 175.656µs 176.706µs ± 0.329µs 176.781µs ± 0.204µs 176.942µs 177.142µs 177.236µs 177.348µs 0.32% -0.708 0.191 0.19% 0.023µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5638625.188op/s 5659127.424op/s ± 10535.676op/s 5656705.967op/s ± 6513.242op/s 5665951.999op/s 5676916.374op/s 5690114.071op/s 5692942.140op/s 0.64% 0.717 0.211 0.19% 744.985op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.535µs 37.653µs ± 0.045µs 37.648µs ± 0.032µs 37.685µs 37.724µs 37.774µs 37.802µs 0.41% 0.395 0.359 0.12% 0.003µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26453354.201op/s 26558544.429op/s ± 31581.935op/s 26561641.114op/s ± 22302.096op/s 26581901.314op/s 26607651.520op/s 26620418.996op/s 26641644.174op/s 0.30% -0.387 0.347 0.12% 2233.180op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.102µs 48.321µs ± 0.216µs 48.295µs ± 0.182µs 48.515µs 48.631µs 48.700µs 49.572µs 2.64% 1.162 3.802 0.44% 0.015µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20172660.237op/s 20695191.933op/s ± 91839.996op/s 20706212.077op/s ± 78279.130op/s 20784020.058op/s 20786712.421op/s 20787414.373op/s 20789099.384op/s 0.40% -1.105 3.380 0.44% 6494.068op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [505.584µs; 506.034µs] or [-0.045%; +0.045%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1976193.938op/s; 1977909.711op/s] or [-0.043%; +0.043%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [453.214µs; 453.315µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2205972.849op/s; 2206464.326op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [176.661µs; 176.752µs] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5657667.280op/s; 5660587.567op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.647µs; 37.659µs] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26554167.476op/s; 26562921.381op/s] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.291µs; 48.351µs] or [-0.062%; +0.062%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20682463.793op/s; 20707920.074op/s] or [-0.062%; +0.062%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 147.886µs 149.071µs ± 0.355µs 149.035µs ± 0.122µs 149.169µs 149.498µs 149.844µs 152.585µs 2.38% 4.827 46.800 0.24% 0.025µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [149.022µs; 149.120µs] or [-0.033%; +0.033%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.896µs 3.914µs ± 0.003µs 3.914µs ± 0.002µs 3.916µs 3.920µs 3.921µs 3.923µs 0.23% -0.741 4.497 0.08% 0.000µs 1 200
credit_card/is_card_number/ throughput 254895163.893op/s 255488911.896op/s ± 212087.431op/s 255492118.454op/s ± 120655.921op/s 255601380.264op/s 255831919.310op/s 255933776.804op/s 256699057.013op/s 0.47% 0.756 4.573 0.08% 14996.846op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 82.119µs 82.665µs ± 0.355µs 82.600µs ± 0.215µs 82.841µs 83.254µs 83.958µs 84.367µs 2.14% 1.538 4.552 0.43% 0.025µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11852971.045op/s 12097226.323op/s ± 51626.022op/s 12106577.505op/s ± 31379.960op/s 12132636.796op/s 12161419.497op/s 12172679.775op/s 12177493.918op/s 0.59% -1.485 4.275 0.43% 3650.511op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 76.366µs 77.222µs ± 0.360µs 77.205µs ± 0.217µs 77.432µs 77.811µs 78.158µs 78.268µs 1.38% 0.296 0.049 0.46% 0.025µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 12776661.403op/s 12949935.071op/s ± 60246.556op/s 12952447.036op/s ± 36547.391op/s 12988483.764op/s 13050132.475op/s 13076157.319op/s 13094859.160op/s 1.10% -0.268 0.026 0.46% 4260.075op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.895µs 3.913µs ± 0.003µs 3.913µs ± 0.002µs 3.915µs 3.918µs 3.919µs 3.922µs 0.22% -1.089 5.966 0.08% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254969511.392op/s 255535163.961op/s ± 195325.429op/s 255536767.427op/s ± 101613.524op/s 255624284.307op/s 255814305.890op/s 255991923.404op/s 256719760.250op/s 0.46% 1.104 6.058 0.08% 13811.594op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 72.764µs 73.645µs ± 0.408µs 73.597µs ± 0.254µs 73.885µs 74.314µs 74.710µs 75.199µs 2.18% 0.710 0.815 0.55% 0.029µs 1 200
credit_card/is_card_number/378282246310005 throughput 13298007.567op/s 13579091.397op/s ± 74951.697op/s 13587597.942op/s ± 46994.384op/s 13631863.118op/s 13683099.121op/s 13710327.715op/s 13743126.416op/s 1.14% -0.672 0.720 0.55% 5299.885op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.882µs 52.137µs ± 0.083µs 52.140µs ± 0.061µs 52.197µs 52.266µs 52.322µs 52.353µs 0.41% -0.203 0.018 0.16% 0.006µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19101212.350op/s 19180273.172op/s ± 30482.243op/s 19179257.635op/s ± 22298.322op/s 19202593.310op/s 19229533.692op/s 19255618.377op/s 19274524.951op/s 0.50% 0.213 0.025 0.16% 2155.420op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.426µs 6.562µs ± 0.076µs 6.555µs ± 0.056µs 6.617µs 6.696µs 6.746µs 6.765µs 3.22% 0.419 -0.533 1.16% 0.005µs 1 200
credit_card/is_card_number/x371413321323331 throughput 147808764.433op/s 152408937.929op/s ± 1763259.760op/s 152562877.231op/s ± 1304411.263op/s 153767085.865op/s 155046371.687op/s 155381756.538op/s 155614755.577op/s 2.00% -0.375 -0.585 1.15% 124681.293op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.894µs 3.914µs ± 0.003µs 3.914µs ± 0.002µs 3.916µs 3.918µs 3.921µs 3.926µs 0.31% -1.250 9.554 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254713063.641op/s 255497770.693op/s ± 198429.489op/s 255504492.422op/s ± 110489.764op/s 255579668.671op/s 255807660.823op/s 255957849.809op/s 256816075.481op/s 0.51% 1.273 9.688 0.08% 14031.084op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 63.428µs 63.899µs ± 0.174µs 63.883µs ± 0.100µs 63.995µs 64.198µs 64.369µs 64.501µs 0.97% 0.379 0.643 0.27% 0.012µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15503623.355op/s 15649814.948op/s ± 42550.531op/s 15653738.507op/s ± 24447.649op/s 15675514.177op/s 15714157.435op/s 15740092.075op/s 15766017.043op/s 0.72% -0.359 0.622 0.27% 3008.777op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.395µs 57.559µs ± 0.095µs 57.540µs ± 0.044µs 57.589µs 57.715µs 57.940µs 58.151µs 1.06% 2.379 9.791 0.17% 0.007µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17196486.461op/s 17373438.589op/s ± 28675.232op/s 17379295.177op/s ± 13253.285op/s 17391332.598op/s 17403507.677op/s 17418075.939op/s 17423158.863op/s 0.25% -2.349 9.575 0.16% 2027.645op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.898µs 3.915µs ± 0.003µs 3.915µs ± 0.002µs 3.916µs 3.919µs 3.920µs 3.929µs 0.36% -0.555 7.328 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254535767.605op/s 255447835.023op/s ± 189006.214op/s 255456291.429op/s ± 101309.552op/s 255542528.667op/s 255738042.494op/s 255927510.273op/s 256552120.245op/s 0.43% 0.575 7.375 0.07% 13364.758op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.552µs 54.724µs ± 0.168µs 54.675µs ± 0.034µs 54.714µs 55.118µs 55.232µs 55.573µs 1.64% 2.581 7.106 0.31% 0.012µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17994388.701op/s 18273623.128op/s ± 55506.678op/s 18289996.288op/s ± 11426.647op/s 18300631.830op/s 18316313.154op/s 18330859.544op/s 18331218.064op/s 0.23% -2.559 6.946 0.30% 3924.915op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.847µs 52.157µs ± 0.108µs 52.147µs ± 0.072µs 52.220µs 52.323µs 52.451µs 52.698µs 1.06% 0.755 2.745 0.21% 0.008µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 18975881.556op/s 19173132.588op/s ± 39580.390op/s 19176534.637op/s ± 26260.239op/s 19202048.863op/s 19229539.459op/s 19246369.590op/s 19287689.552op/s 0.58% -0.729 2.646 0.21% 2798.756op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.431µs 6.572µs ± 0.075µs 6.572µs ± 0.058µs 6.630µs 6.700µs 6.734µs 6.770µs 3.01% 0.206 -0.569 1.13% 0.005µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 147718588.989op/s 152188265.648op/s ± 1726998.082op/s 152169637.049op/s ± 1339000.148op/s 153492085.996op/s 154694987.630op/s 155417947.955op/s 155493581.053op/s 2.18% -0.159 -0.602 1.13% 122117.205op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.914µs; 3.915µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/ throughput [255459518.618op/s; 255518305.175op/s] or [-0.012%; +0.012%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [82.616µs; 82.714µs] or [-0.060%; +0.060%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12090071.453op/s; 12104381.193op/s] or [-0.059%; +0.059%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [77.172µs; 77.272µs] or [-0.065%; +0.065%] None None None
credit_card/is_card_number/ 378282246310005 throughput [12941585.478op/s; 12958284.665op/s] or [-0.064%; +0.064%] None None None
credit_card/is_card_number/37828224631 execution_time [3.913µs; 3.914µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/37828224631 throughput [255508093.735op/s; 255562234.186op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/378282246310005 execution_time [73.588µs; 73.701µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/378282246310005 throughput [13568703.813op/s; 13589478.982op/s] or [-0.076%; +0.076%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.126µs; 52.149µs] or [-0.022%; +0.022%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19176048.627op/s; 19184497.718op/s] or [-0.022%; +0.022%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.552µs; 6.573µs] or [-0.161%; +0.161%] None None None
credit_card/is_card_number/x371413321323331 throughput [152164567.085op/s; 152653308.774op/s] or [-0.160%; +0.160%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.914µs; 3.914µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ throughput [255470270.274op/s; 255525271.112op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [63.875µs; 63.923µs] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15643917.854op/s; 15655712.042op/s] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [57.546µs; 57.573µs] or [-0.023%; +0.023%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17369464.478op/s; 17377412.701op/s] or [-0.023%; +0.023%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.914µs; 3.915µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255421640.580op/s; 255474029.467op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.701µs; 54.747µs] or [-0.042%; +0.042%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18265930.436op/s; 18281315.820op/s] or [-0.042%; +0.042%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.142µs; 52.171µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19167647.127op/s; 19178618.049op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.561µs; 6.582µs] or [-0.158%; +0.158%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [151948920.323op/s; 152427610.972op/s] or [-0.157%; +0.157%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.584µs 33.292µs ± 0.956µs 32.721µs ± 0.069µs 34.358µs 35.268µs 35.470µs 35.532µs 8.59% 1.103 -0.455 2.86% 0.068µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.159µs; 33.424µs] or [-0.398%; +0.398%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f44a94d 1742582537 ivoanjo/automatically-add-runtime-platform-profiler
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 54.336ms 54.598ms ± 0.191ms 54.557ms ± 0.081ms 54.652ms 54.844ms 55.255ms 56.114ms 2.85% 3.753 22.985 0.35% 0.013ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [54.572ms; 54.624ms] or [-0.048%; +0.048%] None None None

Baseline

Omitted due to size.

@ivoanjo
Simplify test for exporter
50859a5 
We already have tests for reporting data so while there's some value
for testing that ffi works end-to-end, I'm not sure it's worth
repeating the `runtime_platform` tests in ffi.
@codecov-commenter
Copy link

codecov-commenter commented Mar 20, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.85%. Comparing base (dec5b9c) to head (a3a85c7).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #954      +/-   ##
==========================================
+ Coverage   72.79%   72.85%   +0.05%     
==========================================
  Files         334      334              
  Lines       50903    50915      +12     
==========================================
+ Hits        37057    37095      +38     
+ Misses      13846    13820      -26
Components Coverage Δ
crashtracker 42.85% <ø> (ø)
crashtracker-ffi 6.25% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 91.62% <ø> (-0.20%) ⬇️
data-pipeline-ffi 90.29% <ø> (ø)
ddcommon 82.81% <ø> (ø)
ddcommon-ffi 70.13% <ø> (ø)
ddtelemetry 61.87% <ø> (ø)
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 89.60% <ø> (ø)
dogstatsd-client 82.57% <ø> (ø)
ipc 82.41% <ø> (ø)
profiling 81.87% <100.00%> (+0.10%) ⬆️
profiling-ffi 69.88% <ø> (-0.03%) ⬇️
serverless 0.00% <ø> (ø)
sidecar 41.02% <ø> (+0.16%) ⬆️
sidecar-ffi 1.61% <ø> (+1.32%) ⬆️
spawn-worker 54.37% <ø> (ø)
tinybytes 91.59% <ø> (ø)
trace-mini-agent 74.66% <ø> (ø)
trace-normalization 98.24% <ø> (ø)
trace-obfuscation 96.00% <ø> (ø)
trace-protobuf 78.13% <ø> (ø)
trace-utils 92.91% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@r1viollet
Copy link
Contributor

r1viollet commented Mar 20, 2025 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 77.23 MB 77.23 MB -0% (-1.85 KB) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.83 MB 23.83 MB +0% (+736 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.69 MB 7.69 MB +0% (+120 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 22.42 MB 22.42 MB -0% (-2.03 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 71.53 MB 71.54 MB +0% (+1.94 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 7.62 MB 7.62 MB +0% (+360 B) 👌
i686-alpine-linux-musl
Artifact Baseline Commit Change
/i686-alpine-linux-musl/lib/libdatadog_profiling.a 66.56 MB 66.56 MB +0% (+2.26 KB) 👌
/i686-alpine-linux-musl/lib/libdatadog_profiling.so 8.18 MB 8.19 MB +0% (+152 B) 👌
/i686-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.01 MB 23.01 MB +0% (+664 B) 👌
i686-unknown-linux-gnu
Artifact Baseline Commit Change
/i686-unknown-linux-gnu/lib/libdatadog_profiling.a 67.45 MB 67.45 MB +0% (+808 B) 👌
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so 8.07 MB 8.07 MB +0% (+96 B) 👌
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.62 MB 20.62 MB -0% (-1.09 KB) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 16.59 MB 16.59 MB +0% (+1.00 KB) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 55.10 KB 55.10 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 111.21 MB 111.21 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 625.28 MB 625.62 MB +.05% (+352.33 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 4.99 MB 4.99 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 55.10 KB 55.10 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 15.85 MB 15.85 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.38 MB 26.38 MB -0% (-106 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.03 MB 14.03 MB +.01% (+2.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 55.94 KB 55.94 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 113.18 MB 113.19 MB +0% (+8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 617.99 MB 618.14 MB +.02% (+155.46 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.78 MB 3.78 MB +.02% (+1.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 55.94 KB 55.94 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.51 MB 16.51 MB +.04% (+8.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.35 MB 24.36 MB +.02% (+6.90 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 66.56 MB 66.56 MB +0% (+2.26 KB) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.18 MB 8.19 MB +0% (+152 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.01 MB 23.01 MB +0% (+664 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 67.45 MB 67.45 MB +0% (+808 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.07 MB 8.07 MB +0% (+96 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.62 MB 20.62 MB -0% (-1.09 KB) 👌

hoolioh
hoolioh approved these changes Mar 21, 2025
View reviewed changes
Copy link
Contributor

@hoolioh hoolioh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just left a small suggestion.

@morrisonlevi morrisonlevi changed the title feat: add runtime_platform tag automatically to profiles feat(profiling): add runtime_platform tag automatically Mar 21, 2025
@morrisonlevi morrisonlevi enabled auto-merge (squash) March 21, 2025 18:48
morrisonlevi
morrisonlevi approved these changes Mar 21, 2025
View reviewed changes
Copy link
Contributor

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. We should probably talk to the Datadog platform team to figure out if there's a "standard" or "approved" tag name, but this one is already in use so let's not block on that. We can migrate later as needed.

@morrisonlevi morrisonlevi disabled auto-merge March 22, 2025 02:11
@morrisonlevi
Copy link
Contributor

Two jobs are in limbo. GitHub is waiting for them to be completed, but they have already completed.

I'm bypassing requirements and merging.

@morrisonlevi morrisonlevi merged commit ffd213b into main Mar 22, 2025
14 checks passed
@morrisonlevi morrisonlevi deleted the ivoanjo/automatically-add-runtime-platform-profiler branch March 22, 2025 02:13
@ivoanjo
Copy link
Member Author

ivoanjo commented Mar 25, 2025

Thanks Levi for picking this up and getting it across the line! :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hoolioh hoolioh hoolioh approved these changes

@morrisonlevi morrisonlevi morrisonlevi approved these changes

Assignees
No one assigned
Labels
common profiling Relates to the profiling* modules.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ivoanjo @codecov-commenter @r1viollet @morrisonlevi @hoolioh