Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(profiling): label handling and fuzzing #959

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

fix(profiling): label handling and fuzzing #959

wants to merge 3 commits into from

Conversation

morrisonlevi
Copy link
Contributor

@morrisonlevi morrisonlevi commented Mar 21, 2025 •
edited
Loading

What does this PR do?

This fixes GH #958 by:

  • Being more cautious around label conversion errors (moved from From to TryFrom).
  • Modeling the exclusive label.str or label.num+label.num_value a bit better.
  • Simplifying Option<*Id> to just be *Id. This removes potential issues around Some(0) and None not always being considered the same despite the fact they ought to be. Now there's just one way to represent it, (which is 0).

Motivation

I had a fuzzer error locally, and also hit one in CI. I figured since it cropped up both places recently, it was time to attack it.

Additional Notes

I hit a 3rd party license job failure. I updated the licenses, but it the CI seems to fetch different values. Not sure what's going on there; we'll see if it keeps happening.

I found another fuzzer error, but it doesn't appear to be around label handling, so I'll leave that to another PR.

How to test the change?

If you use the FFI, there's nothing to change. If you use the api::* layer, you'll have to account for some shifts in representation.

@github-actions github-actions bot added the profiling Relates to the profiling* modules. label Mar 21, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 21, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-03-26 14:54:04

Comparing candidate commit a47b8cc in PR branch PROF-11530-Fuzz-tests-failure-in-profiling-with-labels with baseline commit 3922cd6 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.001ms 6.014ms ± 0.006ms 6.013ms ± 0.002ms 6.016ms 6.021ms 6.030ms 6.067ms 0.89% 4.569 34.803 0.10% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.013ms; 6.015ms] or [-0.014%; +0.014%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 65.725µs 65.944µs ± 0.273µs 65.925µs ± 0.049µs 65.978µs 66.048µs 66.205µs 69.622µs 5.61% 12.317 163.464 0.41% 0.019µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [65.906µs; 65.982µs] or [-0.057%; +0.057%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 208.922µs 209.263µs ± 0.121µs 209.265µs ± 0.078µs 209.343µs 209.459µs 209.549µs 209.571µs 0.15% -0.005 -0.184 0.06% 0.009µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4771643.712op/s 4778670.496op/s ± 2772.922op/s 4778618.866op/s ± 1784.666op/s 4780414.006op/s 4783393.556op/s 4784698.458op/s 4786483.608op/s 0.16% 0.008 -0.184 0.06% 196.075op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.230µs 18.324µs ± 0.049µs 18.321µs ± 0.040µs 18.359µs 18.404µs 18.413µs 18.512µs 1.04% 0.235 -0.055 0.27% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 54019112.779op/s 54572563.077op/s ± 145444.427op/s 54581222.153op/s ± 118443.687op/s 54707820.587op/s 54831662.451op/s 54849479.137op/s 54855570.814op/s 0.50% -0.220 -0.088 0.27% 10284.474op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.663µs 10.728µs ± 0.030µs 10.722µs ± 0.018µs 10.748µs 10.774µs 10.808µs 10.891µs 1.58% 1.098 3.779 0.28% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 91817441.500op/s 93215917.655op/s ± 260512.556op/s 93267720.761op/s ± 156146.346op/s 93381312.252op/s 93584134.850op/s 93742126.116op/s 93781222.835op/s 0.55% -1.061 3.586 0.28% 18421.020op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [209.246µs; 209.280µs] or [-0.008%; +0.008%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4778286.196op/s; 4779054.796op/s] or [-0.008%; +0.008%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.318µs; 18.331µs] or [-0.037%; +0.037%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54552405.879op/s; 54592720.276op/s] or [-0.037%; +0.037%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.724µs; 10.732µs] or [-0.039%; +0.039%] None None None
normalization/normalize_name/normalize_name/good throughput [93179813.120op/s; 93252022.190op/s] or [-0.039%; +0.039%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.536µs 33.404µs ± 1.187µs 32.655µs ± 0.107µs 35.046µs 35.139µs 36.389µs 36.479µs 11.71% 0.990 -0.740 3.55% 0.084µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.240µs; 33.569µs] or [-0.493%; +0.493%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.898µs 3.917µs ± 0.002µs 3.917µs ± 0.001µs 3.918µs 3.921µs 3.922µs 3.926µs 0.22% -3.267 32.841 0.06% 0.000µs 1 200
credit_card/is_card_number/ throughput 254720510.838op/s 255267304.610op/s ± 142270.028op/s 255273757.858op/s ± 58043.358op/s 255326834.855op/s 255411429.635op/s 255534595.358op/s 256557826.054op/s 0.50% 3.307 33.245 0.06% 10060.010op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 82.156µs 82.581µs ± 0.407µs 82.417µs ± 0.103µs 82.706µs 83.261µs 84.270µs 84.934µs 3.05% 2.899 10.975 0.49% 0.029µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11773879.744op/s 12109681.772op/s ± 58872.219op/s 12133349.691op/s ± 15099.379op/s 12143658.889op/s 12155022.406op/s 12163522.491op/s 12171912.096op/s 0.32% -2.833 10.475 0.48% 4162.895op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 76.491µs 77.102µs ± 0.426µs 76.978µs ± 0.208µs 77.283µs 77.956µs 78.643µs 78.879µs 2.47% 1.690 3.357 0.55% 0.030µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 12677615.836op/s 12970208.086op/s ± 71084.809op/s 12990689.574op/s ± 35203.296op/s 13019253.855op/s 13043651.755op/s 13062279.894op/s 13073384.831op/s 0.64% -1.649 3.176 0.55% 5026.455op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.896µs 3.917µs ± 0.002µs 3.917µs ± 0.001µs 3.918µs 3.920µs 3.921µs 3.926µs 0.23% -3.916 44.754 0.06% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254714462.394op/s 255302423.900op/s ± 143058.860op/s 255308871.983op/s ± 51548.928op/s 255358521.789op/s 255442673.674op/s 255505724.778op/s 256696764.103op/s 0.54% 3.969 45.289 0.06% 10115.789op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 72.884µs 73.954µs ± 0.706µs 73.884µs ± 0.376µs 74.308µs 74.816µs 75.215µs 78.863µs 6.74% 3.116 18.720 0.95% 0.050µs 1 200
credit_card/is_card_number/378282246310005 throughput 12680288.529op/s 13523092.987op/s ± 125631.552op/s 13534799.565op/s ± 69322.484op/s 13602367.501op/s 13665961.799op/s 13690720.809op/s 13720390.942op/s 1.37% -2.817 16.147 0.93% 8883.492op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 52.029µs 52.256µs ± 0.088µs 52.248µs ± 0.058µs 52.315µs 52.392µs 52.514µs 52.563µs 0.60% 0.409 0.551 0.17% 0.006µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19024762.909op/s 19136664.896op/s ± 32221.935op/s 19139442.807op/s ± 21067.749op/s 19158267.068op/s 19186642.453op/s 19205943.197op/s 19220148.482op/s 0.42% -0.397 0.533 0.17% 2278.435op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.454µs 6.595µs ± 0.063µs 6.595µs ± 0.042µs 6.637µs 6.706µs 6.723µs 6.735µs 2.11% 0.167 -0.569 0.95% 0.004µs 1 200
credit_card/is_card_number/x371413321323331 throughput 148482850.871op/s 151649520.951op/s ± 1447406.676op/s 151619877.335op/s ± 954775.778op/s 152553774.701op/s 153866724.650op/s 154577624.038op/s 154939455.456op/s 2.19% -0.127 -0.575 0.95% 102347.108op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.898µs 3.917µs ± 0.002µs 3.917µs ± 0.001µs 3.918µs 3.920µs 3.923µs 3.931µs 0.36% -1.492 25.081 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254376389.409op/s 255281815.183op/s ± 158683.195op/s 255285708.588op/s ± 59914.396op/s 255343639.279op/s 255459235.892op/s 255549125.148op/s 256567226.942op/s 0.50% 1.538 25.325 0.06% 11220.596op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 63.778µs 63.914µs ± 0.094µs 63.893µs ± 0.040µs 63.944µs 64.070µs 64.306µs 64.433µs 0.84% 2.235 7.731 0.15% 0.007µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15520111.049op/s 15646077.213op/s ± 23016.207op/s 15651192.539op/s ± 9732.764op/s 15659891.104op/s 15671334.859op/s 15678930.528op/s 15679416.063op/s 0.18% -2.214 7.605 0.15% 1627.492op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.493µs 57.589µs ± 0.058µs 57.580µs ± 0.015µs 57.596µs 57.654µs 57.929µs 57.979µs 0.69% 4.154 23.470 0.10% 0.004µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17247642.946op/s 17364554.434op/s ± 17334.923op/s 17367254.910op/s ± 4651.730op/s 17371465.631op/s 17383714.067op/s 17387162.068op/s 17393352.261op/s 0.15% -4.129 23.262 0.10% 1225.764op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.897µs 3.917µs ± 0.002µs 3.917µs ± 0.001µs 3.918µs 3.920µs 3.922µs 3.933µs 0.39% -1.557 36.359 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254278200.861op/s 255289799.977op/s ± 150062.869op/s 255279839.780op/s ± 50416.761op/s 255342892.573op/s 255441948.705op/s 255524460.607op/s 256590245.170op/s 0.51% 1.620 36.614 0.06% 10611.047op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.660µs 54.898µs ± 0.301µs 54.719µs ± 0.036µs 55.135µs 55.582µs 55.701µs 56.032µs 2.40% 1.479 1.573 0.55% 0.021µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17846981.154op/s 18216235.252op/s ± 99068.971op/s 18275036.300op/s ± 12148.955op/s 18284070.004op/s 18289264.293op/s 18291330.147op/s 18294987.174op/s 0.11% -1.457 1.469 0.54% 7005.234op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 52.007µs 52.246µs ± 0.105µs 52.233µs ± 0.065µs 52.304µs 52.417µs 52.534µs 52.606µs 0.71% 0.424 0.545 0.20% 0.007µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19009357.059op/s 19140334.186op/s ± 38347.893op/s 19145059.429op/s ± 23777.920op/s 19164329.434op/s 19200497.101op/s 19224634.004op/s 19228134.607op/s 0.43% -0.410 0.521 0.20% 2711.606op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.472µs 6.615µs ± 0.054µs 6.605µs ± 0.037µs 6.644µs 6.704µs 6.750µs 6.793µs 2.85% 0.290 0.178 0.81% 0.004µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 147200504.732op/s 151189283.712op/s ± 1225411.085op/s 151395930.640op/s ± 853181.450op/s 151956387.619op/s 153126917.859op/s 153702142.192op/s 154512059.103op/s 2.06% -0.239 0.139 0.81% 86649.649op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.917µs; 3.918µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/ throughput [255247587.352op/s; 255287021.867op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [82.524µs; 82.637µs] or [-0.068%; +0.068%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12101522.649op/s; 12117840.896op/s] or [-0.067%; +0.067%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [77.043µs; 77.161µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/ 378282246310005 throughput [12960356.415op/s; 12980059.757op/s] or [-0.076%; +0.076%] None None None
credit_card/is_card_number/37828224631 execution_time [3.917µs; 3.917µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/37828224631 throughput [255282597.318op/s; 255322250.482op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/378282246310005 execution_time [73.856µs; 74.052µs] or [-0.132%; +0.132%] None None None
credit_card/is_card_number/378282246310005 throughput [13505681.662op/s; 13540504.312op/s] or [-0.129%; +0.129%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.244µs; 52.268µs] or [-0.023%; +0.023%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19132199.245op/s; 19141130.546op/s] or [-0.023%; +0.023%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.586µs; 6.603µs] or [-0.132%; +0.132%] None None None
credit_card/is_card_number/x371413321323331 throughput [151448924.306op/s; 151850117.596op/s] or [-0.132%; +0.132%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.917µs; 3.918µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ throughput [255259823.218op/s; 255303807.147op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [63.901µs; 63.927µs] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15642887.389op/s; 15649267.038op/s] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [57.581µs; 57.597µs] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17362151.980op/s; 17366956.888op/s] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.917µs; 3.917µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255269002.707op/s; 255310597.248op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.856µs; 54.939µs] or [-0.076%; +0.076%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18202505.245op/s; 18229965.258op/s] or [-0.075%; +0.075%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.231µs; 52.260µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19135019.537op/s; 19145648.835op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.607µs; 6.622µs] or [-0.113%; +0.113%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [151019453.521op/s; 151359113.902op/s] or [-0.112%; +0.112%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 244.036ns 254.412ns ± 12.725ns 247.727ns ± 2.375ns 259.414ns 284.534ns 292.958ns 293.713ns 18.56% 1.485 1.166 4.99% 0.900ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [252.649ns; 256.176ns] or [-0.693%; +0.693%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 504.990µs 506.105µs ± 0.369µs 506.094µs ± 0.250µs 506.349µs 506.689µs 506.852µs 507.838µs 0.34% 0.359 1.703 0.07% 0.026µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1969131.027op/s 1975877.074op/s ± 1441.341op/s 1975917.084op/s ± 976.174op/s 1976868.660op/s 1977956.423op/s 1979493.906op/s 1980235.707op/s 0.22% -0.351 1.680 0.07% 101.918op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 453.309µs 453.945µs ± 0.280µs 453.950µs ± 0.178µs 454.112µs 454.402µs 454.626µs 454.742µs 0.17% 0.192 -0.019 0.06% 0.020µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2199046.955op/s 2202912.598op/s ± 1358.471op/s 2202883.718op/s ± 863.780op/s 2203824.451op/s 2205228.146op/s 2205817.934op/s 2205999.060op/s 0.14% -0.188 -0.022 0.06% 96.058op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 175.367µs 176.706µs ± 0.388µs 176.754µs ± 0.266µs 176.983µs 177.254µs 177.366µs 177.587µs 0.47% -0.634 0.602 0.22% 0.027µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5631058.280op/s 5659144.429op/s ± 12452.328op/s 5657581.763op/s ± 8505.604op/s 5666315.321op/s 5680458.687op/s 5696330.155op/s 5702326.327op/s 0.79% 0.649 0.635 0.22% 880.513op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.561µs 37.666µs ± 0.068µs 37.662µs ± 0.030µs 37.688µs 37.721µs 37.796µs 38.266µs 1.60% 5.308 41.467 0.18% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26132926.956op/s 26549167.150op/s ± 47340.491op/s 26551947.845op/s ± 20883.709op/s 26574378.384op/s 26595480.469op/s 26610846.888op/s 26623079.114op/s 0.27% -5.226 40.561 0.18% 3347.478op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.269µs 48.361µs ± 0.107µs 48.349µs ± 0.029µs 48.383µs 48.422µs 48.498µs 49.733µs 2.86% 10.715 135.028 0.22% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20107534.179op/s 20678018.338op/s ± 44584.620op/s 20682853.822op/s ± 12475.879op/s 20693631.723op/s 20710896.095op/s 20715972.591op/s 20717137.852op/s 0.17% -10.568 132.494 0.22% 3152.609op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [506.053µs; 506.156µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1975677.318op/s; 1976076.830op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [453.906µs; 453.983µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2202724.327op/s; 2203100.869op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [176.652µs; 176.760µs] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5657418.656op/s; 5660870.202op/s] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.657µs; 37.675µs] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26542606.213op/s; 26555728.087op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.346µs; 48.376µs] or [-0.031%; +0.031%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20671839.338op/s; 20684197.337op/s] or [-0.030%; +0.030%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 150.240µs 151.026µs ± 0.330µs 150.976µs ± 0.154µs 151.139µs 151.576µs 152.412µs 152.513µs 1.02% 1.478 4.704 0.22% 0.023µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [150.980µs; 151.072µs] or [-0.030%; +0.030%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.318µs 2.388µs ± 0.017µs 2.387µs ± 0.004µs 2.394µs 2.421µs 2.427µs 2.439µs 2.18% -0.378 2.851 0.70% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.385µs; 2.390µs] or [-0.097%; +0.097%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.929µs 5.002µs ± 0.038µs 5.010µs ± 0.039µs 5.034µs 5.052µs 5.055µs 5.056µs 0.92% -0.164 -1.584 0.75% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [4.997µs; 5.007µs] or [-0.104%; +0.104%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 55.082ms 55.361ms ± 0.170ms 55.344ms ± 0.085ms 55.425ms 55.679ms 55.929ms 56.247ms 1.63% 1.795 4.948 0.31% 0.012ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [55.338ms; 55.385ms] or [-0.042%; +0.042%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.182µs 3.211µs ± 1.438µs 3.019µs ± 0.025µs 3.041µs 3.659µs 13.983µs 15.037µs 398.11% 7.419 55.897 44.66% 0.102µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.012µs; 3.410µs] or [-6.206%; +6.206%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a47b8cc 1742999987 PROF-11530-Fuzz-tests-failure-in-profiling-with-labels
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.607µs 26.209µs ± 11.472µs 18.184µs ± 0.375µs 35.324µs 45.881µs 47.599µs 91.474µs 403.05% 1.888 6.513 43.66% 0.811µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.619µs; 27.799µs] or [-6.066%; +6.066%] None None None

Baseline

Omitted due to size.

@r1viollet
Copy link
Contributor

r1viollet commented Mar 21, 2025 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 77.45 MB 77.51 MB +.07% (+58.58 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.92 MB 23.95 MB +.11% (+27.37 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.77 MB 7.77 MB +.05% (+4.17 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 22.48 MB 22.51 MB +.13% (+30.46 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 7.63 MB 7.70 MB +.87% (+68.40 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 71.66 MB 71.73 MB +.08% (+64.18 KB) 🔍
i686-alpine-linux-musl
Artifact Baseline Commit Change
/i686-alpine-linux-musl/lib/libdatadog_profiling.a 66.75 MB 66.81 MB +.08% (+58.25 KB) 🔍
/i686-alpine-linux-musl/lib/libdatadog_profiling.so 8.22 MB 8.22 MB +.07% (+6.48 KB) 🔍
/i686-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.08 MB 23.11 MB +.11% (+26.92 KB) 🔍
i686-unknown-linux-gnu
Artifact Baseline Commit Change
/i686-unknown-linux-gnu/lib/libdatadog_profiling.a 67.66 MB 67.72 MB +.09% (+63.94 KB) 🔍
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so 8.10 MB 8.10 MB +.08% (+6.69 KB) 🔍
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.70 MB 20.73 MB +.14% (+30.25 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 16.83 MB 16.85 MB +.09% (+17.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 61.18 KB 61.18 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 112.73 MB 112.93 MB +.17% (+200.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 628.59 MB 629.81 MB +.19% (+1.21 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 5.03 MB 5.03 MB +.06% (+3.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 61.18 KB 61.18 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 16.01 MB 16.03 MB +.09% (+16.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.62 MB 26.66 MB +.12% (+34.09 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.22 MB 14.24 MB +.12% (+17.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 62.13 KB 62.13 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 114.63 MB 114.89 MB +.22% (+264.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 621.24 MB 621.99 MB +.12% (+764.62 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.80 MB 3.80 MB +.10% (+4.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 62.13 KB 62.13 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.63 MB 16.65 MB +.09% (+16.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.52 MB 24.55 MB +.12% (+32.30 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 66.75 MB 66.81 MB +.08% (+58.25 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.22 MB 8.22 MB +.07% (+6.48 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.08 MB 23.11 MB +.11% (+26.92 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 67.66 MB 67.72 MB +.09% (+63.94 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.10 MB 8.10 MB +.08% (+6.69 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.70 MB 20.73 MB +.14% (+30.25 KB) 🔍

@morrisonlevi morrisonlevi force-pushed the PROF-11530-Fuzz-tests-failure-in-profiling-with-labels branch from 17b001c to 0b1769d Compare March 21, 2025 06:59
@codecov-commenter
Copy link

codecov-commenter commented Mar 21, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 76.10063% with 76 lines in your changes missing coverage. Please review.

Project coverage is 72.81%. Comparing base (dec5b9c) to head (a2d8627).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #959      +/-   ##
==========================================
+ Coverage   72.79%   72.81%   +0.01%     
==========================================
  Files         334      334              
  Lines       50903    50988      +85     
==========================================
+ Hits        37057    37126      +69     
- Misses      13846    13862      +16
Components Coverage Δ
crashtracker 42.85% <ø> (ø)
crashtracker-ffi 6.25% <ø> (ø)
datadog-alloc 98.73% <100.00%> (ø)
data-pipeline 91.81% <ø> (ø)
data-pipeline-ffi 90.29% <ø> (ø)
ddcommon 82.81% <ø> (ø)
ddcommon-ffi 70.13% <ø> (ø)
ddtelemetry 61.87% <ø> (ø)
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 89.60% <ø> (ø)
dogstatsd-client 82.57% <ø> (ø)
ipc 82.41% <ø> (ø)
profiling 81.65% <75.48%> (-0.12%) ⬇️
profiling-ffi 69.83% <13.04%> (-0.08%) ⬇️
serverless 0.00% <ø> (ø)
sidecar 40.85% <ø> (ø)
sidecar-ffi 0.29% <ø> (ø)
spawn-worker 54.37% <ø> (ø)
tinybytes 91.59% <ø> (ø)
trace-mini-agent 74.66% <ø> (ø)
trace-normalization 98.24% <ø> (ø)
trace-obfuscation 96.00% <ø> (ø)
trace-protobuf 78.13% <ø> (ø)
trace-utils 92.91% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ivoanjo
Copy link
Member

ivoanjo commented Mar 21, 2025

I hit a 3rd party license job failure. I updated the licenses, but it the CI seems to fetch different values. Not sure what's going on there; we'll see if it keeps happening.

Sorry if obvious, but I've been bit in the past by not having the same version of the licenses tool as CI -- running cargo install cargo-bundle-licenses before and making sure your version matches the CI version may help :)

@morrisonlevi
Copy link
Contributor Author

morrisonlevi commented Mar 21, 2025 •
edited
Loading

I know we pinned cargo-bundle-licenses in the past. I have pinned it again 😆
Hopefully this works now.

@morrisonlevi morrisonlevi changed the title fix(profiling): label handling fix(profiling): label handling and pin cargo-bundle-licenses Mar 21, 2025
@github-actions github-actions bot removed the ci-build label Mar 21, 2025
@morrisonlevi morrisonlevi changed the title fix(profiling): label handling and pin cargo-bundle-licenses fix(profiling): label handling and fuzzing Mar 21, 2025
@morrisonlevi morrisonlevi force-pushed the PROF-11530-Fuzz-tests-failure-in-profiling-with-labels branch from 25eac43 to 69994a9 Compare March 25, 2025 21:29
@morrisonlevi
fix(profiling): label handling and fuzzing
229cef5 
This is mostly a testing/fuzzing fix than a real error. But it did
catch one error in an existing test that went previously undetected.
@morrisonlevi morrisonlevi force-pushed the PROF-11530-Fuzz-tests-failure-in-profiling-with-labels branch from 69994a9 to 229cef5 Compare March 25, 2025 21:31
@morrisonlevi
Merge branch 'main' into PROF-11530-Fuzz-tests-failure-in-profiling-w…
dbee822 
…ith-labels
@morrisonlevi
test: skip certain tests on macos+miri
a47b8cc 
error: unsupported operation: can't call foreign function `kqueue` on OS `macos`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
data-pipeline mini-agent profiling Relates to the profiling* modules.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@morrisonlevi @r1viollet @codecov-commenter @ivoanjo