Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

sarahchen6 · 2025-12-02T17:27:24Z

What Does This Do

Now that our macrobenchmarks have been moved to the apm-sdks-benchmarks repo (https://github.com/DataDog/apm-sdks-benchmarks/pull/34), we can remove the local implementation.

Motivation

Additional Notes

To check this, see the manually triggered macrobenchmark runs in CI: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-java/-/pipelines/86577685. Note that we'll have to confirm that they're automatically run on master and when scheduled (ref).

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/APMLP-715

pr-commenter · 2025-12-02T18:47:56Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765570938	1765571318
git_commit_sha	`0f79569`	`1b6bc24`
release_version	1.57.0-SNAPSHOT~0f79569cdb	1.57.0-SNAPSHOT~1b6bc245d4

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765573175	1765573175
ci_job_id	1292232262	1292232262
ci_pipeline_id	86577685	86577685
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-xp57649s 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-xp57649s 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.095 s) : 0, 1095140
Total [baseline] (8.78 s) : 0, 8779619
Agent [candidate] (1.081 s) : 0, 1080842
Total [candidate] (8.73 s) : 0, 8729522
section iast
Agent [baseline] (1.222 s) : 0, 1221660
Total [baseline] (9.412 s) : 0, 9412395
Agent [candidate] (1.233 s) : 0, 1232758
Total [candidate] (9.413 s) : 0, 9412813

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.095 s	-
Agent	iast	1.222 s	126.52 ms (11.6%)
Total	tracing	8.78 s	-
Total	iast	9.412 s	632.776 ms (7.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.081 s	-
Agent	iast	1.233 s	151.916 ms (14.1%)
Total	tracing	8.73 s	-
Total	iast	9.413 s	683.291 ms (7.8%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.227 ms) : 0, 1227
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (658.567 ms) : 0, 658567
BytebuddyAgent [candidate] (648.938 ms) : 0, 648938
GlobalTracer [baseline] (284.658 ms) : 0, 284658
GlobalTracer [candidate] (282.037 ms) : 0, 282037
AppSec [baseline] (33.069 ms) : 0, 33069
AppSec [candidate] (32.46 ms) : 0, 32460
Debugger [baseline] (68.152 ms) : 0, 68152
Debugger [candidate] (67.134 ms) : 0, 67134
Remote Config [baseline] (654.043 µs) : 0, 654
Remote Config [candidate] (654.225 µs) : 0, 654
Telemetry [baseline] (9.155 ms) : 0, 9155
Telemetry [candidate] (9.105 ms) : 0, 9105
Flare Poller [baseline] (3.758 ms) : 0, 3758
Flare Poller [candidate] (3.755 ms) : 0, 3755
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (790.521 ms) : 0, 790521
BytebuddyAgent [candidate] (796.04 ms) : 0, 796040
GlobalTracer [baseline] (255.63 ms) : 0, 255630
GlobalTracer [candidate] (259.356 ms) : 0, 259356
IAST [baseline] (26.762 ms) : 0, 26762
IAST [candidate] (27.344 ms) : 0, 27344
AppSec [baseline] (34.538 ms) : 0, 34538
AppSec [candidate] (34.466 ms) : 0, 34466
Debugger [baseline] (65.081 ms) : 0, 65081
Debugger [candidate] (66.05 ms) : 0, 66050
Remote Config [baseline] (566.886 µs) : 0, 567
Remote Config [candidate] (578.733 µs) : 0, 579
Telemetry [baseline] (8.439 ms) : 0, 8439
Telemetry [candidate] (8.581 ms) : 0, 8581
Flare Poller [baseline] (3.478 ms) : 0, 3478
Flare Poller [candidate] (3.596 ms) : 0, 3596

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1084117
Total [baseline] (10.804 s) : 0, 10803982
Agent [candidate] (1.084 s) : 0, 1084050
Total [candidate] (11.017 s) : 0, 11017011
section appsec
Agent [baseline] (1.273 s) : 0, 1272852
Total [baseline] (11.137 s) : 0, 11137473
Agent [candidate] (1.266 s) : 0, 1266401
Total [candidate] (10.923 s) : 0, 10923174
section iast
Agent [baseline] (1.222 s) : 0, 1221512
Total [baseline] (11.245 s) : 0, 11245256
Agent [candidate] (1.232 s) : 0, 1232214
Total [candidate] (11.248 s) : 0, 11247963
section profiling
Agent [baseline] (1.218 s) : 0, 1217831
Total [baseline] (10.865 s) : 0, 10865045
Agent [candidate] (1.204 s) : 0, 1203921
Total [candidate] (10.944 s) : 0, 10943645

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	appsec	1.273 s	188.734 ms (17.4%)
Agent	iast	1.222 s	137.394 ms (12.7%)
Agent	profiling	1.218 s	133.713 ms (12.3%)
Total	tracing	10.804 s	-
Total	appsec	11.137 s	333.492 ms (3.1%)
Total	iast	11.245 s	441.274 ms (4.1%)
Total	profiling	10.865 s	61.063 ms (0.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	appsec	1.266 s	182.351 ms (16.8%)
Agent	iast	1.232 s	148.164 ms (13.7%)
Agent	profiling	1.204 s	119.872 ms (11.1%)
Total	tracing	11.017 s	-
Total	appsec	10.923 s	-93.837 ms (-0.9%)
Total	iast	11.248 s	230.951 ms (2.1%)
Total	profiling	10.944 s	-73.367 ms (-0.7%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (650.488 ms) : 0, 650488
BytebuddyAgent [candidate] (650.497 ms) : 0, 650497
GlobalTracer [baseline] (282.52 ms) : 0, 282520
GlobalTracer [candidate] (282.453 ms) : 0, 282453
AppSec [baseline] (32.5 ms) : 0, 32500
AppSec [candidate] (32.558 ms) : 0, 32558
Debugger [baseline] (68.217 ms) : 0, 68217
Debugger [candidate] (68.185 ms) : 0, 68185
Remote Config [baseline] (651.585 µs) : 0, 652
Remote Config [candidate] (678.081 µs) : 0, 678
Telemetry [baseline] (9.134 ms) : 0, 9134
Telemetry [candidate] (9.147 ms) : 0, 9147
Flare Poller [baseline] (3.741 ms) : 0, 3741
Flare Poller [candidate] (3.783 ms) : 0, 3783
section appsec
crashtracking [baseline] (1.186 ms) : 0, 1186
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (692.832 ms) : 0, 692832
BytebuddyAgent [candidate] (689.495 ms) : 0, 689495
GlobalTracer [baseline] (260.448 ms) : 0, 260448
GlobalTracer [candidate] (259.097 ms) : 0, 259097
AppSec [baseline] (176.156 ms) : 0, 176156
AppSec [candidate] (174.698 ms) : 0, 174698
Debugger [baseline] (67.659 ms) : 0, 67659
Debugger [candidate] (68.129 ms) : 0, 68129
Remote Config [baseline] (745.591 µs) : 0, 746
Remote Config [candidate] (733.929 µs) : 0, 734
Telemetry [baseline] (9.243 ms) : 0, 9243
Telemetry [candidate] (9.063 ms) : 0, 9063
Flare Poller [baseline] (3.975 ms) : 0, 3975
Flare Poller [candidate] (3.78 ms) : 0, 3780
IAST [baseline] (24.836 ms) : 0, 24836
IAST [candidate] (24.553 ms) : 0, 24553
section iast
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (789.387 ms) : 0, 789387
BytebuddyAgent [candidate] (796.455 ms) : 0, 796455
GlobalTracer [baseline] (255.605 ms) : 0, 255605
GlobalTracer [candidate] (257.761 ms) : 0, 257761
AppSec [baseline] (35.178 ms) : 0, 35178
AppSec [candidate] (35.276 ms) : 0, 35276
Debugger [baseline] (65.187 ms) : 0, 65187
Debugger [candidate] (66.162 ms) : 0, 66162
Remote Config [baseline] (553.447 µs) : 0, 553
Remote Config [candidate] (548.008 µs) : 0, 548
Telemetry [baseline] (8.475 ms) : 0, 8475
Telemetry [candidate] (8.525 ms) : 0, 8525
Flare Poller [baseline] (3.448 ms) : 0, 3448
Flare Poller [candidate] (3.507 ms) : 0, 3507
IAST [baseline] (26.995 ms) : 0, 26995
IAST [candidate] (27.201 ms) : 0, 27201
section profiling
ProfilingAgent [baseline] (97.065 ms) : 0, 97065
ProfilingAgent [candidate] (97.207 ms) : 0, 97207
crashtracking [baseline] (1.232 ms) : 0, 1232
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (710.516 ms) : 0, 710516
BytebuddyAgent [candidate] (701.56 ms) : 0, 701560
GlobalTracer [baseline] (223.082 ms) : 0, 223082
GlobalTracer [candidate] (220.399 ms) : 0, 220399
AppSec [baseline] (32.703 ms) : 0, 32703
AppSec [candidate] (31.963 ms) : 0, 31963
Debugger [baseline] (68.939 ms) : 0, 68939
Debugger [candidate] (68.316 ms) : 0, 68316
Remote Config [baseline] (660.647 µs) : 0, 661
Remote Config [candidate] (618.248 µs) : 0, 618
Telemetry [baseline] (8.983 ms) : 0, 8983
Telemetry [candidate] (9.132 ms) : 0, 9132
Flare Poller [baseline] (3.829 ms) : 0, 3829
Flare Poller [candidate] (3.784 ms) : 0, 3784
Profiling [baseline] (97.667 ms) : 0, 97667
Profiling [candidate] (97.769 ms) : 0, 97769

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765570938	1765571318
git_commit_sha	`0f79569`	`1b6bc24`
release_version	1.57.0-SNAPSHOT~0f79569cdb	1.57.0-SNAPSHOT~1b6bc245d4

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765573681	1765573681
ci_job_id	1292232263	1292232263
ci_pipeline_id	86577685	86577685
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-bgrlfb0v 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-bgrlfb0v 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:no_agent:high_load	worse [+0.545ms; +2.193ms] or [+3.093%; +12.454%]	unstable [+0.559ms; +3.902ms] or [+1.911%; +13.345%]	unstable [-44.140op/s; +7.140op/s] or [-17.063%; +2.760%]	18.975ms	31.469ms	240.188op/s	17.606ms	29.238ms	258.688op/s
scenario:load:petclinic:tracing:high_load	better [-1.533ms; -0.630ms] or [-8.355%; -3.431%]	unsure [-1655.240µs; -147.008µs] or [-5.619%; -0.499%]	unstable [-12.428op/s; +36.865op/s] or [-4.945%; +14.669%]	17.269ms	28.557ms	263.531op/s	18.350ms	29.458ms	251.312op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.189 ms) : 1177, 1200
.   : milestone, 1189,
iast (3.255 ms) : 3209, 3300
.   : milestone, 3255,
iast_FULL (5.767 ms) : 5711, 5824
.   : milestone, 5767,
iast_GLOBAL (3.655 ms) : 3603, 3708
.   : milestone, 3655,
profiling (2.104 ms) : 2085, 2124
.   : milestone, 2104,
tracing (1.784 ms) : 1769, 1799
.   : milestone, 1784,
section candidate
no_agent (1.225 ms) : 1212, 1238
.   : milestone, 1225,
iast (3.299 ms) : 3251, 3347
.   : milestone, 3299,
iast_FULL (5.634 ms) : 5578, 5689
.   : milestone, 5634,
iast_GLOBAL (3.617 ms) : 3564, 3669
.   : milestone, 3617,
profiling (1.969 ms) : 1953, 1986
.   : milestone, 1969,
tracing (1.848 ms) : 1830, 1866
.   : milestone, 1848,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.189 ms [1.177 ms, 1.2 ms]	-
iast	3.255 ms [3.209 ms, 3.3 ms]	2.066 ms (173.8%)
iast_FULL	5.767 ms [5.711 ms, 5.824 ms]	4.579 ms (385.1%)
iast_GLOBAL	3.655 ms [3.603 ms, 3.708 ms]	2.466 ms (207.5%)
profiling	2.104 ms [2.085 ms, 2.124 ms]	915.571 µs (77.0%)
tracing	1.784 ms [1.769 ms, 1.799 ms]	595.483 µs (50.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.225 ms [1.212 ms, 1.238 ms]	-
iast	3.299 ms [3.251 ms, 3.347 ms]	2.074 ms (169.3%)
iast_FULL	5.634 ms [5.578 ms, 5.689 ms]	4.409 ms (359.9%)
iast_GLOBAL	3.617 ms [3.564 ms, 3.669 ms]	2.392 ms (195.2%)
profiling	1.969 ms [1.953 ms, 1.986 ms]	744.541 µs (60.8%)
tracing	1.848 ms [1.83 ms, 1.866 ms]	622.73 µs (50.8%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.038 ms) : 17855, 18221
.   : milestone, 18038,
appsec (18.74 ms) : 18548, 18932
.   : milestone, 18740,
code_origins (17.731 ms) : 17554, 17907
.   : milestone, 17731,
iast (17.621 ms) : 17446, 17796
.   : milestone, 17621,
profiling (18.414 ms) : 18231, 18597
.   : milestone, 18414,
tracing (18.57 ms) : 18384, 18756
.   : milestone, 18570,
section candidate
no_agent (19.436 ms) : 19235, 19636
.   : milestone, 19436,
appsec (18.691 ms) : 18499, 18883
.   : milestone, 18691,
code_origins (17.673 ms) : 17499, 17847
.   : milestone, 17673,
iast (17.61 ms) : 17433, 17787
.   : milestone, 17610,
profiling (18.691 ms) : 18506, 18876
.   : milestone, 18691,
tracing (17.704 ms) : 17529, 17879
.   : milestone, 17704,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.038 ms [17.855 ms, 18.221 ms]	-
appsec	18.74 ms [18.548 ms, 18.932 ms]	701.981 µs (3.9%)
code_origins	17.731 ms [17.554 ms, 17.907 ms]	-307.223 µs (-1.7%)
iast	17.621 ms [17.446 ms, 17.796 ms]	-417.175 µs (-2.3%)
profiling	18.414 ms [18.231 ms, 18.597 ms]	376.068 µs (2.1%)
tracing	18.57 ms [18.384 ms, 18.756 ms]	531.888 µs (2.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.436 ms [19.235 ms, 19.636 ms]	-
appsec	18.691 ms [18.499 ms, 18.883 ms]	-744.89 µs (-3.8%)
code_origins	17.673 ms [17.499 ms, 17.847 ms]	-1.762 ms (-9.1%)
iast	17.61 ms [17.433 ms, 17.787 ms]	-1.826 ms (-9.4%)
profiling	18.691 ms [18.506 ms, 18.876 ms]	-744.252 µs (-3.8%)
tracing	17.704 ms [17.529 ms, 17.879 ms]	-1.732 ms (-8.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765570938	1765571318
git_commit_sha	`0f79569`	`1b6bc24`
release_version	1.57.0-SNAPSHOT~0f79569cdb	1.57.0-SNAPSHOT~1b6bc245d4

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1765573381	1765573381
ci_job_id	1292232264	1292232264
ci_pipeline_id	86577685	86577685
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-ddysp1lm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-ddysp1lm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.370ms; -1.035ms] or [-37.202%; -28.108%]	2.480ms	3.683ms

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1471, 1494
.   : milestone, 1482,
appsec (3.683 ms) : 3469, 3896
.   : milestone, 3683,
iast (2.235 ms) : 2170, 2300
.   : milestone, 2235,
iast_GLOBAL (2.276 ms) : 2211, 2342
.   : milestone, 2276,
profiling (2.086 ms) : 2033, 2139
.   : milestone, 2086,
tracing (2.061 ms) : 2010, 2112
.   : milestone, 2061,
section candidate
no_agent (1.482 ms) : 1470, 1493
.   : milestone, 1482,
appsec (2.48 ms) : 2428, 2532
.   : milestone, 2480,
iast (2.232 ms) : 2167, 2296
.   : milestone, 2232,
iast_GLOBAL (2.27 ms) : 2205, 2335
.   : milestone, 2270,
profiling (2.081 ms) : 2028, 2134
.   : milestone, 2081,
tracing (2.063 ms) : 2012, 2114
.   : milestone, 2063,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.482 ms [1.471 ms, 1.494 ms]	-
appsec	3.683 ms [3.469 ms, 3.896 ms]	2.2 ms (148.4%)
iast	2.235 ms [2.17 ms, 2.3 ms]	752.9 µs (50.8%)
iast_GLOBAL	2.276 ms [2.211 ms, 2.342 ms]	794.185 µs (53.6%)
profiling	2.086 ms [2.033 ms, 2.139 ms]	603.888 µs (40.7%)
tracing	2.061 ms [2.01 ms, 2.112 ms]	578.37 µs (39.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.482 ms [1.47 ms, 1.493 ms]	-
appsec	2.48 ms [2.428 ms, 2.532 ms]	998.322 µs (67.4%)
iast	2.232 ms [2.167 ms, 2.296 ms]	749.952 µs (50.6%)
iast_GLOBAL	2.27 ms [2.205 ms, 2.335 ms]	788.188 µs (53.2%)
profiling	2.081 ms [2.028 ms, 2.134 ms]	599.297 µs (40.4%)
tracing	2.063 ms [2.012 ms, 2.114 ms]	580.97 µs (39.2%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~1b6bc245d4, baseline=1.57.0-SNAPSHOT~0f79569cdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.03 s) : 15030000, 15030000
.   : milestone, 15030000,
appsec (14.89 s) : 14890000, 14890000
.   : milestone, 14890000,
iast (18.304 s) : 18304000, 18304000
.   : milestone, 18304000,
iast_GLOBAL (17.858 s) : 17858000, 17858000
.   : milestone, 17858000,
profiling (14.688 s) : 14688000, 14688000
.   : milestone, 14688000,
tracing (14.656 s) : 14656000, 14656000
.   : milestone, 14656000,
section candidate
no_agent (15.429 s) : 15429000, 15429000
.   : milestone, 15429000,
appsec (14.459 s) : 14459000, 14459000
.   : milestone, 14459000,
iast (17.963 s) : 17963000, 17963000
.   : milestone, 17963000,
iast_GLOBAL (17.821 s) : 17821000, 17821000
.   : milestone, 17821000,
profiling (14.661 s) : 14661000, 14661000
.   : milestone, 14661000,
tracing (14.833 s) : 14833000, 14833000
.   : milestone, 14833000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.03 s [15.03 s, 15.03 s]	-
appsec	14.89 s [14.89 s, 14.89 s]	-140.0 ms (-0.9%)
iast	18.304 s [18.304 s, 18.304 s]	3.274 s (21.8%)
iast_GLOBAL	17.858 s [17.858 s, 17.858 s]	2.828 s (18.8%)
profiling	14.688 s [14.688 s, 14.688 s]	-342.0 ms (-2.3%)
tracing	14.656 s [14.656 s, 14.656 s]	-374.0 ms (-2.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.429 s [15.429 s, 15.429 s]	-
appsec	14.459 s [14.459 s, 14.459 s]	-970.0 ms (-6.3%)
iast	17.963 s [17.963 s, 17.963 s]	2.534 s (16.4%)
iast_GLOBAL	17.821 s [17.821 s, 17.821 s]	2.392 s (15.5%)
profiling	14.661 s [14.661 s, 14.661 s]	-768.0 ms (-5.0%)
tracing	14.833 s [14.833 s, 14.833 s]	-596.0 ms (-3.9%)

sarahchen6 force-pushed the sarahchen6/move-macrobenchmarks branch from f391147 to 9602f73 Compare December 2, 2025 17:28

sarahchen6 added 2 commits December 2, 2025 11:43

Use apm-sdks-benchmarks macrobenchmarks

d7ff239

Move to gitlab-ci file

b376785

sarahchen6 force-pushed the sarahchen6/move-macrobenchmarks branch from ba1ae21 to b376785 Compare December 2, 2025 17:44

sarahchen6 added 2 commits December 2, 2025 11:46

Test main

bf66715

Fix includes and stages

ddf61d5

sarahchen6 added 3 commits December 2, 2025 17:31

Delete macrobenchmarks file

d69aeaf

Empty commit to run benchmarks again

a9e13c9

Empty commit to run benchmarks again 2

4a173b8

sarahchen6 temporarily deployed to ci-build December 3, 2025 17:59 — with GitHub Actions Inactive

Empty commit to run benchmarks again 3

de5ab06

sarahchen6 temporarily deployed to ci-build December 4, 2025 14:33 — with GitHub Actions Inactive

Empty commit to run benchmarks again 4

6c41b8c

sarahchen6 temporarily deployed to ci-build December 4, 2025 14:48 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

e1a52bc

sarahchen6 temporarily deployed to ci-build December 5, 2025 15:38 — with GitHub Actions Inactive

Exclude startup benchmarks from slo check for now

bd827c3

sarahchen6 temporarily deployed to ci-build December 5, 2025 16:24 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

39bf2ea

sarahchen6 temporarily deployed to ci-build December 5, 2025 19:54 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

4485a05

sarahchen6 temporarily deployed to ci-build December 8, 2025 20:51 — with GitHub Actions Inactive

Move slo to separate file

555700c

sarahchen6 temporarily deployed to ci-build December 10, 2025 18:29 — with GitHub Actions Inactive

sarahchen6 added 2 commits December 11, 2025 13:50

Fix space

632929e

Merge branch 'master' into sarahchen6/move-macrobenchmarks

853b8ab

sarahchen6 had a problem deploying to ci-build December 11, 2025 18:52 — with GitHub Actions Error

Switch ref branch back to main

c72ab1f

sarahchen6 temporarily deployed to ci-build December 11, 2025 18:56 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

1b6bc24

sarahchen6 temporarily deployed to ci-build December 12, 2025 20:28 — with GitHub Actions Inactive

sarahchen6 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Dec 12, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

Uh oh!

sarahchen6 commented Dec 2, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Dec 2, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Move macrobenchmarks to apm-sdks-benchmarks repo #10066

Are you sure you want to change the base?

Move macrobenchmarks to apm-sdks-benchmarks repo #10066

Uh oh!

Conversation

sarahchen6 commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

sarahchen6 commented Dec 2, 2025 •

edited

Loading

pr-commenter bot commented Dec 2, 2025 •

edited

Loading