Skip to content

Conversation

sarahchen6
Copy link
Contributor

@sarahchen6 sarahchen6 commented Jul 2, 2025

What Does This Do

Test openjdk ea image for Java 25: DataDog/dd-trace-java-docker-build#111

Also prints the java version being run for :dd-java-agent:instrumentation:java-lang:latestDepTest on test_inst_latest: [X, 3/6].

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@sarahchen6 sarahchen6 added the tag: do not merge Do not merge changes label Jul 2, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jul 2, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/test-jdk-25-ea
git_commit_date 1752084850 1752113302
git_commit_sha 89e92d5 4755b8d
release_version 1.52.0-SNAPSHOT~89e92d59f6 1.52.0-SNAPSHOT~4755b8d197
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1752115220 1752115220
ci_job_id 1022080875 1022080875
ci_pipeline_id 70129933 70129933
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-bqi3v0ht 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-bqi3v0ht 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 43 metrics, 10 unstable metrics.

Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (996.312 ms) : 0, 996312
Total [baseline] (8.61 s) : 0, 8609854
Agent [candidate] (997.228 ms) : 0, 997228
Total [candidate] (8.62 s) : 0, 8620105
section iast
Agent [baseline] (1.142 s) : 0, 1142082
Total [baseline] (9.308 s) : 0, 9308230
Agent [candidate] (1.151 s) : 0, 1151141
Total [candidate] (9.336 s) : 0, 9336169
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 996.312 ms -
Agent iast 1.142 s 145.77 ms (14.6%)
Total tracing 8.61 s -
Total iast 9.308 s 698.376 ms (8.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 997.228 ms -
Agent iast 1.151 s 153.913 ms (15.4%)
Total tracing 8.62 s -
Total iast 9.336 s 716.064 ms (8.3%)
gantt
    title insecure-bank - break down per module: candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.87 ms) : 0, 687870
BytebuddyAgent [candidate] (688.386 ms) : 0, 688386
GlobalTracer [baseline] (242.392 ms) : 0, 242392
GlobalTracer [candidate] (242.856 ms) : 0, 242856
AppSec [baseline] (30.269 ms) : 0, 30269
AppSec [candidate] (30.314 ms) : 0, 30314
Debugger [baseline] (5.996 ms) : 0, 5996
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (689.512 µs) : 0, 690
Remote Config [candidate] (679.18 µs) : 0, 679
Telemetry [baseline] (8.279 ms) : 0, 8279
Telemetry [candidate] (8.232 ms) : 0, 8232
section iast
BytebuddyAgent [baseline] (814.878 ms) : 0, 814878
BytebuddyAgent [candidate] (822.114 ms) : 0, 822114
GlobalTracer [baseline] (233.935 ms) : 0, 233935
GlobalTracer [candidate] (234.991 ms) : 0, 234991
AppSec [baseline] (29.159 ms) : 0, 29159
AppSec [candidate] (29.269 ms) : 0, 29269
Debugger [baseline] (5.764 ms) : 0, 5764
Debugger [candidate] (6.695 ms) : 0, 6695
Remote Config [baseline] (585.895 µs) : 0, 586
Remote Config [candidate] (593.251 µs) : 0, 593
Telemetry [baseline] (7.92 ms) : 0, 7920
Telemetry [candidate] (8.046 ms) : 0, 8046
IAST [baseline] (28.922 ms) : 0, 28922
IAST [candidate] (27.528 ms) : 0, 27528
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.004 s) : 0, 1004493
Total [baseline] (10.705 s) : 0, 10704793
Agent [candidate] (997.05 ms) : 0, 997050
Total [candidate] (10.691 s) : 0, 10690955
section appsec
Agent [baseline] (1.179 s) : 0, 1178677
Total [baseline] (10.827 s) : 0, 10826515
Agent [candidate] (1.177 s) : 0, 1176888
Total [candidate] (10.797 s) : 0, 10797126
section iast
Agent [baseline] (1.133 s) : 0, 1132914
Total [baseline] (10.9 s) : 0, 10899766
Agent [candidate] (1.14 s) : 0, 1139981
Total [candidate] (10.914 s) : 0, 10914430
section profiling
Agent [baseline] (1.244 s) : 0, 1244255
Total [baseline] (10.934 s) : 0, 10934303
Agent [candidate] (1.245 s) : 0, 1244777
Total [candidate] (11.009 s) : 0, 11008655
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.004 s -
Agent appsec 1.179 s 174.185 ms (17.3%)
Agent iast 1.133 s 128.422 ms (12.8%)
Agent profiling 1.244 s 239.762 ms (23.9%)
Total tracing 10.705 s -
Total appsec 10.827 s 121.722 ms (1.1%)
Total iast 10.9 s 194.972 ms (1.8%)
Total profiling 10.934 s 229.51 ms (2.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 997.05 ms -
Agent appsec 1.177 s 179.838 ms (18.0%)
Agent iast 1.14 s 142.931 ms (14.3%)
Agent profiling 1.245 s 247.726 ms (24.8%)
Total tracing 10.691 s -
Total appsec 10.797 s 106.171 ms (1.0%)
Total iast 10.914 s 223.475 ms (2.1%)
Total profiling 11.009 s 317.7 ms (3.0%)
gantt
    title petclinic - break down per module: candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.613 ms) : 0, 693613
BytebuddyAgent [candidate] (687.814 ms) : 0, 687814
GlobalTracer [baseline] (244.212 ms) : 0, 244212
GlobalTracer [candidate] (243.024 ms) : 0, 243024
AppSec [baseline] (30.572 ms) : 0, 30572
AppSec [candidate] (30.374 ms) : 0, 30374
Debugger [baseline] (6.029 ms) : 0, 6029
Debugger [candidate] (6.029 ms) : 0, 6029
Remote Config [baseline] (693.173 µs) : 0, 693
Remote Config [candidate] (677.253 µs) : 0, 677
Telemetry [baseline] (8.312 ms) : 0, 8312
Telemetry [candidate] (8.239 ms) : 0, 8239
section appsec
BytebuddyAgent [baseline] (711.844 ms) : 0, 711844
BytebuddyAgent [candidate] (711.528 ms) : 0, 711528
GlobalTracer [baseline] (236.296 ms) : 0, 236296
GlobalTracer [candidate] (235.815 ms) : 0, 235815
IAST [baseline] (23.265 ms) : 0, 23265
IAST [candidate] (23.244 ms) : 0, 23244
AppSec [baseline] (171.905 ms) : 0, 171905
AppSec [candidate] (170.958 ms) : 0, 170958
Debugger [baseline] (5.716 ms) : 0, 5716
Debugger [candidate] (5.72 ms) : 0, 5720
Remote Config [baseline] (604.255 µs) : 0, 604
Remote Config [candidate] (605.788 µs) : 0, 606
Telemetry [baseline] (8.125 ms) : 0, 8125
Telemetry [candidate] (8.064 ms) : 0, 8064
section iast
BytebuddyAgent [baseline] (807.468 ms) : 0, 807468
BytebuddyAgent [candidate] (813.208 ms) : 0, 813208
GlobalTracer [baseline] (232.457 ms) : 0, 232457
GlobalTracer [candidate] (233.553 ms) : 0, 233553
IAST [baseline] (27.117 ms) : 0, 27117
IAST [candidate] (26.39 ms) : 0, 26390
AppSec [baseline] (29.906 ms) : 0, 29906
AppSec [candidate] (31.423 ms) : 0, 31423
Debugger [baseline] (6.615 ms) : 0, 6615
Debugger [candidate] (5.773 ms) : 0, 5773
Remote Config [baseline] (577.022 µs) : 0, 577
Remote Config [candidate] (579.416 µs) : 0, 579
Telemetry [baseline] (7.897 ms) : 0, 7897
Telemetry [candidate] (8.011 ms) : 0, 8011
section profiling
BytebuddyAgent [baseline] (678.417 ms) : 0, 678417
BytebuddyAgent [candidate] (677.583 ms) : 0, 677583
GlobalTracer [baseline] (361.183 ms) : 0, 361183
GlobalTracer [candidate] (361.671 ms) : 0, 361671
AppSec [baseline] (31.593 ms) : 0, 31593
AppSec [candidate] (30.774 ms) : 0, 30774
Debugger [baseline] (12.014 ms) : 0, 12014
Debugger [candidate] (11.885 ms) : 0, 11885
Remote Config [baseline] (663.714 µs) : 0, 664
Remote Config [candidate] (661.44 µs) : 0, 661
Telemetry [baseline] (8.732 ms) : 0, 8732
Telemetry [candidate] (9.654 ms) : 0, 9654
ProfilingAgent [baseline] (103.048 ms) : 0, 103048
ProfilingAgent [candidate] (103.994 ms) : 0, 103994
Profiling [baseline] (103.072 ms) : 0, 103072
Profiling [candidate] (104.019 ms) : 0, 104019
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/test-jdk-25-ea
git_commit_date 1752084850 1752113302
git_commit_sha 89e92d5 4755b8d
release_version 1.52.0-SNAPSHOT~89e92d59f6 1.52.0-SNAPSHOT~4755b8d197
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1752114887 1752114887
ci_job_id 1022080876 1022080876
ci_pipeline_id 70129933 70129933
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-vtqwziog 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-vtqwziog 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 12 unstable metrics.

Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.406 ms) : 4357, 4455
.   : milestone, 4406,
iast (9.403 ms) : 9249, 9557
.   : milestone, 9403,
iast_FULL (13.935 ms) : 13658, 14213
.   : milestone, 13935,
iast_GLOBAL (10.418 ms) : 10232, 10605
.   : milestone, 10418,
profiling (9.132 ms) : 8990, 9274
.   : milestone, 9132,
tracing (7.598 ms) : 7490, 7706
.   : milestone, 7598,
section candidate
no_agent (4.34 ms) : 4292, 4387
.   : milestone, 4340,
iast (9.599 ms) : 9443, 9756
.   : milestone, 9599,
iast_FULL (14.066 ms) : 13780, 14351
.   : milestone, 14066,
iast_GLOBAL (10.304 ms) : 10110, 10498
.   : milestone, 10304,
profiling (9.473 ms) : 9284, 9663
.   : milestone, 9473,
tracing (7.748 ms) : 7623, 7874
.   : milestone, 7748,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.406 ms [4.357 ms, 4.455 ms] -
iast 9.403 ms [9.249 ms, 9.557 ms] 4.997 ms (113.4%)
iast_FULL 13.935 ms [13.658 ms, 14.213 ms] 9.529 ms (216.3%)
iast_GLOBAL 10.418 ms [10.232 ms, 10.605 ms] 6.012 ms (136.5%)
profiling 9.132 ms [8.99 ms, 9.274 ms] 4.726 ms (107.3%)
tracing 7.598 ms [7.49 ms, 7.706 ms] 3.192 ms (72.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.34 ms [4.292 ms, 4.387 ms] -
iast 9.599 ms [9.443 ms, 9.756 ms] 5.26 ms (121.2%)
iast_FULL 14.066 ms [13.78 ms, 14.351 ms] 9.726 ms (224.1%)
iast_GLOBAL 10.304 ms [10.11 ms, 10.498 ms] 5.964 ms (137.4%)
profiling 9.473 ms [9.284 ms, 9.663 ms] 5.133 ms (118.3%)
tracing 7.748 ms [7.623 ms, 7.874 ms] 3.409 ms (78.5%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.834 ms) : 37542, 38127
.   : milestone, 37834,
appsec (47.739 ms) : 47328, 48150
.   : milestone, 47739,
code_origins (45.404 ms) : 45020, 45788
.   : milestone, 45404,
iast (43.974 ms) : 43589, 44360
.   : milestone, 43974,
profiling (48.123 ms) : 47694, 48552
.   : milestone, 48123,
tracing (43.623 ms) : 43256, 43991
.   : milestone, 43623,
section candidate
no_agent (38.199 ms) : 37886, 38513
.   : milestone, 38199,
appsec (47.094 ms) : 46691, 47497
.   : milestone, 47094,
code_origins (45.906 ms) : 45497, 46316
.   : milestone, 45906,
iast (44.25 ms) : 43878, 44622
.   : milestone, 44250,
profiling (48.115 ms) : 47654, 48576
.   : milestone, 48115,
tracing (43.449 ms) : 43097, 43801
.   : milestone, 43449,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.834 ms [37.542 ms, 38.127 ms] -
appsec 47.739 ms [47.328 ms, 48.15 ms] 9.905 ms (26.2%)
code_origins 45.404 ms [45.02 ms, 45.788 ms] 7.57 ms (20.0%)
iast 43.974 ms [43.589 ms, 44.36 ms] 6.14 ms (16.2%)
profiling 48.123 ms [47.694 ms, 48.552 ms] 10.289 ms (27.2%)
tracing 43.623 ms [43.256 ms, 43.991 ms] 5.789 ms (15.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 38.199 ms [37.886 ms, 38.513 ms] -
appsec 47.094 ms [46.691 ms, 47.497 ms] 8.895 ms (23.3%)
code_origins 45.906 ms [45.497 ms, 46.316 ms] 7.707 ms (20.2%)
iast 44.25 ms [43.878 ms, 44.622 ms] 6.051 ms (15.8%)
profiling 48.115 ms [47.654 ms, 48.576 ms] 9.916 ms (26.0%)
tracing 43.449 ms [43.097 ms, 43.801 ms] 5.25 ms (13.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/test-jdk-25-ea
git_commit_date 1752084850 1752113302
git_commit_sha 89e92d5 4755b8d
release_version 1.52.0-SNAPSHOT~89e92d59f6 1.52.0-SNAPSHOT~4755b8d197
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1752115423 1752115423
ci_job_id 1022080877 1022080877
ci_pipeline_id 70129933 70129933
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-3clvuit5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-3clvuit5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.479 s) : 15479000, 15479000
.   : milestone, 15479000,
appsec (14.831 s) : 14831000, 14831000
.   : milestone, 14831000,
iast (18.403 s) : 18403000, 18403000
.   : milestone, 18403000,
iast_GLOBAL (17.748 s) : 17748000, 17748000
.   : milestone, 17748000,
profiling (14.772 s) : 14772000, 14772000
.   : milestone, 14772000,
tracing (14.924 s) : 14924000, 14924000
.   : milestone, 14924000,
section candidate
no_agent (14.821 s) : 14821000, 14821000
.   : milestone, 14821000,
appsec (14.675 s) : 14675000, 14675000
.   : milestone, 14675000,
iast (18.447 s) : 18447000, 18447000
.   : milestone, 18447000,
iast_GLOBAL (18.094 s) : 18094000, 18094000
.   : milestone, 18094000,
profiling (15.194 s) : 15194000, 15194000
.   : milestone, 15194000,
tracing (14.983 s) : 14983000, 14983000
.   : milestone, 14983000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.479 s [15.479 s, 15.479 s] -
appsec 14.831 s [14.831 s, 14.831 s] -648.0 ms (-4.2%)
iast 18.403 s [18.403 s, 18.403 s] 2.924 s (18.9%)
iast_GLOBAL 17.748 s [17.748 s, 17.748 s] 2.269 s (14.7%)
profiling 14.772 s [14.772 s, 14.772 s] -707.0 ms (-4.6%)
tracing 14.924 s [14.924 s, 14.924 s] -555.0 ms (-3.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.821 s [14.821 s, 14.821 s] -
appsec 14.675 s [14.675 s, 14.675 s] -146.0 ms (-1.0%)
iast 18.447 s [18.447 s, 18.447 s] 3.626 s (24.5%)
iast_GLOBAL 18.094 s [18.094 s, 18.094 s] 3.273 s (22.1%)
profiling 15.194 s [15.194 s, 15.194 s] 373.0 ms (2.5%)
tracing 14.983 s [14.983 s, 14.983 s] 162.0 ms (1.1%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~4755b8d197, baseline=1.52.0-SNAPSHOT~89e92d59f6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.409 ms) : 2360, 2459
.   : milestone, 2409,
iast (2.199 ms) : 2137, 2261
.   : milestone, 2199,
iast_GLOBAL (2.239 ms) : 2176, 2302
.   : milestone, 2239,
profiling (2.053 ms) : 2003, 2104
.   : milestone, 2053,
tracing (2.005 ms) : 1957, 2053
.   : milestone, 2005,
section candidate
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.415 ms) : 2365, 2464
.   : milestone, 2415,
iast (2.2 ms) : 2138, 2262
.   : milestone, 2200,
iast_GLOBAL (2.231 ms) : 2169, 2294
.   : milestone, 2231,
profiling (2.031 ms) : 1981, 2080
.   : milestone, 2031,
tracing (2.015 ms) : 1967, 2064
.   : milestone, 2015,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.409 ms [2.36 ms, 2.459 ms] 933.679 µs (63.3%)
iast 2.199 ms [2.137 ms, 2.261 ms] 723.085 µs (49.0%)
iast_GLOBAL 2.239 ms [2.176 ms, 2.302 ms] 763.469 µs (51.7%)
profiling 2.053 ms [2.003 ms, 2.104 ms] 577.691 µs (39.1%)
tracing 2.005 ms [1.957 ms, 2.053 ms] 529.362 µs (35.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.415 ms [2.365 ms, 2.464 ms] 939.072 µs (63.6%)
iast 2.2 ms [2.138 ms, 2.262 ms] 724.464 µs (49.1%)
iast_GLOBAL 2.231 ms [2.169 ms, 2.294 ms] 755.733 µs (51.2%)
profiling 2.031 ms [1.981 ms, 2.08 ms] 555.038 µs (37.6%)
tracing 2.015 ms [1.967 ms, 2.064 ms] 539.793 µs (36.6%)

@sarahchen6 sarahchen6 force-pushed the sarahchen6/test-jdk-25-ea branch from fcf5e17 to 3751eea Compare July 9, 2025 19:54
@sarahchen6
Copy link
Contributor Author

JDK 25-EA has been added to CI via #9140

@sarahchen6 sarahchen6 closed this Aug 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tag: do not merge Do not merge changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant