-
Notifications
You must be signed in to change notification settings - Fork 310
Surface potential root cause when agent initialization errors #9170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…ror occurs during the agent's premain initialization phase.
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 11 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.037 s) : 0, 1036668
Total [baseline] (10.573 s) : 0, 10572952
Agent [candidate] (1.038 s) : 0, 1037531
Total [candidate] (10.668 s) : 0, 10667986
section appsec
Agent [baseline] (1.223 s) : 0, 1222597
Total [baseline] (10.771 s) : 0, 10771005
Agent [candidate] (1.222 s) : 0, 1221541
Total [candidate] (10.759 s) : 0, 10759274
section iast
Agent [baseline] (1.173 s) : 0, 1173391
Total [baseline] (10.971 s) : 0, 10971142
Agent [candidate] (1.171 s) : 0, 1171159
Total [candidate] (10.912 s) : 0, 10911594
section profiling
Agent [baseline] (1.184 s) : 0, 1183608
Total [baseline] (10.82 s) : 0, 10819686
Agent [candidate] (1.182 s) : 0, 1181682
Total [candidate] (10.816 s) : 0, 10816285
gantt
title petclinic - break down per module: candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.421 ms) : 0, 1421
crashtracking [candidate] (1.428 ms) : 0, 1428
BytebuddyAgent [baseline] (728.394 ms) : 0, 728394
BytebuddyAgent [candidate] (728.9 ms) : 0, 728900
GlobalTracer [baseline] (240.735 ms) : 0, 240735
GlobalTracer [candidate] (241.075 ms) : 0, 241075
AppSec [baseline] (30.343 ms) : 0, 30343
AppSec [candidate] (30.376 ms) : 0, 30376
Debugger [baseline] (5.991 ms) : 0, 5991
Debugger [candidate] (6.008 ms) : 0, 6008
Remote Config [baseline] (652.732 µs) : 0, 653
Remote Config [candidate] (651.586 µs) : 0, 652
Telemetry [baseline] (8.183 ms) : 0, 8183
Telemetry [candidate] (8.214 ms) : 0, 8214
section appsec
crashtracking [baseline] (1.443 ms) : 0, 1443
crashtracking [candidate] (1.444 ms) : 0, 1444
BytebuddyAgent [baseline] (755.552 ms) : 0, 755552
BytebuddyAgent [candidate] (755.694 ms) : 0, 755694
GlobalTracer [baseline] (235.108 ms) : 0, 235108
GlobalTracer [candidate] (234.269 ms) : 0, 234269
IAST [baseline] (23.702 ms) : 0, 23702
IAST [candidate] (23.56 ms) : 0, 23560
AppSec [baseline] (167.414 ms) : 0, 167414
AppSec [candidate] (167.951 ms) : 0, 167951
Debugger [baseline] (8.593 ms) : 0, 8593
Debugger [candidate] (7.92 ms) : 0, 7920
Remote Config [baseline] (605.516 µs) : 0, 606
Remote Config [candidate] (601.009 µs) : 0, 601
Telemetry [baseline] (8.872 ms) : 0, 8872
Telemetry [candidate] (8.841 ms) : 0, 8841
section iast
crashtracking [baseline] (1.427 ms) : 0, 1427
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (847.682 ms) : 0, 847682
BytebuddyAgent [candidate] (845.281 ms) : 0, 845281
GlobalTracer [baseline] (231.354 ms) : 0, 231354
GlobalTracer [candidate] (231.075 ms) : 0, 231075
IAST [baseline] (32.181 ms) : 0, 32181
IAST [candidate] (29.168 ms) : 0, 29168
AppSec [baseline] (24.812 ms) : 0, 24812
AppSec [candidate] (28.206 ms) : 0, 28206
Debugger [baseline] (6.615 ms) : 0, 6615
Debugger [candidate] (6.636 ms) : 0, 6636
Remote Config [baseline] (603.829 µs) : 0, 604
Remote Config [candidate] (581.928 µs) : 0, 582
Telemetry [baseline] (7.879 ms) : 0, 7879
Telemetry [candidate] (7.906 ms) : 0, 7906
section profiling
crashtracking [baseline] (1.425 ms) : 0, 1425
crashtracking [candidate] (1.393 ms) : 0, 1393
BytebuddyAgent [baseline] (759.316 ms) : 0, 759316
BytebuddyAgent [candidate] (757.898 ms) : 0, 757898
GlobalTracer [baseline] (220.488 ms) : 0, 220488
GlobalTracer [candidate] (220.545 ms) : 0, 220545
AppSec [baseline] (30.282 ms) : 0, 30282
AppSec [candidate] (30.233 ms) : 0, 30233
Debugger [baseline] (6.26 ms) : 0, 6260
Debugger [candidate] (6.253 ms) : 0, 6253
Remote Config [baseline] (665.94 µs) : 0, 666
Remote Config [candidate] (669.938 µs) : 0, 670
Telemetry [baseline] (12.492 ms) : 0, 12492
Telemetry [candidate] (14.572 ms) : 0, 14572
ProfilingAgent [baseline] (104.259 ms) : 0, 104259
ProfilingAgent [candidate] (101.908 ms) : 0, 101908
Profiling [baseline] (104.282 ms) : 0, 104282
Profiling [candidate] (101.932 ms) : 0, 101932
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1043281
Total [baseline] (8.58 s) : 0, 8580089
Agent [candidate] (1.036 s) : 0, 1035683
Total [candidate] (8.617 s) : 0, 8616564
section iast
Agent [baseline] (1.169 s) : 0, 1169487
Total [baseline] (9.21 s) : 0, 9210219
Agent [candidate] (1.175 s) : 0, 1175144
Total [candidate] (9.253 s) : 0, 9252844
gantt
title insecure-bank - break down per module: candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.439 ms) : 0, 1439
crashtracking [candidate] (1.421 ms) : 0, 1421
BytebuddyAgent [baseline] (733.078 ms) : 0, 733078
BytebuddyAgent [candidate] (727.685 ms) : 0, 727685
GlobalTracer [baseline] (241.683 ms) : 0, 241683
GlobalTracer [candidate] (240.691 ms) : 0, 240691
AppSec [baseline] (30.535 ms) : 0, 30535
AppSec [candidate] (30.29 ms) : 0, 30290
Debugger [baseline] (6.003 ms) : 0, 6003
Debugger [candidate] (5.989 ms) : 0, 5989
Remote Config [baseline] (658.541 µs) : 0, 659
Remote Config [candidate] (640.543 µs) : 0, 641
Telemetry [baseline] (8.943 ms) : 0, 8943
Telemetry [candidate] (8.17 ms) : 0, 8170
section iast
crashtracking [baseline] (1.424 ms) : 0, 1424
crashtracking [candidate] (1.438 ms) : 0, 1438
BytebuddyAgent [baseline] (844.465 ms) : 0, 844465
BytebuddyAgent [candidate] (848.832 ms) : 0, 848832
GlobalTracer [baseline] (230.654 ms) : 0, 230654
GlobalTracer [candidate] (231.75 ms) : 0, 231750
IAST [baseline] (29.728 ms) : 0, 29728
IAST [candidate] (29.922 ms) : 0, 29922
AppSec [baseline] (25.624 ms) : 0, 25624
AppSec [candidate] (28.058 ms) : 0, 28058
Debugger [baseline] (8.351 ms) : 0, 8351
Debugger [candidate] (5.741 ms) : 0, 5741
Remote Config [baseline] (586.255 µs) : 0, 586
Remote Config [candidate] (593.42 µs) : 0, 593
Telemetry [baseline] (7.868 ms) : 0, 7868
Telemetry [candidate] (7.931 ms) : 0, 7931
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 2 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section baseline
no_agent (4.448 ms) : 4398, 4498
. : milestone, 4448,
iast (9.064 ms) : 8911, 9217
. : milestone, 9064,
iast_FULL (13.9 ms) : 13625, 14175
. : milestone, 13900,
iast_GLOBAL (10.219 ms) : 10041, 10396
. : milestone, 10219,
profiling (9.123 ms) : 8977, 9270
. : milestone, 9123,
tracing (8.001 ms) : 7878, 8123
. : milestone, 8001,
section candidate
no_agent (4.289 ms) : 4239, 4338
. : milestone, 4289,
iast (9.681 ms) : 9521, 9841
. : milestone, 9681,
iast_FULL (14.35 ms) : 14069, 14631
. : milestone, 14350,
iast_GLOBAL (10.51 ms) : 10321, 10698
. : milestone, 10510,
profiling (8.757 ms) : 8621, 8893
. : milestone, 8757,
tracing (7.773 ms) : 7663, 7883
. : milestone, 7773,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section baseline
no_agent (37.49 ms) : 37187, 37793
. : milestone, 37490,
appsec (46.976 ms) : 46560, 47392
. : milestone, 46976,
code_origins (47.257 ms) : 46828, 47686
. : milestone, 47257,
iast (44.456 ms) : 44071, 44841
. : milestone, 44456,
profiling (47.374 ms) : 46891, 47858
. : milestone, 47374,
tracing (43.559 ms) : 43187, 43932
. : milestone, 43559,
section candidate
no_agent (35.712 ms) : 35418, 36007
. : milestone, 35712,
appsec (48.033 ms) : 47615, 48450
. : milestone, 48033,
code_origins (46.957 ms) : 46547, 47368
. : milestone, 46957,
iast (46.069 ms) : 45667, 46471
. : milestone, 46069,
profiling (48.255 ms) : 47810, 48700
. : milestone, 48255,
tracing (44.684 ms) : 44297, 45071
. : milestone, 44684,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section baseline
no_agent (14.87 s) : 14870000, 14870000
. : milestone, 14870000,
appsec (14.611 s) : 14611000, 14611000
. : milestone, 14611000,
iast (18.475 s) : 18475000, 18475000
. : milestone, 18475000,
iast_GLOBAL (18.169 s) : 18169000, 18169000
. : milestone, 18169000,
profiling (15.304 s) : 15304000, 15304000
. : milestone, 15304000,
tracing (15.063 s) : 15063000, 15063000
. : milestone, 15063000,
section candidate
no_agent (15.259 s) : 15259000, 15259000
. : milestone, 15259000,
appsec (14.695 s) : 14695000, 14695000
. : milestone, 14695000,
iast (18.452 s) : 18452000, 18452000
. : milestone, 18452000,
iast_GLOBAL (18.004 s) : 18004000, 18004000
. : milestone, 18004000,
profiling (15.825 s) : 15825000, 15825000
. : milestone, 15825000,
tracing (14.764 s) : 14764000, 14764000
. : milestone, 14764000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~bbb59b8726, baseline=1.52.0-SNAPSHOT~c73f8b6de5
dateFormat X
axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1484
. : milestone, 1473,
appsec (3.646 ms) : 3428, 3863
. : milestone, 3646,
iast (2.202 ms) : 2139, 2265
. : milestone, 2202,
iast_GLOBAL (2.24 ms) : 2176, 2303
. : milestone, 2240,
profiling (2.065 ms) : 2013, 2117
. : milestone, 2065,
tracing (2.017 ms) : 1968, 2065
. : milestone, 2017,
section candidate
no_agent (1.472 ms) : 1461, 1484
. : milestone, 1472,
appsec (3.642 ms) : 3426, 3857
. : milestone, 3642,
iast (2.193 ms) : 2130, 2256
. : milestone, 2193,
iast_GLOBAL (2.241 ms) : 2178, 2304
. : milestone, 2241,
profiling (2.051 ms) : 1999, 2103
. : milestone, 2051,
tracing (2.014 ms) : 1966, 2063
. : milestone, 2014,
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! To check my understanding -- this now returns the "top cause" (i.e. what was originally returned) along with the most root causes (max 4 levels of this).
@sarahchen6 Yes, by default it will return the top one (as before) and up to 4 from the most root. |
components/environment/src/main/java/datadog/environment/EnvironmentVariables.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some comment about the keeping the environment component with a single responsibility and avoiding to create another config system.
components/environment/src/main/java/datadog/environment/EnvironmentVariables.java
Outdated
Show resolved
Hide resolved
components/environment/src/main/java/datadog/environment/EnvironmentVariables.java
Outdated
Show resolved
Hide resolved
dd-java-agent/src/main/java/datadog/trace/bootstrap/BootstrapInitializationTelemetry.java
Outdated
Show resolved
Hide resolved
dd-java-agent/src/main/java/datadog/trace/bootstrap/BootstrapInitializationTelemetry.java
Outdated
Show resolved
Hide resolved
dd-java-agent/src/main/java/datadog/trace/bootstrap/BootstrapInitializationTelemetry.java
Outdated
Show resolved
Hide resolved
dd-java-agent/src/main/java/datadog/trace/bootstrap/BootstrapInitializationTelemetry.java
Outdated
Show resolved
Hide resolved
I trimmed down the PR title for changelog generation (otherwise it gets truncated with "…"). Feel free to reword it to your liking if I miss the spot 😓 |
# Conflicts: # dd-java-agent/src/main/java/datadog/trace/bootstrap/BootstrapInitializationTelemetry.java # dd-java-agent/src/test/groovy/datadog/trace/bootstrap/BootstrapInitializationTelemetryTest.groovy
What Does This Do
Added logic to unwind and surface the potential root cause when an error occurs during the agent's pre-main initialization phase.
Motivation
Add more context in case of error during pre-main step.
Additional Notes
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]