-
Notifications
You must be signed in to change notification settings - Fork 319
Improve TempLocationManagerTest stability #10118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jbachorik
wants to merge
2
commits into
master
Choose a base branch
from
jb/tlm_test_stability
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081003
Total [baseline] (10.912 s) : 0, 10912272
Agent [candidate] (1.082 s) : 0, 1081973
Total [candidate] (10.837 s) : 0, 10836894
section appsec
Agent [baseline] (1.273 s) : 0, 1273373
Total [baseline] (11.22 s) : 0, 11220268
Agent [candidate] (1.27 s) : 0, 1270219
Total [candidate] (11.122 s) : 0, 11121580
section iast
Agent [baseline] (1.232 s) : 0, 1231612
Total [baseline] (11.102 s) : 0, 11101639
Agent [candidate] (1.232 s) : 0, 1232165
Total [candidate] (11.211 s) : 0, 11210950
section profiling
Agent [baseline] (1.221 s) : 0, 1220821
Total [baseline] (11.227 s) : 0, 11226581
Agent [candidate] (1.213 s) : 0, 1213034
Total [candidate] (11.075 s) : 0, 11074604
gantt
title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (648.539 ms) : 0, 648539
BytebuddyAgent [candidate] (648.412 ms) : 0, 648412
GlobalTracer [baseline] (281.823 ms) : 0, 281823
GlobalTracer [candidate] (282.379 ms) : 0, 282379
AppSec [baseline] (32.416 ms) : 0, 32416
AppSec [candidate] (32.451 ms) : 0, 32451
Debugger [baseline] (68.106 ms) : 0, 68106
Debugger [candidate] (68.45 ms) : 0, 68450
Remote Config [baseline] (673.397 µs) : 0, 673
Remote Config [candidate] (645.844 µs) : 0, 646
Telemetry [baseline] (8.95 ms) : 0, 8950
Telemetry [candidate] (9.105 ms) : 0, 9105
Flare Poller [baseline] (3.706 ms) : 0, 3706
Flare Poller [candidate] (3.776 ms) : 0, 3776
section appsec
crashtracking [baseline] (1.217 ms) : 0, 1217
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (693.452 ms) : 0, 693452
BytebuddyAgent [candidate] (691.977 ms) : 0, 691977
GlobalTracer [baseline] (262.171 ms) : 0, 262171
GlobalTracer [candidate] (260.269 ms) : 0, 260269
IAST [baseline] (25.134 ms) : 0, 25134
IAST [candidate] (24.812 ms) : 0, 24812
AppSec [baseline] (175.434 ms) : 0, 175434
AppSec [candidate] (174.636 ms) : 0, 174636
Debugger [baseline] (66.722 ms) : 0, 66722
Debugger [candidate] (68.108 ms) : 0, 68108
Remote Config [baseline] (712.708 µs) : 0, 713
Remote Config [candidate] (704.261 µs) : 0, 704
Telemetry [baseline] (9.124 ms) : 0, 9124
Telemetry [candidate] (9.093 ms) : 0, 9093
Flare Poller [baseline] (3.886 ms) : 0, 3886
Flare Poller [candidate] (3.915 ms) : 0, 3915
section iast
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (796.828 ms) : 0, 796828
BytebuddyAgent [candidate] (797.686 ms) : 0, 797686
GlobalTracer [baseline] (257.833 ms) : 0, 257833
GlobalTracer [candidate] (257.021 ms) : 0, 257021
IAST [baseline] (26.794 ms) : 0, 26794
IAST [candidate] (27.013 ms) : 0, 27013
AppSec [baseline] (35.498 ms) : 0, 35498
AppSec [candidate] (35.465 ms) : 0, 35465
Debugger [baseline] (65.568 ms) : 0, 65568
Debugger [candidate] (65.53 ms) : 0, 65530
Remote Config [baseline] (551.714 µs) : 0, 552
Remote Config [candidate] (608.558 µs) : 0, 609
Telemetry [baseline] (8.344 ms) : 0, 8344
Telemetry [candidate] (8.501 ms) : 0, 8501
Flare Poller [baseline] (3.446 ms) : 0, 3446
Flare Poller [candidate] (3.496 ms) : 0, 3496
section profiling
crashtracking [baseline] (1.217 ms) : 0, 1217
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (710.71 ms) : 0, 710710
BytebuddyAgent [candidate] (706.79 ms) : 0, 706790
GlobalTracer [baseline] (223.431 ms) : 0, 223431
GlobalTracer [candidate] (222.542 ms) : 0, 222542
AppSec [baseline] (32.758 ms) : 0, 32758
AppSec [candidate] (32.569 ms) : 0, 32569
Debugger [baseline] (68.824 ms) : 0, 68824
Debugger [candidate] (68.365 ms) : 0, 68365
Remote Config [baseline] (665.616 µs) : 0, 666
Remote Config [candidate] (644.483 µs) : 0, 644
Telemetry [baseline] (9.164 ms) : 0, 9164
Telemetry [candidate] (9.071 ms) : 0, 9071
Flare Poller [baseline] (3.835 ms) : 0, 3835
Flare Poller [candidate] (3.86 ms) : 0, 3860
ProfilingAgent [baseline] (99.73 ms) : 0, 99730
ProfilingAgent [candidate] (98.011 ms) : 0, 98011
Profiling [baseline] (100.329 ms) : 0, 100329
Profiling [candidate] (98.592 ms) : 0, 98592
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.08 s) : 0, 1079839
Total [baseline] (8.725 s) : 0, 8725281
Agent [candidate] (1.088 s) : 0, 1088343
Total [candidate] (8.769 s) : 0, 8769333
section iast
Agent [baseline] (1.222 s) : 0, 1221809
Total [baseline] (9.468 s) : 0, 9468191
Agent [candidate] (1.223 s) : 0, 1222634
Total [candidate] (9.46 s) : 0, 9459558
gantt
title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.182 ms) : 0, 1182
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (648.133 ms) : 0, 648133
BytebuddyAgent [candidate] (653.393 ms) : 0, 653393
GlobalTracer [baseline] (281.381 ms) : 0, 281381
GlobalTracer [candidate] (284.01 ms) : 0, 284010
AppSec [baseline] (32.449 ms) : 0, 32449
AppSec [candidate] (32.663 ms) : 0, 32663
Debugger [baseline] (67.648 ms) : 0, 67648
Debugger [candidate] (67.765 ms) : 0, 67765
Remote Config [baseline] (661.05 µs) : 0, 661
Remote Config [candidate] (632.972 µs) : 0, 633
Telemetry [baseline] (8.999 ms) : 0, 8999
Telemetry [candidate] (9.087 ms) : 0, 9087
Flare Poller [baseline] (3.761 ms) : 0, 3761
Flare Poller [candidate] (3.736 ms) : 0, 3736
section iast
crashtracking [baseline] (1.18 ms) : 0, 1180
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (790.927 ms) : 0, 790927
BytebuddyAgent [candidate] (790.724 ms) : 0, 790724
GlobalTracer [baseline] (255.721 ms) : 0, 255721
GlobalTracer [candidate] (255.619 ms) : 0, 255619
IAST [baseline] (26.753 ms) : 0, 26753
IAST [candidate] (26.899 ms) : 0, 26899
AppSec [baseline] (35.432 ms) : 0, 35432
AppSec [candidate] (34.679 ms) : 0, 34679
Debugger [baseline] (63.996 ms) : 0, 63996
Debugger [candidate] (65.733 ms) : 0, 65733
Remote Config [baseline] (543.696 µs) : 0, 544
Remote Config [candidate] (561.485 µs) : 0, 561
Telemetry [baseline] (8.411 ms) : 0, 8411
Telemetry [candidate] (8.424 ms) : 0, 8424
Flare Poller [baseline] (3.433 ms) : 0, 3433
Flare Poller [candidate] (3.423 ms) : 0, 3423
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 17 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section baseline
no_agent (1.209 ms) : 1197, 1221
. : milestone, 1209,
iast (3.152 ms) : 3113, 3192
. : milestone, 3152,
iast_FULL (5.693 ms) : 5636, 5750
. : milestone, 5693,
iast_GLOBAL (3.465 ms) : 3415, 3514
. : milestone, 3465,
profiling (2.249 ms) : 2228, 2269
. : milestone, 2249,
tracing (1.819 ms) : 1804, 1835
. : milestone, 1819,
section candidate
no_agent (1.212 ms) : 1200, 1224
. : milestone, 1212,
iast (3.256 ms) : 3216, 3296
. : milestone, 3256,
iast_FULL (5.659 ms) : 5604, 5714
. : milestone, 5659,
iast_GLOBAL (3.563 ms) : 3512, 3615
. : milestone, 3563,
profiling (2.057 ms) : 2040, 2075
. : milestone, 2057,
tracing (1.785 ms) : 1771, 1799
. : milestone, 1785,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section baseline
no_agent (17.891 ms) : 17711, 18070
. : milestone, 17891,
appsec (18.611 ms) : 18420, 18802
. : milestone, 18611,
code_origins (18.442 ms) : 18258, 18626
. : milestone, 18442,
iast (17.69 ms) : 17517, 17863
. : milestone, 17690,
profiling (18.47 ms) : 18289, 18651
. : milestone, 18470,
tracing (18.036 ms) : 17850, 18223
. : milestone, 18036,
section candidate
no_agent (17.802 ms) : 17623, 17981
. : milestone, 17802,
appsec (18.488 ms) : 18297, 18678
. : milestone, 18488,
code_origins (17.635 ms) : 17460, 17810
. : milestone, 17635,
iast (17.606 ms) : 17433, 17780
. : milestone, 17606,
profiling (20.194 ms) : 19981, 20408
. : milestone, 20194,
tracing (17.98 ms) : 17800, 18159
. : milestone, 17980,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
. : milestone, 1470,
appsec (3.629 ms) : 3413, 3845
. : milestone, 3629,
iast (2.207 ms) : 2142, 2271
. : milestone, 2207,
iast_GLOBAL (2.245 ms) : 2181, 2310
. : milestone, 2245,
profiling (2.063 ms) : 2011, 2116
. : milestone, 2063,
tracing (2.036 ms) : 1985, 2087
. : milestone, 2036,
section candidate
no_agent (1.47 ms) : 1459, 1482
. : milestone, 1470,
appsec (3.688 ms) : 3468, 3907
. : milestone, 3688,
iast (2.196 ms) : 2132, 2260
. : milestone, 2196,
iast_GLOBAL (2.243 ms) : 2178, 2308
. : milestone, 2243,
profiling (2.08 ms) : 2026, 2134
. : milestone, 2080,
tracing (2.036 ms) : 1985, 2087
. : milestone, 2036,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~739ec923e5, baseline=1.57.0-SNAPSHOT~0d1a541243
dateFormat X
axisFormat %s
section baseline
no_agent (14.694 s) : 14694000, 14694000
. : milestone, 14694000,
appsec (14.544 s) : 14544000, 14544000
. : milestone, 14544000,
iast (18.134 s) : 18134000, 18134000
. : milestone, 18134000,
iast_GLOBAL (17.972 s) : 17972000, 17972000
. : milestone, 17972000,
profiling (14.47 s) : 14470000, 14470000
. : milestone, 14470000,
tracing (14.723 s) : 14723000, 14723000
. : milestone, 14723000,
section candidate
no_agent (15.33 s) : 15330000, 15330000
. : milestone, 15330000,
appsec (14.801 s) : 14801000, 14801000
. : milestone, 14801000,
iast (18.442 s) : 18442000, 18442000
. : milestone, 18442000,
iast_GLOBAL (17.946 s) : 17946000, 17946000
. : milestone, 17946000,
profiling (14.54 s) : 14540000, 14540000
. : milestone, 14540000,
tracing (14.605 s) : 14605000, 14605000
. : milestone, 14605000,
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Improves the stability of
TempLocationManagerTestby fixing timing-related test flakiness. The changes ensure proper timeout handling in the cleanup hook interface and improve test synchronization using phaser coordination with timeouts.Motivation
The test was previously marked as
@Flakydue to intermittent failures related to race conditions and timing issues in the cleanup process. This PR addresses those issues by:timeoutparameter to all callback methods to explicitly signal when cleanup operations are timing out, allowing hooks to handle timeouts gracefully@FlakyannotationChanges
TempLocationManager.java:CleanupHookinterface methods to accept atimeoutboolean parameterTempLocationManagerTest.java:@Flakyannotation from timeout testarriveOrAwaitAdvancewith 5-second timeout)LockSupport.parkNanoswith more robustwaitForhelper methodContributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: PROF-13219