-
Notifications
You must be signed in to change notification settings - Fork 0
/
report_2024-05-01T09:40:51.497855_nmt_vs_plain.txt
116 lines (107 loc) · 9 KB
/
report_2024-05-01T09:40:51.497855_nmt_vs_plain.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
MODE: work
ITERATIONS: 10
JAVA_HOME: /home/rtoyonag/JDKs/labsjdk-ce-22+22-jvmci-b01-linux-amd64/labsjdk-ce-22-jvmci-b01
GRAALVM_HOME: /home/rtoyonag/IdeaProjects/graal/vm/latest_graalvm_home
HYPERFOIL_HOME: /home/rtoyonag/tools/hyperfoil-0.24.1
Image size with NMT: 47292872
Image size without NMT: 47290704
------------------------------------------------
Average Performance Difference:
These values are averages calculated using the results in the 'Performance Difference' section.
mean (NI): 0.07365151275430139
mean (JAVA): 0.060695229006738814
p50 (NI): 0.06407097121771664
p50 (JAVA): 0.05896116760851734
max (NI): 0.013644642758163926
max (JAVA): 0.007186313129321394
rss (NI): -0.0020684554274524643
rss (JAVA): 0.016950444411030705
startup (NI): 0.02211952152839578
startup (JAVA): 0.010429167834129458
p99 (NI): 0.3293703494907361
p99 (JAVA): 0.05557644644931442
p90 (NI): 0.08519698896692662
p90 (JAVA): 0.06987713729450581
------------------------------------------------
Average Measurments:
These values are averages calculated using the results in the 'Raw Measurements' section.
mean (Without NMT in build): 3766.4115
mean (With NMT): 4040.2791
mean (Java mode without NMT): 4613.2660000000005
mean (Java mode with NMT): 4881.235000000001
p50 (Without NMT in build): 3615.9478
p50 (With NMT): 3846.9622000000004
p50 (Java mode without NMT): 4440.063
p50 (Java mode with NMT): 4695.653399999999
max (Without NMT in build): 22891.7238
max (With NMT): 22950.7062
max (Java mode without NMT): 17563.646999999997
max (Java mode with NMT): 17399.807000000004
rss (Without NMT in build): 53390.799999999996
rss (With NMT): 53030.40000000001
rss (Java mode without NMT): 148952.0
rss (Java mode with NMT): 151460.4
startup (Without NMT in build): 0.14428472518920898
startup (With NMT): 0.14740407466888428
startup (Java mode without NMT): 1.364481234550476
startup (Java mode with NMT): 1.3785443782806397
p99 (Without NMT in build): 11347.557400000002
p99 (With NMT): 13772.389399999998
p99 (Java mode without NMT): 12602.571800000002
p99 (Java mode with NMT): 12497.714199999999
p90 (Without NMT in build): 4544.920600000001
p90 (With NMT): 4925.0294
p90 (Java mode without NMT): 5924.4534
p90 (Java mode with NMT): 6337.330199999999
------------------------------------------------
Performance Difference:
These values are calculated pair-wise for each iteration. They are percentages calcluated using (With NMT - Without NMT) / Without NMT.
mean (NI): [-0.004778451284023546, 0.12158572154025726, 0.1809942013847039, 0.0017600183487067119, 0.0670148871141651, 0.14240754290032678, 0.06720504621697422, 0.0746619784222708, 0.17383753273361272, -0.08817334983398031]
mean (JAVA): [0.1014344385671121, 0.026907704022976982, 0.0036747804859069313, -0.004215612071598952, 0.1194139881577771, 0.04431603843054306, 0.05064615108134602, 0.07046487381243183, 0.030325507071704335, 0.1639844205091888]
p50 (NI): [0.030434790685183528, 0.0765765976300208, 0.12385324568533132, 0.02816902215634106, 0.04424779956049278, 0.09049776254997781, 0.10747666616754721, 0.07692309816748114, 0.12162165505944467, -0.05909092548465388]
p50 (JAVA): [0.057142869599014046, 0.07246378414071866, 0.02142857609963034, 0.028169020138382225, 0.06779662770329556, 0.03623189207035943, 0.057971027312575085, 0.05223881786718068, 0.028169020138382225, 0.16800004101563518]
max (NI): [0.11282051723462681, 0.5801527055436173, 0.04368932200642273, -0.036764707944797546, 0.03864734441959346, 0.010152284657135973, -0.31250001354651075, -0.04232804403670558, -0.04313725748322479, -0.2142857232685183]
max (JAVA): [-0.03649635239740931, 0.21457491204110654, -0.18750000941126016, -0.12738854122227664, 0.32510290107274326, -0.33125001579523167, -0.18326694341207816, 0.08290156095843222, 0.22857144102758356, 0.08661417843160432]
rss (NI): [-0.03487527246306612, 0.03283475783475783, 0.009916492693110648, 0.25427118926577214, -0.07290167865707434, -0.11621561831723325, -0.07269789983844911, -0.1057877599840521, 0.008038958027363376, 0.0767322771643463]
rss (JAVA): [0.02481878348566026, 0.00048216007714561236, 0.01485042735042735, 0.020271005659555943, 0.0620461866026096, -0.010078327585745983, -0.01129506008010681, 0.013931681178834561, 0.028565982244975766, 0.025911605176950753]
startup (NI): [-0.06167332953525477, 0.09245989760664303, 0.0661510975415405, 0.012305222917729034, 0.03591669511778764, -0.04588765525669594, 0.00013069041194574117, 0.037382322474814315, 0.025061211056387308, 0.05934906294906095]
startup (JAVA): [0.009989219906584428, -0.004795710606959067, 0.024814317859274147, 0.004239914329849034, 0.030218031058546237, 0.017930285169193868, 0.004689582458709767, -0.007026509975403115, 0.009771750734936739, 0.01446079740656254]
p99 (NI): [-0.04979253427292017, 1.382353096272311, 1.793893338722443, -0.3395062048176146, 0.6606061216972695, 0.2941176822578798, -0.5632411406892402, 0.1238938136746441, 0.2844827960419221, -0.2931034739793339]
p99 (JAVA): [0.3666666977493878, -0.3020833573407616, -0.23404257218726715, -0.2977941343528599, 1.1984734220401, -0.36842107376563704, -0.006622517225507247, 0.07236842831746443, 0.20909092359306505, -0.08187135233484039]
p90 (NI): [0.027586212702512942, 0.07092200116572832, 0.10000002179827473, 0.03787879663612985, 0.0357142934993839, 0.1953125465661398, 0.0714285869987676, 0.034482765878141225, 0.2148148633750324, 0.06382980104915546]
p90 (JAVA): [0.10734465127626423, 0.07222223446693164, 0.09574469639306527, 0.03867403966982339, 0.07344634034691759, 0.0652174021210716, 0.10674159133424067, 0.06593407698982613, 0.0, 0.07344634034691759]
------------------------------------------------
Raw Measurements:
These are individual measurements for each iteration. rss is in kB, startup time is s, all others are in us.
Without NMT in build:
mean: [4082.285, 3761.19, 3733.65, 3540.304, 3912.847, 3699.797, 3665.32, 3874.087, 3611.349, 3783.286]
p50: [3768.319, 3637.247, 3571.711, 3489.791, 3702.783, 3620.863, 3506.175, 3620.863, 3637.247, 3604.479]
max: [25559.039, 17170.431, 27000.831, 17825.791, 27131.903, 25821.183, 23068.671, 24772.607, 16711.679, 23855.103]
rss: [49548, 56160, 53648, 47996, 58380, 54244, 51996, 60196, 51748, 49992]
startup: [0.14688992500305176, 0.14264130592346191, 0.14212250709533691, 0.1393284797668457, 0.1431572437286377, 0.14690327644348145, 0.14959263801574707, 0.14592504501342773, 0.1434342861175537, 0.1428525447845459]
p99: [15794.175, 8912.895, 8585.215, 10616.831, 10813.439, 8355.839, 16580.607, 14811.135, 7602.175, 11403.263]
p90: [4751.359, 4620.287, 4587.519, 4325.375, 4587.519, 4194.303, 4587.519, 4751.359, 4423.679, 4620.287]
With NMT:
mean: [4062.778, 4218.497, 4409.419, 3546.535, 4175.066, 4226.676, 3911.648, 4163.334, 4239.137, 3449.701]
p50: [3883.007, 3915.775, 4014.079, 3588.095, 3866.623, 3948.543, 3883.007, 3899.391, 4079.615, 3391.487]
max: [28442.623, 27131.903, 28180.479, 17170.431, 28180.479, 26083.327, 15859.711, 23724.031, 15990.783, 18743.295]
rss: [47820, 58004, 54180, 60200, 54124, 47940, 48216, 53828, 52164, 53828]
startup: [0.1378307342529297, 0.15582990646362305, 0.15152406692504883, 0.14104294776916504, 0.148298978805542, 0.14016222953796387, 0.1496121883392334, 0.15138006210327148, 0.14702892303466797, 0.15133070945739746]
p99: [15007.743, 21233.663, 23986.175, 7012.351, 17956.863, 10813.439, 7241.727, 16646.143, 9764.863, 8060.927]
p90: [4882.431, 4947.967, 5046.271, 4489.215, 4751.359, 5013.503, 4915.199, 4915.199, 5373.951, 4915.199]
Java mode without NMT:
mean: [4733.629, 4791.899, 4857.433, 4864.774, 4007.018, 4719.059, 4607.359, 4475.322, 4873.719, 4202.448]
p50: [4587.519, 4521.983, 4587.519, 4653.055, 3866.623, 4521.983, 4521.983, 4390.911, 4653.055, 4095.999]
max: [17956.863, 16187.391, 19922.943, 20578.303, 15925.247, 20971.519, 16449.535, 12648.447, 18350.079, 16646.143]
rss: [152304, 149328, 149760, 147008, 146536, 149628, 149800, 149300, 146888, 148968]
startup: [1.335176706314087, 1.3798885345458984, 1.3427903652191162, 1.3836991786956787, 1.341700792312622, 1.3701603412628174, 1.3658177852630615, 1.3957977294921875, 1.377431869506836, 1.352349042892456]
p99: [11796.479, 12582.911, 12320.767, 17825.791, 8585.215, 17432.575, 9895.935, 9961.471, 14417.919, 11206.655]
p90: [5799.935, 5898.239, 6160.383, 5931.007, 5799.935, 6029.311, 5832.703, 5963.775, 6029.311, 5799.935]
Java mode with NMT:
mean: [5213.782, 4920.838, 4875.283, 4844.266, 4485.512, 4928.189, 4840.704, 4790.675, 5021.517, 4891.584]
p50: [4849.663, 4849.663, 4685.823, 4784.127, 4128.767, 4685.823, 4784.127, 4620.287, 4784.127, 4784.127]
max: [17301.503, 19660.799, 16187.391, 17956.863, 21102.591, 14024.703, 13434.879, 13697.023, 22544.383, 18087.935]
rss: [156084, 149400, 151984, 149988, 155628, 148120, 148108, 151380, 151084, 152828]
startup: [1.3485140800476074, 1.3732709884643555, 1.3761107921600342, 1.3895659446716309, 1.382244348526001, 1.3947277069091797, 1.372222900390625, 1.3859901428222656, 1.3908917903900146, 1.3719050884246826]
p99: [16121.855, 8781.823, 9437.183, 12517.375, 18874.367, 11010.047, 9830.399, 10682.367, 17432.575, 10289.151]
p90: [6422.527, 6324.223, 6750.207, 6160.383, 6225.919, 6422.527, 6455.295, 6356.991, 6029.311, 6225.919]