Cache the length of arrays in core utilities. #58920

DanielRosenwasser · 2024-06-18T23:16:54Z

Follow-up to #58873 to measure effects of caching the length on all arrays passed into our utilities.

DanielRosenwasser · 2024-06-18T23:17:32Z

typescript-bot · 2024-06-18T23:17:35Z

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
`perf test this`	✅ Started	👀 Results

typescript-bot · 2024-06-19T00:01:16Z

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-Unions - node (v18.15.0, x64)}
_Errors	₃₀	₃₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_62,153	_62,153	_~	_~	_~	_{p=1.000 n=6}
_Types	_50,242	_50,242	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{192,699k (± 0.78%)}	_{193,881k (± 0.98%)}	_~	_192,104k	_195,807k	_{p=0.378 n=6}
_{Parse Time}	_{1.95s (± 0.50%)}	_{1.95s (± 1.09%)}	_~	_1.91s	_1.97s	_{p=0.315 n=6}
_{Bind Time}	_{1.05s (± 0.77%)}	_{1.06s (± 0.84%)}	_~	_1.05s	_1.07s	_{p=0.270 n=6}
_{Check Time}	_{13.82s (± 0.38%)}	_{13.84s (± 0.43%)}	_~	_13.76s	_13.92s	_{p=0.421 n=6}
_{Emit Time}	_{4.08s (± 3.22%)}	_{4.09s (± 3.84%)}	_~	_4.00s	_4.41s	_{p=0.935 n=6}
_{Total Time}	_{20.90s (± 0.59%)}	_{20.95s (± 0.60%)}	_~	_20.84s	_21.17s	_{p=0.520 n=6}
_{angular-1 - node (v18.15.0, x64)}
_Errors	₅	₅	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_944,114	_944,114	_~	_~	_~	_{p=1.000 n=6}
_Types	_407,051	_407,051	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{1,218,312k (± 0.00%)}	_{1,218,268k (± 0.00%)}	_~	_1,218,206k	_1,218,306k	_{p=0.093 n=6}
_{Parse Time}	_{7.93s (± 0.60%)}	_{7.90s (± 0.58%)}	_~	_7.85s	_7.96s	_{p=0.419 n=6}
_{Bind Time}	_{2.21s (± 0.78%)}	_{2.21s (± 0.53%)}	_~	_2.20s	_2.23s	_{p=0.742 n=6}
_{Check Time}	_{35.65s (± 0.29%)}	_{35.69s (± 0.28%)}	_~	_35.50s	_35.79s	_{p=0.688 n=6}
_{Emit Time}	_{16.01s (± 0.89%)}	_{16.07s (± 0.27%)}	_~	_15.99s	_16.11s	_{p=0.229 n=6}
_{Total Time}	_{61.80s (± 0.34%)}	_{61.86s (± 0.27%)}	_~	_61.55s	_62.05s	_{p=0.467 n=6}
_{mui-docs - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_2,133,542	_2,133,542	_~	_~	_~	_{p=1.000 n=6}
_Types	_926,558	_926,558	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{2,115,732k (± 0.01%)}	_{2,115,769k (± 0.01%)}	_~	_2,115,605k	_2,116,001k	_{p=0.471 n=6}
_{Parse Time}	_{9.69s (± 0.36%)}	_{9.71s (± 0.17%)}	_~	_9.69s	_9.73s	_{p=0.258 n=6}
_{Bind Time}	_{3.38s (± 0.78%)}	_{3.37s (± 0.59%)}	_~	_3.35s	_3.40s	_{p=0.464 n=6}
_{Check Time}	_{102.76s (± 0.47%)}	_{102.37s (± 0.41%)}	_~	_102.00s	_103.09s	_{p=0.261 n=6}
_{Emit Time}	_{0.20s (± 4.15%)}	_{0.19s (± 5.34%)}	_~	_0.18s	_0.21s	_{p=0.546 n=6}
_{Total Time}	_{116.04s (± 0.39%)}	_{115.64s (± 0.37%)}	_~	_115.27s	_116.37s	_{p=0.199 n=6}
_{self-build-src - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,230,726	_1,230,755	_{+29 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_261,163	_261,163	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{2,346,907k (± 0.02%)}	_{2,345,968k (± 0.05%)}	_~	_2,343,692k	_2,347,347k	_{p=0.128 n=6}
_{Parse Time}	_{5.95s (± 0.43%)}	_{5.97s (± 1.52%)}	_~	_5.85s	_6.11s	_{p=0.471 n=6}
_{Bind Time}	_{2.26s (± 0.43%)}	_{2.27s (± 0.76%)}	_~	_2.25s	_2.30s	_{p=0.413 n=6}
_{Check Time}	_{39.75s (± 0.22%)}	_{39.74s (± 0.18%)}	_~	_39.69s	_39.88s	_{p=1.000 n=6}
_{Emit Time}	_{3.11s (± 1.14%)}	_{3.15s (± 3.81%)}	_~	_3.04s	_3.35s	_{p=1.000 n=6}
_{Total Time}	_{51.09s (± 0.21%)}	_{51.14s (± 0.27%)}	_~	_50.93s	_51.34s	_{p=0.575 n=6}
_{self-build-src-public-api - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,230,726	_1,230,755	_{+29 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_261,163	_261,163	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{2,422,882k (± 0.04%)}	_{2,423,552k (± 0.02%)}	_~	_2,422,835k	_2,424,172k	_{p=0.230 n=6}
_{Parse Time}	_{6.23s (± 1.22%)}	_{6.21s (± 0.80%)}	_~	_6.14s	_6.26s	_{p=0.630 n=6}
_{Bind Time}	_{2.02s (± 0.93%)}	_{2.02s (± 1.49%)}	_~	_1.97s	_2.06s	_{p=0.261 n=6}
_{Check Time}	_{39.92s (± 0.24%)}	_{40.06s (± 0.52%)}	_~	_39.87s	_40.46s	_{p=0.126 n=6}
_{Emit Time}	_{3.20s (± 2.70%)}	_{3.14s (± 1.61%)}	_{-0.06s (- 1.82%)}	_3.04s	_3.17s	_{p=0.045 n=6}
_{Total Time}	_{51.37s (± 0.29%)}	_{51.44s (± 0.53%)}	_~	_51.20s	_51.94s	_{p=1.000 n=6}
_{self-compiler - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_258,573	_258,602	_{+29 (+ 0.01%)}	_~	_~	_{p=0.001 n=6}
_Types	_104,819	_104,819	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{428,169k (± 0.01%)}	_{428,210k (± 0.01%)}	_~	_428,151k	_428,251k	_{p=0.298 n=6}
_{Parse Time}	_{4.10s (± 0.58%)}	_{4.09s (± 0.91%)}	_~	_4.03s	_4.13s	_{p=0.872 n=6}
_{Bind Time}	_{1.62s (± 1.43%)}	_{1.62s (± 0.96%)}	_~	_1.60s	_1.64s	_{p=1.000 n=6}
_{Check Time}	_{21.94s (± 0.46%)}	_{21.95s (± 0.25%)}	_~	_21.89s	_22.04s	_{p=0.872 n=6}
_{Emit Time}	_{1.69s (± 1.48%)}	_{1.73s (± 1.79%)}	_{+0.04s (+ 2.57%)}	_1.69s	_1.78s	_{p=0.035 n=6}
_{Total Time}	_{29.35s (± 0.19%)}	_{29.39s (± 0.34%)}	_~	_29.26s	_29.52s	_{p=0.574 n=6}
_{ts-pre-modules - node (v18.15.0, x64)}
_Errors	₃₅	₃₅	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_224,565	_224,565	_~	_~	_~	_{p=1.000 n=6}
_Types	_93,734	_93,734	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{369,486k (± 0.03%)}	_{369,459k (± 0.01%)}	_~	_369,434k	_369,491k	_{p=0.748 n=6}
_{Parse Time}	_{2.78s (± 0.62%)}	_{2.76s (± 1.24%)}	_~	_2.71s	_2.81s	_{p=0.332 n=6}
_{Bind Time}	_{1.59s (± 1.01%)}	_{1.58s (± 0.65%)}	_~	_1.57s	_1.60s	_{p=0.365 n=6}
_{Check Time}	_{15.42s (± 0.16%)}	_{15.49s (± 0.39%)}	_{+0.07s (+ 0.45%)}	_15.44s	_15.58s	_{p=0.024 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{19.79s (± 0.10%)}	_{19.83s (± 0.34%)}	_~	_19.74s	_19.94s	_{p=0.226 n=6}
_{vscode - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_2,865,581	_2,865,581	_~	_~	_~	_{p=1.000 n=6}
_Types	_970,775	_970,775	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{3,028,142k (± 0.00%)}	_{3,028,123k (± 0.00%)}	_~	_3,028,071k	_3,028,256k	_{p=0.377 n=6}
_{Parse Time}	_{13.49s (± 0.33%)}	_{13.53s (± 0.30%)}	_~	_13.49s	_13.60s	_{p=0.261 n=6}
_{Bind Time}	_{4.14s (± 0.36%)}	_{4.13s (± 0.43%)}	_~	_4.11s	_4.16s	_{p=0.256 n=6}
_{Check Time}	_{72.70s (± 0.27%)}	_{72.80s (± 0.62%)}	_~	_72.47s	_73.62s	_{p=0.810 n=6}
_{Emit Time}	_{23.92s (± 0.62%)}	_{23.85s (± 0.43%)}	_~	_23.72s	_23.95s	_{p=0.335 n=6}
_{Total Time}	_{114.25s (± 0.21%)}	_{114.31s (± 0.46%)}	_~	_113.86s	_115.19s	_{p=0.689 n=6}
_{webpack - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_266,914	_266,914	_~	_~	_~	_{p=1.000 n=6}
_Types	_108,685	_108,685	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{411,324k (± 0.01%)}	_{411,313k (± 0.02%)}	_~	_411,262k	_411,466k	_{p=0.378 n=6}
_{Parse Time}	_{3.79s (± 0.46%)}	_{3.80s (± 0.43%)}	_~	_3.78s	_3.82s	_{p=0.934 n=6}
_{Bind Time}	_{1.68s (± 0.32%)}	_{1.68s (± 0.24%)}	_~	_1.67s	_1.68s	_{p=0.054 n=6}
_{Check Time}	_{16.83s (± 0.34%)}	_{16.78s (± 0.19%)}	_~	_16.73s	_16.82s	_{p=0.145 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{22.31s (± 0.26%)}	_{22.25s (± 0.08%)}	_~	_22.24s	_22.29s	_{p=0.077 n=6}
_{xstate-main - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_523,765	_523,765	_~	_~	_~	_{p=1.000 n=6}
_Types	_178,055	_178,055	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{461,687k (± 0.06%)}	_{461,874k (± 0.07%)}	_~	_461,380k	_462,239k	_{p=0.471 n=6}
_{Parse Time}	_{3.17s (± 0.99%)}	_{3.18s (± 0.55%)}	_~	_3.16s	_3.21s	_{p=0.934 n=6}
_{Bind Time}	_{1.19s (± 0.69%)}	_{1.18s (± 0.35%)}	_~	_1.18s	_1.19s	_{p=0.248 n=6}
_{Check Time}	_{18.00s (± 0.55%)}	_{17.98s (± 0.43%)}	_~	_17.91s	_18.11s	_{p=0.468 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{22.36s (± 0.49%)}	_{22.34s (± 0.41%)}	_~	_22.25s	_22.50s	_{p=0.470 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Compiler-Unions - node (v18.15.0, x64)
angular-1 - node (v18.15.0, x64)
mui-docs - node (v18.15.0, x64)
self-build-src - node (v18.15.0, x64)
self-build-src-public-api - node (v18.15.0, x64)
self-compiler - node (v18.15.0, x64)
ts-pre-modules - node (v18.15.0, x64)
vscode - node (v18.15.0, x64)
webpack - node (v18.15.0, x64)
xstate-main - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,743ms (± 0.88%)}	_{2,743ms (± 0.37%)}	_~	_2,725ms	_2,755ms	_{p=0.936 n=6}
_{Req 2 - geterr}	_{6,961ms (± 9.67%)}	_{6,982ms (±10.36%)}	_~	_6,026ms	_7,510ms	_{p=0.810 n=6}
_{Req 3 - references}	_{372ms (±10.72%)}	_{374ms (± 9.20%)}	_~	_321ms	_397ms	_{p=0.936 n=6}
_{Req 4 - navto}	_{331ms (± 7.67%)}	_{328ms (± 9.10%)}	_~	_267ms	_341ms	_{p=0.454 n=6}
_{Req 5 - completionInfo count}	_1,357	_1,357	_~	_~	_~	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{129ms (± 8.31%)}	_{127ms (± 4.22%)}	_~	_116ms	_131ms	_{p=0.332 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{3,160ms (±11.08%)}	_{3,193ms (±10.01%)}	_~	_2,855ms	_3,599ms	_{p=0.936 n=6}
_{Req 2 - geterr}	_{5,220ms (±10.40%)}	_{4,856ms (±11.00%)}	_~	_4,483ms	_5,546ms	_{p=0.229 n=6}
_{Req 3 - references}	_{389ms (± 9.47%)}	_{377ms (±10.73%)}	_~	_331ms	_417ms	_{p=0.687 n=6}
_{Req 4 - navto}	_{327ms (± 7.85%)}	_{311ms (±11.90%)}	_~	_277ms	_358ms	_{p=0.567 n=6}
_{Req 5 - completionInfo count}	_1,519	_1,519	_~	_~	_~	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{99ms (± 7.74%)}	_{99ms (± 8.21%)}	_~	_82ms	_102ms	_{p=1.000 n=6}
_{xstate-main-1-tsserver - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{7,669ms (± 0.27%)}	_{7,661ms (± 0.24%)}	_~	_7,645ms	_7,698ms	_{p=0.470 n=6}
_{Req 2 - geterr}	_{1,705ms (± 0.82%)}	_{1,679ms (± 0.97%)}	_~	_1,660ms	_1,703ms	_{p=0.065 n=6}
_{Req 3 - references}	_{112ms (± 0.98%)}	_{113ms (± 0.74%)}	_~	_112ms	_114ms	_{p=0.282 n=6}
_{Req 4 - navto}	_{584ms (± 1.49%)}	_{587ms (± 1.41%)}	_~	_573ms	_596ms	_{p=0.521 n=6}
_{Req 5 - completionInfo count}	_3,413	_3,413	_~	_~	_~	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{1,240ms (± 1.78%)}	_{1,248ms (± 1.12%)}	_~	_1,223ms	_1,260ms	_{p=0.332 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstate-main-1-tsserver - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{187.04ms (± 0.16%)}	_{187.04ms (± 0.19%)}	_~	_185.42ms	_192.14ms	_{p=0.356 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{296.31ms (± 0.25%)}	_{296.15ms (± 0.25%)}	_{-0.16ms (- 0.05%)}	_289.41ms	_301.79ms	_{p=0.017 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{288.41ms (± 0.29%)}	_{288.45ms (± 0.30%)}	_~	_280.83ms	_295.55ms	_{p=0.434 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{288.09ms (± 0.28%)}	_{288.11ms (± 0.31%)}	_~	_281.12ms	_297.44ms	_{p=0.962 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Cache the length of arrays in core utilities.

f75c9a3

typescript-bot assigned DanielRosenwasser Jun 18, 2024

typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Cache the length of arrays in core utilities. #58920

Cache the length of arrays in core utilities. #58920

Uh oh!

DanielRosenwasser commented Jun 18, 2024

Uh oh!

DanielRosenwasser commented Jun 18, 2024

Uh oh!

typescript-bot commented Jun 18, 2024 •

edited

Loading

Uh oh!

typescript-bot commented Jun 19, 2024

tsc

tsserver

startup

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Cache the length of arrays in core utilities. #58920

Are you sure you want to change the base?

Cache the length of arrays in core utilities. #58920

Uh oh!

Conversation

DanielRosenwasser commented Jun 18, 2024

Uh oh!

DanielRosenwasser commented Jun 18, 2024

Uh oh!

typescript-bot commented Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

typescript-bot commented Jun 19, 2024

tsc

tsserver

startup

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

typescript-bot commented Jun 18, 2024 •

edited

Loading