Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use concat instead of spread in concatenate #58946

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Use concat instead of spread in concatenate #58946

wants to merge 8 commits into from

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Jun 20, 2024 •
edited
Loading

Per #58873 (comment) (Thanks @dmichon-msft)

Benchmarked these, and yeah concat is a lot faster.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 20, 2024
@jakebailey jakebailey mentioned this pull request Jun 20, 2024
Merged
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 20, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started ❌ Results

@typescript-bot
Copy link
Collaborator

@jakebailey, the perf run you requested failed. You can check the log here.

@jakebailey
Copy link
Member Author

nodejs.org outage of all things; need to look into retrying those requests.

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 20, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started ❌ Results

@typescript-bot
Copy link
Collaborator

@jakebailey, the perf run you requested failed. You can check the log here.

@jakebailey
Copy link
Member Author

Jeeze, everything's breaking today.

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 194,068k (± 1.00%) 193,040k (± 0.93%) ~ 191,737k 195,374k p=0.128 n=6
Parse Time 1.58s (± 0.57%) 1.59s (± 0.74%) ~ 1.58s 1.61s p=0.270 n=6
Bind Time 0.85s (± 0.98%) 0.86s (± 0.95%) ~ 0.85s 0.87s p=0.718 n=6
Check Time 11.19s (± 0.68%) 11.15s (± 0.55%) ~ 11.11s 11.27s p=0.295 n=6
Emit Time 3.29s (± 0.27%) 3.28s (± 0.71%) ~ 3.25s 3.31s p=0.367 n=6
Total Time 16.92s (± 0.51%) 16.87s (± 0.28%) ~ 16.83s 16.96s p=0.229 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,281k (± 0.00%) 1,208,252k (± 0.00%) -10,030k (- 0.82%) 1,208,215k 1,208,276k p=0.005 n=6
Parse Time 7.93s (± 0.22%) 7.90s (± 0.34%) ~ 7.85s 7.93s p=0.051 n=6
Bind Time 2.20s (± 0.76%) 2.21s (± 0.37%) ~ 2.20s 2.22s p=0.231 n=6
Check Time 35.57s (± 0.42%) 35.49s (± 0.30%) ~ 35.34s 35.59s p=0.378 n=6
Emit Time 15.96s (± 0.74%) 16.08s (± 0.29%) ~ 16.00s 16.13s p=0.126 n=6
Total Time 61.66s (± 0.39%) 61.67s (± 0.24%) ~ 61.47s 61.82s p=0.936 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,134,309 2,134,309 ~ ~ ~ p=1.000 n=6
Types 927,039 927,039 ~ ~ ~ p=1.000 n=6
Memory used 2,115,844k (± 0.01%) 2,095,061k (± 0.00%) -20,783k (- 0.98%) 2,094,995k 2,095,155k p=0.005 n=6
Parse Time 9.66s (± 0.31%) 9.66s (± 0.32%) ~ 9.63s 9.71s p=0.871 n=6
Bind Time 3.38s (± 0.48%) 3.39s (± 0.62%) ~ 3.35s 3.41s p=0.413 n=6
Check Time 102.52s (± 0.29%) 102.02s (± 0.52%) ~ 101.42s 102.89s p=0.066 n=6
Emit Time 0.19s (± 5.12%) 0.19s ~ ~ ~ p=1.000 n=6
Total Time 115.75s (± 0.28%) 115.26s (± 0.49%) ~ 114.64s 116.18s p=0.093 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,692 1,230,727 +35 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,165 261,172 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,346,307k (± 0.03%) 2,342,496k (± 0.02%) -3,811k (- 0.16%) 2,341,657k 2,343,243k p=0.005 n=6
Parse Time 5.98s (± 0.89%) 5.98s (± 0.46%) ~ 5.95s 6.01s p=0.810 n=6
Bind Time 2.25s (± 0.86%) 2.25s (± 0.89%) ~ 2.22s 2.28s p=1.000 n=6
Check Time 39.68s (± 0.22%) 39.64s (± 0.36%) ~ 39.44s 39.76s p=0.810 n=6
Emit Time 3.13s (± 3.20%) 3.17s (± 1.86%) ~ 3.09s 3.23s p=0.470 n=6
Total Time 51.05s (± 0.27%) 51.04s (± 0.31%) ~ 50.78s 51.23s p=0.936 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,692 1,230,727 +35 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,165 261,172 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,422,448k (± 0.03%) 2,418,021k (± 0.05%) -4,427k (- 0.18%) 2,415,820k 2,419,052k p=0.005 n=6
Parse Time 6.12s (± 0.53%) 6.16s (± 1.36%) ~ 6.09s 6.31s p=0.378 n=6
Bind Time 2.02s (± 0.91%) 2.02s (± 1.03%) ~ 1.98s 2.04s p=0.685 n=6
Check Time 40.02s (± 0.27%) 40.04s (± 0.34%) ~ 39.80s 40.21s p=0.575 n=6
Emit Time 3.14s (± 1.65%) 3.17s (± 2.70%) ~ 3.06s 3.27s p=0.521 n=6
Total Time 51.33s (± 0.31%) 51.41s (± 0.37%) ~ 51.24s 51.74s p=0.689 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,573 258,608 +35 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,819 104,826 +7 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 428,193k (± 0.01%) 426,454k (± 0.01%) -1,740k (- 0.41%) 426,410k 426,514k p=0.005 n=6
Parse Time 3.34s (± 1.01%) 3.32s (± 0.78%) ~ 3.30s 3.37s p=0.224 n=6
Bind Time 1.33s (± 1.03%) 1.32s (± 1.14%) ~ 1.30s 1.34s p=0.243 n=6
Check Time 17.76s (± 0.54%) 17.75s (± 0.23%) ~ 17.69s 17.80s p=0.936 n=6
Emit Time 1.37s (± 0.97%) 1.38s (± 2.36%) ~ 1.34s 1.42s p=0.803 n=6
Total Time 23.81s (± 0.45%) 23.77s (± 0.31%) ~ 23.71s 23.90s p=0.520 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,484k (± 0.04%) 367,687k (± 0.02%) -1,797k (- 0.49%) 367,594k 367,813k p=0.005 n=6
Parse Time 2.77s (± 0.97%) 2.77s (± 1.40%) ~ 2.71s 2.82s p=0.685 n=6
Bind Time 1.58s (± 0.33%) 1.59s (± 0.47%) ~ 1.58s 1.60s p=0.247 n=6
Check Time 15.44s (± 0.17%) 15.50s (± 0.22%) +0.06s (+ 0.39%) 15.44s 15.53s p=0.012 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.79s (± 0.13%) 19.86s (± 0.24%) +0.07s (+ 0.36%) 19.81s 19.93s p=0.012 n=6
vscode - node (v18.15.0, x64)
Errors 139 139 ~ ~ ~ p=1.000 n=6
Symbols 2,847,724 2,847,724 ~ ~ ~ p=1.000 n=6
Types 974,145 974,145 ~ ~ ~ p=1.000 n=6
Memory used 3,040,223k (± 0.00%) 3,008,149k (± 0.00%) -32,074k (- 1.05%) 3,008,100k 3,008,233k p=0.005 n=6
Parse Time 13.54s (± 0.45%) 13.55s (± 0.38%) ~ 13.48s 13.63s p=1.000 n=6
Bind Time 4.20s (± 0.29%) 4.20s (± 0.47%) ~ 4.16s 4.21s p=0.560 n=6
Check Time 74.13s (± 0.50%) 74.15s (± 0.32%) ~ 73.88s 74.46s p=1.000 n=6
Emit Time 23.94s (± 0.32%) 24.05s (± 0.85%) ~ 23.79s 24.34s p=0.378 n=6
Total Time 115.80s (± 0.32%) 115.93s (± 0.31%) ~ 115.45s 116.43s p=0.575 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,116 267,116 ~ ~ ~ p=1.000 n=6
Types 108,758 108,758 ~ ~ ~ p=1.000 n=6
Memory used 411,568k (± 0.01%) 409,779k (± 0.03%) -1,789k (- 0.43%) 409,711k 409,988k p=0.005 n=6
Parse Time 3.82s (± 0.51%) 3.82s (± 0.80%) ~ 3.78s 3.86s p=0.935 n=6
Bind Time 1.69s (± 0.61%) 1.69s (± 0.24%) ~ 1.69s 1.70s p=0.924 n=6
Check Time 16.74s (± 0.32%) 16.68s (± 0.56%) ~ 16.57s 16.84s p=0.199 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.25s (± 0.23%) 22.19s (± 0.45%) ~ 22.08s 22.35s p=0.227 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,713k (± 0.07%) 455,964k (± 0.06%) -5,749k (- 1.25%) 455,626k 456,341k p=0.005 n=6
Parse Time 3.91s (± 0.50%) 3.91s (± 0.55%) ~ 3.89s 3.95s p=1.000 n=6
Bind Time 1.47s (± 0.80%) 1.46s (± 0.93%) ~ 1.44s 1.48s p=0.563 n=6
Check Time 22.29s (± 0.41%) 22.76s (± 0.44%) +0.48s (+ 2.14%) 22.64s 22.90s p=0.005 n=6
Emit Time 0.00s 0.00s (±244.70%) ~ 0.00s 0.01s p=0.405 n=6
Total Time 27.67s (± 0.34%) 28.14s (± 0.30%) +0.48s (+ 1.74%) 28.05s 28.26s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,293ms (± 0.60%) 2,297ms (± 0.48%) ~ 2,276ms 2,306ms p=0.810 n=6
Req 2 - geterr 5,074ms (± 0.47%) 5,037ms (± 0.30%) -37ms (- 0.72%) 5,010ms 5,048ms p=0.030 n=6
Req 3 - references 264ms (± 0.44%) 264ms (± 0.24%) ~ 263ms 265ms p=1.000 n=6
Req 4 - navto 226ms (± 0.91%) 228ms (± 0.61%) ~ 226ms 230ms p=0.565 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 86ms (± 7.94%) 87ms (± 5.63%) ~ 78ms 93ms p=1.000 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,432ms (± 0.83%) 2,429ms (± 0.88%) ~ 2,399ms 2,464ms p=1.000 n=6
Req 2 - geterr 3,792ms (± 0.21%) 3,795ms (± 0.39%) ~ 3,770ms 3,812ms p=0.573 n=6
Req 3 - references 277ms (± 0.19%) 277ms (± 0.35%) ~ 276ms 279ms p=0.386 n=6
Req 4 - navto 226ms (± 0.18%) 225ms (± 0.23%) -1ms (- 0.37%) 225ms 226ms p=0.022 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 74ms (± 0.55%) 74ms (± 0.55%) ~ 73ms 74ms p=1.000 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,176ms (± 0.28%) 5,204ms (± 0.83%) ~ 5,155ms 5,284ms p=0.066 n=6
Req 2 - geterr 1,138ms (± 1.09%) 1,126ms (± 0.78%) ~ 1,116ms 1,136ms p=0.092 n=6
Req 3 - references 79ms (± 5.08%) 79ms (± 4.19%) ~ 76ms 83ms p=1.000 n=6
Req 4 - navto 461ms (± 1.19%) 454ms (± 0.67%) -6ms (- 1.38%) 451ms 459ms p=0.036 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 845ms (± 1.73%) 833ms (± 2.16%) ~ 812ms 864ms p=0.173 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.84ms (± 0.19%) 157.75ms (± 0.19%) -0.09ms (- 0.05%) 156.70ms 162.01ms p=0.001 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.47ms (± 0.14%) 232.45ms (± 0.12%) ~ 231.22ms 236.39ms p=0.969 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 227.69ms (± 0.13%) 227.62ms (± 0.16%) -0.07ms (- 0.03%) 226.21ms 235.03ms p=0.005 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.48ms (± 0.15%) 226.41ms (± 0.14%) ~ 225.03ms 231.48ms p=0.057 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Less memory across the board, it seems. The rest is surely noise...

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 20, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,738k (± 0.75%) 192,388k (± 0.77%) ~ 191,739k 195,398k p=0.066 n=6
Parse Time 1.95s (± 0.88%) 1.95s (± 1.64%) ~ 1.90s 1.99s p=0.570 n=6
Bind Time 1.07s 1.06s (± 0.84%) -0.01s (- 0.93%) 1.05s 1.07s p=0.028 n=6
Check Time 13.85s (± 0.37%) 13.80s (± 0.47%) ~ 13.73s 13.91s p=0.227 n=6
Emit Time 4.09s (± 3.25%) 4.02s (± 0.67%) ~ 3.98s 4.05s p=0.374 n=6
Total Time 20.97s (± 0.78%) 20.84s (± 0.32%) ~ 20.74s 20.92s p=0.077 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,318k (± 0.00%) 1,208,213k (± 0.00%) -10,106k (- 0.83%) 1,208,171k 1,208,255k p=0.005 n=6
Parse Time 7.93s (± 0.51%) 7.91s (± 0.54%) ~ 7.89s 8.00s p=0.325 n=6
Bind Time 2.21s (± 0.25%) 2.21s (± 0.25%) ~ 2.20s 2.21s p=1.000 n=6
Check Time 35.63s (± 0.48%) 35.72s (± 0.56%) ~ 35.43s 35.96s p=0.296 n=6
Emit Time 16.06s (± 0.92%) 16.08s (± 0.33%) ~ 16.00s 16.14s p=0.872 n=6
Total Time 61.82s (± 0.40%) 61.92s (± 0.33%) ~ 61.71s 62.16s p=0.470 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,134,309 2,134,309 ~ ~ ~ p=1.000 n=6
Types 927,039 927,039 ~ ~ ~ p=1.000 n=6
Memory used 2,115,990k (± 0.01%) 2,095,027k (± 0.00%) -20,963k (- 0.99%) 2,094,969k 2,095,101k p=0.005 n=6
Parse Time 7.85s (± 0.29%) 7.84s (± 0.43%) ~ 7.79s 7.89s p=0.739 n=6
Bind Time 2.74s (± 0.72%) 2.75s (± 0.84%) ~ 2.72s 2.78s p=0.466 n=6
Check Time 83.63s (± 0.19%) 83.61s (± 0.45%) ~ 83.27s 84.15s p=0.575 n=6
Emit Time 0.16s (± 3.29%) 0.15s (± 3.36%) ~ 0.15s 0.16s p=0.311 n=6
Total Time 94.37s (± 0.18%) 94.36s (± 0.42%) ~ 93.98s 94.93s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,831 +35 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,196 261,203 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,347,139k (± 0.01%) 2,352,558k (± 1.01%) ~ 2,342,585k 2,401,199k p=0.066 n=6
Parse Time 5.93s (± 0.98%) 5.95s (± 0.93%) ~ 5.87s 6.01s p=0.688 n=6
Bind Time 2.26s (± 0.52%) 2.25s (± 0.61%) ~ 2.24s 2.27s p=0.806 n=6
Check Time 39.76s (± 0.13%) 39.67s (± 0.31%) ~ 39.53s 39.86s p=0.149 n=6
Emit Time 3.12s (± 1.30%) 3.13s (± 2.11%) ~ 3.06s 3.25s p=1.000 n=6
Total Time 51.08s (± 0.10%) 51.02s (± 0.23%) ~ 50.88s 51.16s p=0.471 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,831 +35 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,196 261,203 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,422,653k (± 0.02%) 2,418,852k (± 0.03%) -3,801k (- 0.16%) 2,417,540k 2,419,697k p=0.005 n=6
Parse Time 6.15s (± 0.84%) 6.17s (± 0.64%) ~ 6.13s 6.24s p=0.335 n=6
Bind Time 2.03s (± 0.74%) 2.03s (± 1.02%) ~ 2.00s 2.06s p=1.000 n=6
Check Time 40.04s (± 0.15%) 40.01s (± 0.26%) ~ 39.87s 40.11s p=0.873 n=6
Emit Time 3.13s (± 0.81%) 3.17s (± 2.26%) ~ 3.10s 3.30s p=0.328 n=6
Total Time 51.36s (± 0.17%) 51.39s (± 0.32%) ~ 51.18s 51.65s p=0.810 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,574 258,609 +35 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,819 104,826 +7 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 428,161k (± 0.01%) 426,459k (± 0.01%) -1,702k (- 0.40%) 426,411k 426,521k p=0.005 n=6
Parse Time 3.33s (± 0.98%) 3.32s (± 0.69%) ~ 3.29s 3.36s p=0.397 n=6
Bind Time 1.32s (± 1.51%) 1.33s (± 1.22%) ~ 1.31s 1.35s p=0.254 n=6
Check Time 17.78s (± 0.40%) 17.78s (± 0.46%) ~ 17.69s 17.90s p=1.000 n=6
Emit Time 1.37s (± 1.49%) 1.38s (± 1.70%) ~ 1.36s 1.42s p=0.934 n=6
Total Time 23.81s (± 0.39%) 23.81s (± 0.23%) ~ 23.74s 23.87s p=0.936 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,565k (± 0.02%) 367,733k (± 0.03%) -1,831k (- 0.50%) 367,574k 367,837k p=0.005 n=6
Parse Time 3.45s (± 0.48%) 3.44s (± 0.84%) ~ 3.40s 3.47s p=1.000 n=6
Bind Time 1.94s (± 0.39%) 1.93s (± 0.73%) ~ 1.91s 1.95s p=0.280 n=6
Check Time 19.10s (± 0.52%) 19.23s (± 0.28%) ~ 19.16s 19.31s p=0.054 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 24.48s (± 0.42%) 24.60s (± 0.34%) ~ 24.48s 24.71s p=0.078 n=6
vscode - node (v18.15.0, x64)
Errors 139 139 ~ ~ ~ p=1.000 n=6
Symbols 2,847,748 2,847,748 ~ ~ ~ p=1.000 n=6
Types 974,172 974,172 ~ ~ ~ p=1.000 n=6
Memory used 3,040,296k (± 0.00%) 3,008,166k (± 0.00%) -32,131k (- 1.06%) 3,008,044k 3,008,271k p=0.005 n=6
Parse Time 13.59s (± 0.66%) 13.55s (± 0.31%) ~ 13.50s 13.61s p=0.518 n=6
Bind Time 4.20s (± 0.25%) 4.21s (± 0.32%) ~ 4.19s 4.23s p=0.134 n=6
Check Time 74.64s (± 2.17%) 73.71s (± 0.27%) -0.93s (- 1.24%) 73.45s 73.94s p=0.020 n=6
Emit Time 23.29s (± 6.89%) 24.12s (± 0.48%) +0.83s (+ 3.57%) 23.97s 24.25s p=0.013 n=6
Total Time 115.71s (± 0.06%) 115.59s (± 0.21%) ~ 115.16s 115.77s p=0.629 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,116 267,116 ~ ~ ~ p=1.000 n=6
Types 108,758 108,758 ~ ~ ~ p=1.000 n=6
Memory used 411,545k (± 0.02%) 409,644k (± 0.01%) -1,902k (- 0.46%) 409,590k 409,730k p=0.005 n=6
Parse Time 3.18s (± 0.43%) 3.18s (± 0.59%) ~ 3.14s 3.19s p=1.000 n=6
Bind Time 1.42s (± 0.53%) 1.42s (± 0.39%) ~ 1.41s 1.42s p=0.476 n=6
Check Time 14.19s (± 0.25%) 14.15s (± 0.47%) ~ 14.09s 14.23s p=0.416 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.79s (± 0.20%) 18.74s (± 0.41%) ~ 18.65s 18.84s p=0.469 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,895k (± 0.09%) 455,833k (± 0.08%) -6,062k (- 1.31%) 455,574k 456,510k p=0.005 n=6
Parse Time 3.92s (± 0.96%) 3.92s (± 0.47%) ~ 3.91s 3.95s p=0.806 n=6
Bind Time 1.47s (± 1.05%) 1.47s (± 0.61%) ~ 1.46s 1.48s p=1.000 n=6
Check Time 22.18s (± 0.34%) 22.72s (± 0.51%) +0.55s (+ 2.47%) 22.55s 22.86s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.56s (± 0.14%) 28.11s (± 0.38%) +0.55s (+ 2.00%) 27.98s 28.24s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@dmichon-msft
Copy link
Contributor

That's a remarkably large RAM reduction for such a simple change.

@jakebailey
Copy link
Member Author

Yeah, I'm just not sure how to rationalize the 2% slowdown in xstate (which was in both benchmark results).

@jakebailey
Copy link
Member Author

Hm, that one's the only statistically significant one, but the other check benchmarks also seem slightly worse.

@DanielRosenwasser
Copy link
Member

For my own curiosity, can you try a possibly-silly experiment where you effectively reimplement concat?

let result = [];
for (let i = 0; i < array1.length; i++) result.push(array1[i]);
for (let i = 0; i < array2.length; i++) result.push(array2[i]);
return result;

@Andarist
Copy link
Contributor

could preallocating the array's size be helpful in the suggestion above?

@DanielRosenwasser
Copy link
Member

To be honest, if you call new Array(length), I don't know if the true array length is ever really respected.

@jakebailey
Copy link
Member Author

could preallocating the array's size be helpful in the suggestion above?

Nope, preallocation actually doesn't help at all. When you do something like new Array({ length: ... }) you actually get the least optimal V8 array kind. There's a patch series to V8 that would change this but my experimentation showed that it didn't actually help much.

I can try the manual push.

@dmichon-msft
Copy link
Contributor

From some microbenchmarking on node 18, the scaling varies wildly with array size (on smaller arrays concat shows a stronger lead than for large arrays, but I wasn't able to find any regression). Also worth pointing out that there is significant savings when concatenating more than 2 arrays to do them all at once instead of a pair at a time.

i.e.

a1.concat(a2,a3);
// or, faster still
[].concat(a1, a2, a3);

is much faster than

a1.concat(a2).concat(a3);

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 20, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@jakebailey
Copy link
Member Author

Also worth pointing out that there is significant savings when concatenating more than 2 arrays to do them all at once instead of a pair at a time.

I tried to find an example where we concat'd more than once and didn't seem to find any, unfortunately.

@dmichon-msft
Copy link
Contributor

dmichon-msft commented Jun 20, 2024 •
edited
Loading

I tried to find an example where we concat'd more than once and didn't seem to find any, unfortunately.

I find three matches for concatenate(concatenate( in the TypeScript codebase when searching from VSCode online:

TypeScript/src/compiler/binder.ts

Line 1667 in beb375a

finallyLabel.antecedent = concatenate(concatenate(normalExitLabel.antecedent, exceptionLabel.antecedent), returnLabel.antecedent);

https://github.com/microsoft/TypeScript/blob/beb375a9cac75ba29abb707e6736e2d645ddb2f0/src/compiler/checker.ts#L5228

TypeScript/src/compiler/transformers/declarations.ts

Line 1646 in beb375a

const memberNodes = concatenate(concatenate(privateIdentifier, parameterProperties), visitNodes(input.members, visitDeclarationSubtree, isClassElement));

Not sure if any of those callers are the hot paths for concatenate, though. That's a question for the profiler.

@jakebailey
Copy link
Member Author

Hm, yeah, those all may be hot.

@dmichon-msft
Copy link
Contributor

My largest project (90s build time) only spends 37ms in concatenate according to the profiler, so I really want to know how the perf test hit a >1% regression from this.

@jakebailey
Copy link
Member Author

The test case is just https://github.com/statelyai/xstate, if you're interested.

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 194,068k (± 0.93%) 192,866k (± 0.75%) ~ 192,233k 195,839k p=0.298 n=6
Parse Time 1.30s (± 0.97%) 1.30s (± 1.23%) ~ 1.27s 1.31s p=1.000 n=6
Bind Time 0.71s 0.71s ~ ~ ~ p=1.000 n=6
Check Time 9.44s (± 0.67%) 9.45s (± 0.35%) ~ 9.40s 9.49s p=1.000 n=6
Emit Time 2.74s (± 0.86%) 2.70s (± 1.04%) -0.04s (- 1.58%) 2.65s 2.73s p=0.019 n=6
Total Time 14.19s (± 0.52%) 14.16s (± 0.32%) ~ 14.10s 14.22s p=0.520 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,315k (± 0.01%) 1,217,030k (± 0.00%) -1,285k (- 0.11%) 1,216,996k 1,217,063k p=0.005 n=6
Parse Time 6.66s (± 0.68%) 6.65s (± 0.56%) ~ 6.59s 6.70s p=0.421 n=6
Bind Time 1.85s (± 0.56%) 1.86s (± 0.65%) ~ 1.85s 1.88s p=0.282 n=6
Check Time 30.67s (± 0.40%) 30.55s (± 0.61%) ~ 30.27s 30.76s p=0.369 n=6
Emit Time 13.60s (± 0.28%) 13.54s (± 0.19%) -0.06s (- 0.44%) 13.51s 13.58s p=0.024 n=6
Total Time 52.79s (± 0.19%) 52.60s (± 0.44%) ~ 52.26s 52.88s p=0.128 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,134,309 2,134,309 ~ ~ ~ p=1.000 n=6
Types 927,039 927,039 ~ ~ ~ p=1.000 n=6
Memory used 2,115,875k (± 0.00%) 2,112,890k (± 0.00%) -2,985k (- 0.14%) 2,112,824k 2,112,976k p=0.005 n=6
Parse Time 6.59s (± 0.25%) 6.60s (± 0.22%) ~ 6.58s 6.62s p=0.368 n=6
Bind Time 2.33s (± 0.50%) 2.33s (± 0.44%) ~ 2.31s 2.34s p=0.615 n=6
Check Time 71.00s (± 0.74%) 70.54s (± 0.32%) ~ 70.31s 70.82s p=0.066 n=6
Emit Time 0.14s (± 4.05%) 0.13s (± 3.87%) ~ 0.13s 0.14s p=0.640 n=6
Total Time 80.05s (± 0.65%) 79.59s (± 0.29%) ~ 79.36s 79.89s p=0.066 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,834 +38 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,196 261,198 +2 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,346,697k (± 0.03%) 2,345,955k (± 0.03%) ~ 2,345,053k 2,346,571k p=0.128 n=6
Parse Time 4.99s (± 1.02%) 4.96s (± 1.08%) ~ 4.89s 5.04s p=0.335 n=6
Bind Time 1.90s (± 0.67%) 1.90s (± 0.91%) ~ 1.87s 1.92s p=1.000 n=6
Check Time 33.72s (± 0.23%) 33.72s (± 0.18%) ~ 33.64s 33.80s p=0.873 n=6
Emit Time 2.64s (± 2.09%) 2.62s (± 1.76%) ~ 2.56s 2.67s p=0.520 n=6
Total Time 43.28s (± 0.27%) 43.23s (± 0.20%) ~ 43.14s 43.32s p=0.378 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,834 +38 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,196 261,198 +2 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,422,338k (± 0.02%) 2,422,110k (± 0.03%) ~ 2,421,274k 2,422,967k p=0.689 n=6
Parse Time 6.15s (± 0.38%) 6.17s (± 0.22%) ~ 6.15s 6.19s p=0.199 n=6
Bind Time 2.02s (± 1.20%) 2.02s (± 0.73%) ~ 2.00s 2.04s p=0.871 n=6
Check Time 40.13s (± 0.22%) 40.09s (± 0.27%) ~ 39.96s 40.25s p=0.689 n=6
Emit Time 3.18s (± 1.70%) 3.18s (± 2.45%) ~ 3.08s 3.27s p=1.000 n=6
Total Time 51.50s (± 0.22%) 51.49s (± 0.30%) ~ 51.30s 51.72s p=1.000 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,574 258,612 +38 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,819 104,821 +2 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 428,186k (± 0.01%) 428,005k (± 0.01%) -181k (- 0.04%) 427,962k 428,046k p=0.005 n=6
Parse Time 3.33s (± 1.13%) 3.31s (± 0.25%) ~ 3.30s 3.32s p=0.374 n=6
Bind Time 1.33s (± 1.29%) 1.31s (± 1.50%) ~ 1.28s 1.34s p=0.087 n=6
Check Time 17.85s (± 0.15%) 17.76s (± 0.21%) -0.09s (- 0.51%) 17.72s 17.82s p=0.008 n=6
Emit Time 1.38s (± 1.97%) 1.37s (± 1.50%) ~ 1.35s 1.41s p=0.627 n=6
Total Time 23.89s (± 0.31%) 23.75s (± 0.10%) -0.14s (- 0.59%) 23.71s 23.78s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,475k (± 0.04%) 369,207k (± 0.02%) -268k (- 0.07%) 369,122k 369,306k p=0.005 n=6
Parse Time 2.77s (± 1.34%) 2.77s (± 1.15%) ~ 2.74s 2.83s p=0.871 n=6
Bind Time 1.58s (± 1.04%) 1.58s (± 1.01%) ~ 1.57s 1.61s p=0.405 n=6
Check Time 15.42s (± 0.24%) 15.48s (± 0.25%) +0.06s (+ 0.41%) 15.42s 15.54s p=0.019 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.76s (± 0.22%) 19.83s (± 0.32%) ~ 19.73s 19.91s p=0.077 n=6
vscode - node (v18.15.0, x64)
Errors 139 139 ~ ~ ~ p=1.000 n=6
Symbols 2,853,268 2,853,268 ~ ~ ~ p=1.000 n=6
Types 975,680 975,680 ~ ~ ~ p=1.000 n=6
Memory used 3,044,537k (± 0.00%) 3,039,053k (± 0.00%) -5,484k (- 0.18%) 3,038,981k 3,039,130k p=0.005 n=6
Parse Time 13.56s (± 0.22%) 13.55s (± 0.29%) ~ 13.52s 13.61s p=0.681 n=6
Bind Time 4.23s (± 0.38%) 4.22s (± 0.30%) ~ 4.21s 4.24s p=0.315 n=6
Check Time 74.10s (± 0.24%) 74.48s (± 2.36%) ~ 73.27s 78.04s p=0.173 n=6
Emit Time 24.02s (± 0.23%) 23.55s (± 6.93%) ~ 20.22s 24.44s p=0.065 n=6
Total Time 115.91s (± 0.15%) 115.80s (± 0.32%) ~ 115.09s 116.12s p=0.936 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,116 267,116 ~ ~ ~ p=1.000 n=6
Types 108,758 108,758 ~ ~ ~ p=1.000 n=6
Memory used 411,650k (± 0.03%) 411,323k (± 0.01%) -327k (- 0.08%) 411,250k 411,384k p=0.005 n=6
Parse Time 4.71s (± 0.35%) 4.70s (± 0.61%) ~ 4.68s 4.75s p=0.466 n=6
Bind Time 2.08s (± 0.84%) 2.08s (± 0.71%) ~ 2.06s 2.10s p=1.000 n=6
Check Time 20.81s (± 0.41%) 20.67s (± 0.40%) -0.14s (- 0.68%) 20.55s 20.78s p=0.045 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.60s (± 0.30%) 27.46s (± 0.33%) -0.15s (- 0.53%) 27.30s 27.56s p=0.020 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,685k (± 0.06%) 459,442k (± 0.07%) -2,243k (- 0.49%) 459,066k 459,925k p=0.005 n=6
Parse Time 3.16s (± 0.58%) 3.15s (± 0.90%) ~ 3.12s 3.20s p=0.418 n=6
Bind Time 1.18s (± 0.83%) 1.18s (± 0.35%) ~ 1.18s 1.19s p=0.753 n=6
Check Time 17.90s (± 0.33%) 18.35s (± 0.23%) +0.45s (+ 2.51%) 18.29s 18.41s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.24s (± 0.27%) 22.68s (± 0.27%) +0.44s (+ 1.99%) 22.58s 22.75s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Once again, 2% in xstate... Going to try the append version instead.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 20, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@jakebailey
Copy link
Member Author

FWIW this change is totally wrong because append discards undefined... Oof

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,362k (± 0.96%) 194,769k (± 0.96%) +1,407k (+ 0.73%) 192,307k 196,048k p=0.030 n=6
Parse Time 1.58s (± 0.74%) 1.58s (± 0.48%) ~ 1.57s 1.59s p=0.734 n=6
Bind Time 0.86s (± 1.04%) 0.86s (± 1.20%) ~ 0.85s 0.87s p=0.541 n=6
Check Time 11.22s (± 0.54%) 11.23s (± 0.38%) ~ 11.19s 11.31s p=0.871 n=6
Emit Time 3.29s (± 0.65%) 3.24s (± 0.46%) -0.05s (- 1.37%) 3.23s 3.27s p=0.006 n=6
Total Time 16.95s (± 0.42%) 16.92s (± 0.28%) ~ 16.86s 17.00s p=0.228 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,315k (± 0.00%) 1,218,191k (± 0.00%) -124k (- 0.01%) 1,218,151k 1,218,263k p=0.008 n=6
Parse Time 7.94s (± 0.59%) 7.89s (± 0.63%) ~ 7.86s 7.99s p=0.077 n=6
Bind Time 2.21s (± 0.55%) 2.22s (± 0.23%) ~ 2.21s 2.22s p=0.142 n=6
Check Time 35.60s (± 0.35%) 35.56s (± 0.50%) ~ 35.35s 35.85s p=0.575 n=6
Emit Time 16.11s (± 0.48%) 15.91s (± 1.06%) ~ 15.74s 16.08s p=0.109 n=6
Total Time 61.85s (± 0.20%) 61.58s (± 0.43%) ~ 61.25s 62.02s p=0.078 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,134,309 2,134,309 ~ ~ ~ p=1.000 n=6
Types 927,039 927,039 ~ ~ ~ p=1.000 n=6
Memory used 2,115,913k (± 0.01%) 2,115,394k (± 0.00%) -519k (- 0.02%) 2,115,342k 2,115,530k p=0.005 n=6
Parse Time 7.86s (± 0.27%) 7.85s (± 0.53%) ~ 7.80s 7.91s p=0.936 n=6
Bind Time 2.75s (± 0.27%) 2.75s (± 0.67%) ~ 2.73s 2.77s p=0.681 n=6
Check Time 83.70s (± 0.44%) 83.63s (± 0.62%) ~ 82.84s 84.35s p=0.936 n=6
Emit Time 0.16s (± 3.95%) 0.16s (± 4.75%) ~ 0.15s 0.17s p=0.718 n=6
Total Time 94.46s (± 0.40%) 94.39s (± 0.53%) ~ 93.70s 95.14s p=0.936 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,835 +39 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,202 261,206 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,346,445k (± 0.03%) 2,346,355k (± 0.05%) ~ 2,344,351k 2,347,786k p=1.000 n=6
Parse Time 5.98s (± 0.48%) 5.98s (± 0.64%) ~ 5.93s 6.02s p=0.936 n=6
Bind Time 2.27s (± 1.42%) 2.25s (± 0.65%) ~ 2.23s 2.27s p=0.260 n=6
Check Time 39.74s (± 0.32%) 39.71s (± 0.38%) ~ 39.54s 39.95s p=0.575 n=6
Emit Time 3.17s (± 4.37%) 3.15s (± 2.89%) ~ 3.05s 3.27s p=0.810 n=6
Total Time 51.18s (± 0.44%) 51.08s (± 0.35%) ~ 50.90s 51.37s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,835 +39 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,202 261,206 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,422,830k (± 0.04%) 2,422,968k (± 0.03%) ~ 2,421,671k 2,423,600k p=0.298 n=6
Parse Time 7.72s (± 0.28%) 7.68s (± 0.49%) -0.05s (- 0.60%) 7.65s 7.75s p=0.045 n=6
Bind Time 2.50s (± 0.88%) 2.49s (± 0.49%) ~ 2.47s 2.50s p=0.332 n=6
Check Time 49.44s (± 0.32%) 49.47s (± 0.15%) ~ 49.39s 49.57s p=0.470 n=6
Emit Time 3.92s (± 2.13%) 3.83s (± 1.76%) ~ 3.74s 3.92s p=0.078 n=6
Total Time 63.59s (± 0.40%) 63.47s (± 0.13%) ~ 63.32s 63.54s p=0.689 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,574 258,613 +39 (+ 0.02%) ~ ~ p=0.001 n=6
Types 104,825 104,829 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 428,177k (± 0.00%) 428,201k (± 0.01%) ~ 428,165k 428,252k p=0.128 n=6
Parse Time 3.31s (± 0.60%) 3.30s (± 0.51%) ~ 3.28s 3.33s p=0.288 n=6
Bind Time 1.32s (± 1.55%) 1.32s (± 1.31%) ~ 1.29s 1.34s p=0.935 n=6
Check Time 17.74s (± 0.30%) 17.83s (± 0.75%) ~ 17.66s 18.06s p=0.173 n=6
Emit Time 1.38s (± 1.81%) 1.38s (± 1.77%) ~ 1.36s 1.42s p=0.681 n=6
Total Time 23.75s (± 0.17%) 23.83s (± 0.63%) ~ 23.62s 24.09s p=0.109 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,512k (± 0.03%) 369,451k (± 0.03%) ~ 369,361k 369,662k p=0.378 n=6
Parse Time 2.80s (± 0.39%) 2.76s (± 0.54%) -0.04s (- 1.31%) 2.75s 2.79s p=0.007 n=6
Bind Time 1.59s (± 1.17%) 1.59s (± 1.22%) ~ 1.57s 1.62s p=0.681 n=6
Check Time 15.50s (± 0.43%) 15.52s (± 0.31%) ~ 15.47s 15.61s p=0.514 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.89s (± 0.29%) 19.87s (± 0.28%) ~ 19.83s 19.98s p=0.747 n=6
vscode - node (v18.15.0, x64)
Errors 139 139 ~ ~ ~ p=1.000 n=6
Symbols 2,853,268 2,853,268 ~ ~ ~ p=1.000 n=6
Types 975,680 975,680 ~ ~ ~ p=1.000 n=6
Memory used 3,044,497k (± 0.00%) 3,042,804k (± 0.00%) -1,693k (- 0.06%) 3,042,725k 3,042,877k p=0.005 n=6
Parse Time 16.76s (± 0.54%) 16.71s (± 0.23%) ~ 16.66s 16.77s p=0.518 n=6
Bind Time 5.15s (± 0.75%) 5.14s (± 0.33%) ~ 5.12s 5.16s p=0.746 n=6
Check Time 90.04s (± 0.18%) 89.62s (± 0.45%) -0.42s (- 0.47%) 88.98s 90.01s p=0.045 n=6
Emit Time 29.24s (± 0.44%) 29.24s (± 0.40%) ~ 29.05s 29.41s p=1.000 n=6
Total Time 141.18s (± 0.18%) 140.71s (± 0.33%) ~ 140.03s 141.16s p=0.066 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,116 267,116 ~ ~ ~ p=1.000 n=6
Types 108,758 108,758 ~ ~ ~ p=1.000 n=6
Memory used 411,572k (± 0.01%) 411,494k (± 0.01%) -77k (- 0.02%) 411,429k 411,533k p=0.013 n=6
Parse Time 3.82s (± 0.39%) 3.81s (± 0.29%) ~ 3.80s 3.83s p=0.323 n=6
Bind Time 1.69s (± 0.89%) 1.69s (± 0.53%) ~ 1.68s 1.70s p=0.458 n=6
Check Time 16.76s (± 0.38%) 16.65s (± 0.34%) -0.11s (- 0.68%) 16.59s 16.75s p=0.024 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.28s (± 0.30%) 22.15s (± 0.28%) -0.12s (- 0.55%) 22.09s 22.26s p=0.020 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,783k (± 0.07%) 460,192k (± 0.06%) -1,592k (- 0.34%) 459,681k 460,391k p=0.005 n=6
Parse Time 3.17s (± 0.98%) 3.16s (± 0.83%) ~ 3.14s 3.21s p=0.373 n=6
Bind Time 1.19s (± 0.75%) 1.18s (± 0.44%) ~ 1.18s 1.19s p=0.190 n=6
Check Time 17.97s (± 0.44%) 18.38s (± 0.47%) +0.41s (+ 2.25%) 18.24s 18.45s p=0.005 n=6
Emit Time 0.00s 0.00s (±244.70%) ~ 0.00s 0.02s p=0.405 n=6
Total Time 22.33s (± 0.41%) 22.72s (± 0.43%) +0.39s (+ 1.77%) 22.58s 22.84s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@dmichon-msft
Copy link
Contributor

Just for the sake of noise, can you test what happens if you make this change entirely cosmetic (e.g. just add a comment to the previous implementation) and rerun the perf test?

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 21, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,344k (± 0.95%) 193,451k (± 0.90%) ~ 192,146k 195,709k p=0.810 n=6
Parse Time 1.30s (± 0.80%) 1.31s (± 0.39%) ~ 1.31s 1.32s p=0.142 n=6
Bind Time 0.71s 0.71s ~ ~ ~ p=1.000 n=6
Check Time 9.42s (± 0.48%) 9.48s (± 0.47%) ~ 9.40s 9.52s p=0.063 n=6
Emit Time 2.74s (± 0.79%) 2.74s (± 1.05%) ~ 2.71s 2.79s p=0.936 n=6
Total Time 14.18s (± 0.33%) 14.24s (± 0.47%) ~ 14.15s 14.34s p=0.106 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,315k (± 0.00%) 1,218,304k (± 0.00%) ~ 1,218,228k 1,218,351k p=0.810 n=6
Parse Time 7.95s (± 0.70%) 7.92s (± 0.63%) ~ 7.84s 7.99s p=0.687 n=6
Bind Time 2.22s (± 0.40%) 2.23s (± 0.61%) ~ 2.21s 2.25s p=0.452 n=6
Check Time 35.80s (± 0.24%) 35.71s (± 0.24%) ~ 35.54s 35.78s p=0.171 n=6
Emit Time 16.14s (± 0.35%) 16.14s (± 0.41%) ~ 16.01s 16.19s p=0.748 n=6
Total Time 62.11s (± 0.17%) 62.00s (± 0.14%) ~ 61.89s 62.10s p=0.172 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,134,309 2,134,309 ~ ~ ~ p=1.000 n=6
Types 927,039 927,039 ~ ~ ~ p=1.000 n=6
Memory used 2,115,858k (± 0.00%) 2,115,884k (± 0.00%) ~ 2,115,818k 2,115,962k p=0.297 n=6
Parse Time 6.59s (± 0.29%) 6.59s (± 0.30%) ~ 6.56s 6.62s p=1.000 n=6
Bind Time 2.32s (± 0.59%) 2.33s (± 0.36%) ~ 2.31s 2.33s p=0.931 n=6
Check Time 70.81s (± 0.43%) 70.83s (± 0.36%) ~ 70.53s 71.28s p=0.873 n=6
Emit Time 0.13s (± 3.87%) 0.14s 🔻+0.01s (+ 5.00%) ~ ~ p=0.025 n=6
Total Time 79.86s (± 0.40%) 79.88s (± 0.33%) ~ 79.58s 80.34s p=0.810 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,796 ~ ~ ~ p=1.000 n=6
Types 261,202 261,202 ~ ~ ~ p=1.000 n=6
Memory used 2,346,049k (± 0.04%) 2,345,456k (± 0.02%) ~ 2,344,878k 2,346,265k p=0.298 n=6
Parse Time 5.01s (± 1.34%) 4.99s (± 0.59%) ~ 4.94s 5.02s p=0.298 n=6
Bind Time 1.91s (± 1.01%) 1.90s (± 0.79%) ~ 1.88s 1.92s p=0.140 n=6
Check Time 33.69s (± 0.19%) 33.82s (± 0.61%) ~ 33.64s 34.17s p=0.298 n=6
Emit Time 2.67s (± 2.36%) 2.74s (± 3.79%) ~ 2.63s 2.93s p=0.199 n=6
Total Time 43.32s (± 0.22%) 43.48s (± 0.73%) ~ 43.24s 44.06s p=0.630 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,796 1,230,796 ~ ~ ~ p=1.000 n=6
Types 261,202 261,202 ~ ~ ~ p=1.000 n=6
Memory used 2,422,175k (± 0.07%) 2,421,204k (± 0.04%) ~ 2,420,000k 2,422,480k p=0.230 n=6
Parse Time 5.15s (± 0.81%) 5.14s (± 0.88%) ~ 5.07s 5.20s p=0.810 n=6
Bind Time 1.69s (± 0.79%) 1.70s (± 0.96%) ~ 1.68s 1.72s p=0.140 n=6
Check Time 34.18s (± 0.45%) 34.06s (± 0.48%) ~ 33.80s 34.31s p=0.229 n=6
Emit Time 2.67s (± 2.29%) 2.72s (± 3.31%) ~ 2.59s 2.86s p=0.298 n=6
Total Time 43.72s (± 0.40%) 43.66s (± 0.46%) ~ 43.28s 43.87s p=0.689 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,574 258,574 ~ ~ ~ p=1.000 n=6
Types 104,825 104,825 ~ ~ ~ p=1.000 n=6
Memory used 428,209k (± 0.02%) 428,185k (± 0.01%) ~ 428,149k 428,212k p=0.689 n=6
Parse Time 3.31s (± 0.53%) 3.31s (± 0.97%) ~ 3.27s 3.35s p=1.000 n=6
Bind Time 1.31s (± 1.84%) 1.32s (± 1.82%) ~ 1.28s 1.34s p=0.805 n=6
Check Time 17.73s (± 0.14%) 17.74s (± 0.27%) ~ 17.68s 17.81s p=0.686 n=6
Emit Time 1.38s (± 1.51%) 1.37s (± 1.81%) ~ 1.33s 1.40s p=0.871 n=6
Total Time 23.72s (± 0.22%) 23.74s (± 0.19%) ~ 23.69s 23.81s p=0.572 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,523k (± 0.04%) 369,406k (± 0.02%) ~ 369,339k 369,499k p=0.128 n=6
Parse Time 2.78s (± 1.24%) 2.78s (± 1.01%) ~ 2.75s 2.81s p=0.807 n=6
Bind Time 1.58s (± 0.98%) 1.58s (± 0.26%) ~ 1.57s 1.58s p=0.545 n=6
Check Time 15.45s (± 0.21%) 15.45s (± 0.26%) ~ 15.39s 15.50s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.81s (± 0.23%) 19.81s (± 0.30%) ~ 19.72s 19.89s p=1.000 n=6
vscode - node (v18.15.0, x64)
Errors 139 139 ~ ~ ~ p=1.000 n=6
Symbols 2,853,280 2,853,280 ~ ~ ~ p=1.000 n=6
Types 975,685 975,685 ~ ~ ~ p=1.000 n=6
Memory used 3,044,538k (± 0.00%) 3,044,542k (± 0.00%) ~ 3,044,408k 3,044,637k p=1.000 n=6
Parse Time 13.56s (± 0.30%) 13.57s (± 0.33%) ~ 13.50s 13.64s p=0.574 n=6
Bind Time 4.22s (± 0.36%) 4.22s (± 0.62%) ~ 4.20s 4.27s p=1.000 n=6
Check Time 74.70s (± 2.11%) 73.93s (± 0.52%) ~ 73.39s 74.46s p=0.378 n=6
Emit Time 23.41s (± 6.59%) 24.09s (± 0.57%) ~ 23.86s 24.22s p=0.261 n=6
Total Time 115.89s (± 0.22%) 115.81s (± 0.44%) ~ 115.04s 116.50s p=0.810 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,116 267,116 ~ ~ ~ p=1.000 n=6
Types 108,758 108,758 ~ ~ ~ p=1.000 n=6
Memory used 411,584k (± 0.02%) 411,662k (± 0.02%) ~ 411,546k 411,743k p=0.093 n=6
Parse Time 4.70s (± 0.55%) 4.71s (± 0.42%) ~ 4.68s 4.73s p=0.517 n=6
Bind Time 2.09s (± 0.86%) 2.10s (± 0.72%) ~ 2.08s 2.12s p=0.869 n=6
Check Time 20.79s (± 0.67%) 20.82s (± 0.38%) ~ 20.67s 20.91s p=0.575 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.58s (± 0.46%) 27.62s (± 0.33%) ~ 27.46s 27.73s p=0.575 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,745k (± 0.09%) 461,706k (± 0.07%) ~ 461,327k 462,128k p=1.000 n=6
Parse Time 3.17s (± 0.35%) 3.18s (± 0.65%) ~ 3.14s 3.20s p=0.287 n=6
Bind Time 1.17s (± 0.94%) 1.17s (± 0.47%) ~ 1.17s 1.18s p=0.476 n=6
Check Time 17.97s (± 0.30%) 18.00s (± 0.19%) ~ 17.96s 18.04s p=0.332 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.32s (± 0.26%) 22.36s (± 0.16%) ~ 22.31s 22.40s p=0.334 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Yeah, the 2 percent was real.

@ehoogeveen-medweb
Copy link

Spitballing, but could the change be causing an extra garbage collection during the test? i.e. something that would eventually happen anyway (and maybe take more time) but happens to trigger earlier with this pattern.

@jakebailey
Copy link
Member Author

That doesn't feel right, just because these all have reduced memory usage, which implies that GC is actually doing less work overall... This may take some profiling to figure out.

@jakebailey jakebailey marked this pull request as draft July 23, 2024 22:14
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 16, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,324k (± 0.02%) 192,605k (± 0.77%) ~ 191,853k 195,626k p=0.378 n=6
Parse Time 1.96s (± 0.79%) 1.95s (± 1.16%) ~ 1.91s 1.97s p=0.371 n=6
Bind Time 1.06s (± 1.19%) 1.06s (± 1.03%) ~ 1.04s 1.07s p=0.932 n=6
Check Time 13.97s (± 0.58%) 14.02s (± 0.54%) ~ 13.96s 14.17s p=0.229 n=6
Emit Time 4.01s (± 1.12%) 4.02s (± 0.68%) ~ 3.98s 4.05s p=0.872 n=6
Total Time 21.01s (± 0.48%) 21.04s (± 0.33%) ~ 20.98s 21.17s p=0.572 n=6
angular-1 - node (v18.15.0, x64)
Errors 7 7 ~ ~ ~ p=1.000 n=6
Symbols 945,757 945,757 ~ ~ ~ p=1.000 n=6
Types 410,067 410,067 ~ ~ ~ p=1.000 n=6
Memory used 1,222,595k (± 0.00%) 1,212,427k (± 0.00%) -10,167k (- 0.83%) 1,212,370k 1,212,470k p=0.005 n=6
Parse Time 7.92s (± 0.49%) 7.89s (± 0.10%) ~ 7.87s 7.89s p=0.123 n=6
Bind Time 2.23s (± 0.37%) 2.23s (± 0.38%) ~ 2.22s 2.24s p=0.718 n=6
Check Time 36.37s (± 0.41%) 36.15s (± 0.31%) -0.22s (- 0.61%) 36.00s 36.29s p=0.031 n=6
Emit Time 17.74s (± 0.88%) 17.76s (± 0.22%) ~ 17.70s 17.80s p=0.629 n=6
Total Time 64.25s (± 0.41%) 64.02s (± 0.13%) ~ 63.92s 64.12s p=0.173 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,505,734 2,505,734 ~ ~ ~ p=1.000 n=6
Types 993,039 993,039 ~ ~ ~ p=1.000 n=6
Memory used 2,376,665k (± 0.00%) 2,353,840k (± 0.01%) -22,825k (- 0.96%) 2,353,732k 2,354,005k p=0.005 n=6
Parse Time 13.55s (± 0.28%) 13.52s (± 0.21%) ~ 13.49s 13.57s p=0.147 n=6
Bind Time 3.21s (± 1.28%) 3.20s (± 0.85%) ~ 3.16s 3.24s p=0.871 n=6
Check Time 106.77s (± 0.60%) 106.35s (± 0.55%) ~ 105.89s 107.39s p=0.173 n=6
Emit Time 0.39s (± 2.62%) 0.39s (± 3.82%) ~ 0.38s 0.41s p=0.931 n=6
Total Time 123.92s (± 0.55%) 123.46s (± 0.50%) ~ 122.98s 124.56s p=0.173 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,408 1,231,443 +35 (+ 0.00%) ~ ~ p=0.001 n=6
Types 264,357 264,364 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,352,828k (± 0.03%) 2,348,581k (± 0.02%) -4,247k (- 0.18%) 2,347,907k 2,349,166k p=0.005 n=6
Parse Time 4.98s (± 1.05%) 5.00s (± 1.22%) ~ 4.92s 5.09s p=0.521 n=6
Bind Time 1.89s (± 0.67%) 1.89s (± 0.33%) ~ 1.88s 1.90s p=0.799 n=6
Check Time 34.66s (± 0.33%) 34.68s (± 0.28%) ~ 34.53s 34.78s p=0.689 n=6
Emit Time 3.35s (± 1.63%) 3.35s (± 0.40%) ~ 3.33s 3.37s p=0.518 n=6
Total Time 44.90s (± 0.20%) 44.93s (± 0.35%) ~ 44.71s 45.15s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,408 1,231,443 +35 (+ 0.00%) ~ ~ p=0.001 n=6
Types 264,357 264,364 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,426,608k (± 0.03%) 2,422,457k (± 0.01%) -4,151k (- 0.17%) 2,422,024k 2,422,789k p=0.005 n=6
Parse Time 6.24s (± 1.06%) 6.21s (± 1.17%) ~ 6.10s 6.31s p=0.630 n=6
Bind Time 2.03s (± 0.79%) 2.03s (± 1.31%) ~ 2.00s 2.08s p=1.000 n=6
Check Time 41.50s (± 0.69%) 41.49s (± 0.76%) ~ 41.17s 42.02s p=0.936 n=6
Emit Time 4.12s (± 3.13%) 4.06s (± 0.77%) ~ 4.03s 4.11s p=0.571 n=6
Total Time 53.91s (± 0.52%) 53.81s (± 0.59%) ~ 53.46s 54.30s p=0.689 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,907 256,942 +35 (+ 0.01%) ~ ~ p=0.001 n=6
Types 105,594 105,601 +7 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 429,317k (± 0.03%) 427,495k (± 0.02%) -1,822k (- 0.42%) 427,418k 427,632k p=0.005 n=6
Parse Time 2.79s (± 0.87%) 2.79s (± 0.68%) ~ 2.76s 2.81s p=0.935 n=6
Bind Time 1.09s (± 1.71%) 1.09s (± 1.42%) ~ 1.07s 1.10s p=0.929 n=6
Check Time 15.34s (± 0.32%) 15.34s (± 0.46%) ~ 15.27s 15.47s p=0.521 n=6
Emit Time 1.40s (± 1.65%) 1.40s (± 0.83%) ~ 1.39s 1.42s p=0.935 n=6
Total Time 20.62s (± 0.14%) 20.62s (± 0.29%) ~ 20.55s 20.73s p=0.935 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 225,018 225,018 ~ ~ ~ p=1.000 n=6
Types 94,245 94,245 ~ ~ ~ p=1.000 n=6
Memory used 370,280k (± 0.02%) 368,412k (± 0.02%) -1,868k (- 0.50%) 368,338k 368,512k p=0.005 n=6
Parse Time 3.44s (± 1.07%) 3.46s (± 0.56%) ~ 3.44s 3.49s p=0.292 n=6
Bind Time 1.92s (± 0.85%) 1.92s (± 0.87%) ~ 1.90s 1.95s p=0.743 n=6
Check Time 19.44s (± 0.37%) 19.43s (± 0.37%) ~ 19.35s 19.56s p=0.746 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 24.80s (± 0.34%) 24.81s (± 0.30%) ~ 24.74s 24.93s p=0.936 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 3,006,618 3,006,618 ~ ~ ~ p=1.000 n=6
Types 1,035,512 1,035,512 ~ ~ ~ p=1.000 n=6
Memory used 3,130,082k (± 0.00%) 3,096,911k (± 0.00%) -33,171k (- 1.06%) 3,096,845k 3,096,950k p=0.005 n=6
Parse Time 13.93s (± 0.23%) 13.94s (± 0.35%) ~ 13.89s 14.03s p=0.809 n=6
Bind Time 4.29s (± 0.47%) 4.28s (± 0.44%) ~ 4.26s 4.31s p=0.868 n=6
Check Time 79.94s (± 0.09%) 79.58s (± 0.49%) ~ 79.00s 79.94s p=0.054 n=6
Emit Time 20.55s (± 0.57%) 20.50s (± 0.34%) ~ 20.42s 20.61s p=0.521 n=6
Total Time 118.70s (± 0.12%) 118.31s (± 0.35%) -0.40s (- 0.33%) 117.66s 118.67s p=0.031 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 275,352 275,352 ~ ~ ~ p=1.000 n=6
Types 112,436 112,436 ~ ~ ~ p=1.000 n=6
Memory used 424,147k (± 0.02%) 422,120k (± 0.03%) -2,027k (- 0.48%) 421,987k 422,314k p=0.005 n=6
Parse Time 3.98s (± 0.46%) 3.98s (± 0.33%) ~ 3.96s 4.00s p=0.605 n=6
Bind Time 1.72s (± 0.47%) 1.72s (± 0.80%) ~ 1.70s 1.74s p=0.282 n=6
Check Time 17.41s (± 0.24%) 17.37s (± 0.57%) ~ 17.21s 17.50s p=0.470 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 23.11s (± 0.22%) 23.07s (± 0.37%) ~ 22.92s 23.18s p=0.419 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 534,188 534,188 ~ ~ ~ p=1.000 n=6
Types 176,207 176,207 ~ ~ ~ p=1.000 n=6
Memory used 479,324k (± 0.00%) 472,320k (± 0.01%) -7,005k (- 1.46%) 472,254k 472,356k p=0.005 n=6
Parse Time 4.25s (± 0.68%) 4.24s (± 0.57%) ~ 4.22s 4.28s p=0.684 n=6
Bind Time 1.54s (± 0.78%) 1.55s (± 1.03%) ~ 1.53s 1.57s p=0.410 n=6
Check Time 22.24s (± 0.18%) 22.82s (± 0.32%) +0.58s (+ 2.62%) 22.75s 22.92s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 28.03s (± 0.13%) 28.62s (± 0.27%) +0.58s (+ 2.09%) 28.52s 28.73s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,431ms (± 0.73%) 3,436ms (± 0.32%) ~ 3,419ms 3,452ms p=0.936 n=6
Req 2 - geterr 7,639ms (± 0.52%) 7,548ms (± 0.40%) -92ms (- 1.20%) 7,507ms 7,593ms p=0.005 n=6
Req 3 - references 394ms (± 1.81%) 389ms (± 1.58%) ~ 384ms 400ms p=0.195 n=6
Req 4 - navto 343ms (± 2.79%) 343ms (± 1.42%) ~ 337ms 350ms p=0.747 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 113ms (± 3.01%) 115ms (± 2.48%) ~ 111ms 119ms p=0.226 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,395ms (± 0.33%) 2,407ms (± 1.17%) ~ 2,378ms 2,445ms p=0.810 n=6
Req 2 - geterr 3,920ms (± 0.52%) 3,912ms (± 0.39%) ~ 3,898ms 3,940ms p=0.748 n=6
Req 3 - references 271ms (± 0.77%) 272ms (± 0.59%) ~ 269ms 273ms p=0.285 n=6
Req 4 - navto 236ms (± 2.90%) 229ms (± 2.22%) 🟩-7ms (- 3.04%) 226ms 239ms p=0.036 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 71ms (± 3.98%) 77ms (± 8.28%) ~ 69ms 85ms p=0.126 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,201ms (± 0.29%) 5,189ms (± 0.25%) ~ 5,172ms 5,207ms p=0.261 n=6
Req 2 - geterr 1,126ms (± 0.96%) 1,123ms (± 0.55%) ~ 1,112ms 1,130ms p=1.000 n=6
Req 3 - references 88ms (± 2.94%) 88ms (± 2.92%) ~ 86ms 91ms p=0.784 n=6
Req 4 - navto 443ms (± 0.90%) 444ms (± 1.11%) ~ 440ms 450ms p=0.548 n=6
Req 5 - completionInfo count 3,444 3,444 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 827ms (± 0.49%) 828ms (± 0.83%) ~ 822ms 840ms p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.94ms (± 0.17%) 158.08ms (± 0.19%) +0.14ms (+ 0.09%) 156.95ms 160.67ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 282.98ms (± 0.28%) 282.87ms (± 0.27%) -0.12ms (- 0.04%) 275.88ms 287.99ms p=0.001 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 337.73ms (± 0.30%) 338.02ms (± 0.30%) +0.29ms (+ 0.09%) 330.28ms 345.26ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 277.24ms (± 0.30%) 276.95ms (± 0.29%) -0.28ms (- 0.10%) 269.42ms 283.44ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Hm, forgot about xstate being bad with the original change :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmichon-msft dmichon-msft dmichon-msft approved these changes

At least 1 approving review is required to merge this pull request.

Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
PR Backlog
Status: Waiting on author
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jakebailey @typescript-bot @dmichon-msft @DanielRosenwasser @Andarist @ehoogeveen-medweb