Skip to content

Commit 4a50b89

Browse files
committed
Update benchmark results
1 parent b9ef1d6 commit 4a50b89

File tree

3 files changed

+261
-123
lines changed

3 files changed

+261
-123
lines changed

docs/src/benchmarks.md

Lines changed: 60 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# Benchmarks
22

3-
To attain maximal performance, it is recommended to pass `-O3`, `--check-bounds=no` as command line flags to `julia`. As of Julia 1.1, maximizing performance for the `dynamics!` algorithm requires either setting the number of BLAS threads to 1 (`using LinearAlgebra; BLAS.set_num_threads(1)`) if using OpenBLAS (the default), or compiling Julia with MKL. See [this issue](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/issues/500) for more information.
3+
To attain maximal performance, it is recommended to pass `-O3` and `--check-bounds=no` as command line flags to `julia`.
4+
5+
> **Warning**
6+
> For Julia versions previous to `v1.8`, maximizing performance for the `dynamics!` algorithm requires either setting the number of BLAS threads to 1 (`using LinearAlgebra; BLAS.set_num_threads(1)`) if using OpenBLAS (the default), or compiling Julia with MKL. See [this issue](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/issues/500) for more information.
47
58
Run `perf/runbenchmarks.jl` to see benchmark results for the Atlas robot (v5). Results below are for the following scenarios:
69

@@ -9,69 +12,81 @@ Run `perf/runbenchmarks.jl` to see benchmark results for the Atlas robot (v5). R
912
3. Do inverse dynamics.
1013
4. Do forward dynamics.
1114

12-
Note that results on CI builds are **not at all** representative because of code coverage. Results on a reasonably fast laptop at commit [870bea6](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/commit/870bea668d5b11ce0555fa0552592d2c3cb15c54):
15+
> **Note**
16+
> Results on CI builds are **not at all** representative because of code coverage.
1317
14-
Output of `versioninfo()`:
18+
Here are the results on an Apple MacBook Air (M1, 2020) (8GB RAM, 512GB SSD) at commit [b9ef1d](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/commit/b9ef1d6974beff4d4fbe7dffc6dbfa65f71e0132):
1519

20+
Output of `versioninfo()`:
1621
```
17-
Julia Version 1.5.3
18-
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
22+
Julia Version 1.8.5
23+
Commit 17cfb8e65e* (2023-01-08 06:45 UTC)
1924
Platform Info:
20-
OS: macOS (x86_64-apple-darwin18.7.0)
21-
CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
25+
OS: macOS (arm64-apple-darwin22.1.0)
26+
CPU: 8 × Apple M1
2227
WORD_SIZE: 64
2328
LIBM: libopenlibm
24-
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
29+
LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1)
30+
Threads: 1 on 4 virtual cores
2531
```
2632

27-
Note that this is a different machine than the one that was used for earlier benchmarks.
33+
> **Note**
34+
> This is a different machine than the one that was used for earlier benchmarks.
2835
29-
Mass matrix:
30-
31-
```
32-
memory estimate: 0 bytes
33-
allocs estimate: 0
34-
--------------
35-
minimum time: 4.415 μs (0.00% GC)
36-
median time: 4.579 μs (0.00% GC)
37-
mean time: 4.916 μs (0.00% GC)
38-
maximum time: 19.794 μs (0.00% GC)
36+
Mass matrix ([`mass_matrix!`](@ref)):
3937
```
38+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
39+
Range (min … max): 3.796 μs … 6.183 μs ┊ GC (min … max): 0.00% … 0.00%
40+
Time (median): 3.858 μs ┊ GC (median): 0.00%
41+
Time (mean ± σ): 3.879 μs ± 93.310 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
4042
41-
Mass matrix and Jacobian from left hand to right foot:
43+
▁▄▄▆▆██▇█▆▇▄▅▄▂▃▁▃▂▃▃▁▃▁▂ ▁ ▂
44+
▅█████████████████████████████▇█▇█▇▆▇▅▆▅▆▅▅▆▅▆▄▇▆█▇▅▆▆▇▅▄▄ █
45+
3.8 μs Histogram: log(frequency) by time 4.18 μs <
4246
47+
Memory estimate: 0 bytes, allocs estimate: 0.
4348
```
44-
memory estimate: 0 bytes
45-
allocs estimate: 0
46-
--------------
47-
minimum time: 4.860 μs (0.00% GC)
48-
median time: 4.982 μs (0.00% GC)
49-
mean time: 5.399 μs (0.00% GC)
50-
maximum time: 24.712 μs (0.00% GC)
49+
50+
Mass matrix ([`mass_matrix!`](@ref)) and Jacobian ([`geometric_jacobian!`](@ref)) from left hand to right foot:
51+
```julia
52+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
53+
Range (min max): 4.042 μs 6.763 μs ┊ GC (min max): 0.00% 0.00%
54+
Time (median): 4.167 μs ┊ GC (median): 0.00%
55+
Time (mean ± σ): 4.187 μs ± 115.303 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
56+
57+
▁▁▂▁▁ ▁▃▄▆▇██▇█▇▆▄▃▃▃▃▁▃▃▃▂▂▁▁ ▃
58+
▃▅███████████████████████████████▇██▇██▆▇▆▄▆▅▆▇▇██▆██▆▆▅▁▅▄ █
59+
4.04 μs Histogram: log(frequency) by time 4.5 μs <
60+
61+
Memory estimate: 0 bytes, allocs estimate: 0.
5162
```
5263

5364
Note the low additional cost of computing a Jacobian when the mass matrix is already computed. This is because RigidBodyDynamics.jl caches intermediate computation results.
5465

55-
Inverse dynamics:
66+
Inverse dynamics ([`inverse_dynamics!`](@ref)):
67+
```julia
68+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
69+
Range (min max): 2.700 μs 5.700 μs ┊ GC (min max): 0.00% 0.00%
70+
Time (median): 2.767 μs ┊ GC (median): 0.00%
71+
Time (mean ± σ): 2.773 μs ± 79.491 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
5672

57-
```
58-
memory estimate: 0 bytes
59-
allocs estimate: 0
60-
--------------
61-
minimum time: 4.256 μs (0.00% GC)
62-
median time: 4.541 μs (0.00% GC)
63-
mean time: 4.831 μs (0.00% GC)
64-
maximum time: 21.625 μs (0.00% GC)
73+
▁▁█
74+
▂▂▂▂▂▂▃▅▅▆▇█████▇▆▅▄▃▄▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▂▂▁▂▂▂▁▂▂▂▂ ▃
75+
2.7 μs Histogram: frequency by time 2.98 μs <
76+
77+
Memory estimate: 0 bytes, allocs estimate: 0.
6578
```
6679

67-
Forward dynamics:
80+
Forward dynamics ([`dynamics!`](@ref)):
81+
```julia
82+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
83+
Range (min max): 10.554 μs 23.692 μs ┊ GC (min max): 0.00% 0.00%
84+
Time (median): 10.667 μs ┊ GC (median): 0.00%
85+
Time (mean ± σ): 10.703 μs ± 212.557 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
6886

69-
```
70-
memory estimate: 0 bytes
71-
allocs estimate: 0
72-
--------------
73-
minimum time: 13.600 μs (0.00% GC)
74-
median time: 14.419 μs (0.00% GC)
75-
mean time: 16.071 μs (0.00% GC)
76-
maximum time: 55.328 μs (0.00% GC)
87+
▃▇█▇▅▂
88+
▁▁▂▂▄███████▇▄▃▂▂▂▁▁▁▁▁▁▁▁▁▁▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
89+
10.6 μs Histogram: frequency by time 11.3 μs <
90+
91+
Memory estimate: 0 bytes, allocs estimate: 0.
7792
```

0 commit comments

Comments
 (0)