Performance regression in AC_hamiltonian from MPSKit v0.13.10 to v0.13.11

[ZongYongyue]

Hi Lukas,

I noticed a possible performance regression in AC_hamiltonian when upgrading MPSKit from v0.13.10 to v0.13.11.

With the same benchmark setup, the runtime increased by about 3x, and both allocations and memory usage also increased significantly.
The benchmark file:

using Pkg
Pkg.activate("./benchmark/v0.13.11")#Pkg.activate("./benchmark/v0.13.10")
using TensorKit
using MPSKit
using BenchmarkTools
using MPSKitModels
using MPSKit:AC_hamiltonian

H = hubbard_model(ComplexF64, U1Irrep, U1Irrep, FiniteStrip(6, 36);
             t=1.0, U=8.0)
I = FermionParity ⊠ U1Irrep ⊠ U1Irrep
Vs = repeat([Vect[I]((Int(!isinteger(i)), j, i) => 1  for i in -1//2:1//2:1//2 for j in 0:1:2),], length(H) - 1)
Ps = Vect[I]( (0, 0, 0) => 1, (1, 1, 1 // 2) => 1, (1, 1, -1 // 2) => 1, (0, 2, 0) => 1)
ψ = FiniteMPS(rand, ComplexF64, [Ps for _ in 1:length(H)], Vs)
envs = environments(ψ, H)
pos = 18

@benchmark AC_hamiltonian($pos, $ψ, $H, $ψ, $envs)

The results for v0.13.11:

BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  270.792 μs …  38.010 ms  ┊ GC (min … max): 0.00% … 99.10%
 Time  (median):     279.584 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   296.238 μs ± 519.850 μs  ┊ GC (mean ± σ):  4.80% ±  3.07%

     ▄▅▂ ▄█▇▁                                                    
  ▁▂▅████████▇▇▆▅▄▄▄▃▃▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  271 μs           Histogram: frequency by time          331 μs <

 Memory estimate: 224.31 KiB, allocs estimate: 3276.

  Activating project at `~/Library/Mobile Documents/com~apple~CloudDocs/mygit/projects/005_tWSe2_SC/benchmark/v0.13.11`
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  280.667 μs …  10.841 ms  ┊ GC (min … max): 0.00% … 96.77%
 Time  (median):     288.334 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   303.150 μs ± 352.033 μs  ┊ GC (mean ± σ):  4.00% ±  3.35%

    ▃█▅ ▁▃▂                                                      
  ▁▄████████▆▆▆▅▄▄▃▃▂▂▂▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  281 μs           Histogram: frequency by time          342 μs <

 Memory estimate: 245.31 KiB, allocs estimate: 3476.

  Activating project at `~/Library/Mobile Documents/com~apple~CloudDocs/mygit/projects/005_tWSe2_SC/benchmark/v0.13.11`
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  279.083 μs …  11.605 ms  ┊ GC (min … max): 0.00% … 97.08%
 Time  (median):     288.333 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   302.309 μs ± 351.142 μs  ┊ GC (mean ± σ):  3.82% ±  3.20%

      ▆█▃▁▅▇▃                                                    
  ▁▁▂▆███████▇▆▅▄▄▃▃▃▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  279 μs           Histogram: frequency by time          342 μs <

 Memory estimate: 245.31 KiB, allocs estimate: 3476.

The results for v0.13.10:

 Activating project at `~/Library/Mobile Documents/com~apple~CloudDocs/mygit/projects/005_tWSe2_SC/benchmark/v0.13.10`
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):   93.416 μs …  10.863 ms  ┊ GC (min … max): 0.00% … 98.01%
 Time  (median):      97.291 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   102.598 μs ± 212.520 μs  ┊ GC (mean ± σ):  4.11% ±  1.97%

       ▃▃▄█▆▃                                                    
  ▁▁▃▆███████▇▅▄▃▃▃▃▂▂▂▂▂▁▂▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  93.4 μs          Histogram: frequency by time          120 μs <

 Memory estimate: 77.75 KiB, allocs estimate: 1171.

  Activating project at `~/Library/Mobile Documents/com~apple~CloudDocs/mygit/projects/005_tWSe2_SC/benchmark/v0.13.10`
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):   94.000 μs …  14.540 ms  ┊ GC (min … max): 0.00% … 98.23%
 Time  (median):      98.750 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   105.478 μs ± 265.247 μs  ┊ GC (mean ± σ):  4.97% ±  1.97%

    ▃██▅▅▅▇▅▆▅▄▁                                                 
  ▁▄████████████▇▅▄▄▄▄▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▂▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
  94 μs            Histogram: frequency by time          125 μs <

 Memory estimate: 77.75 KiB, allocs estimate: 1171.

  Activating project at `~/Library/Mobile Documents/com~apple~CloudDocs/mygit/projects/005_tWSe2_SC/benchmark/v0.13.10`
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):   93.208 μs … 165.725 ms  ┊ GC (min … max):  0.00% … 99.60%
 Time  (median):      99.666 μs               ┊ GC (median):     0.00%
 Time  (mean ± σ):   131.588 μs ±   1.791 ms  ┊ GC (mean ± σ):  19.49% ±  1.98%

   ▂▅█▃                                                          
  ▂████▇▆▅▅▄▄▄▃▃▃▃▃▃▃▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  93.2 μs          Histogram: frequency by time          162 μs <

 Memory estimate: 77.75 KiB, allocs estimate: 1171.

[lkdvos]

I think this is expected, we moved some of the computations of the application step into the effective hamiltonian constructor, which should make repeated applications (such as in the eigensolver or TDVP exponentiator) faster overall.
For a full discussion and benchmarks, see #345.

I'm not saying that I am 100% sure there is no regression, but it might be more reasonable to look at the combination of computing + applying. This is also what is plotted in these benchmarks, where we do the combination of AC_hamiltonian() * AC, preferably where you only construct the hamiltonian once and apply it a number of times.
Do you also see regressions in the total runtime of the algorithms?

[ZongYongyue]

Thanks for pointing this out. I will read issue #345 and look into the discussion more carefully. Before that, I wanted to share one full-algorithm data point. In this small test case, it seems that there may indeed be a regression in the total runtime.

I test a U(1) x U(1) Hubbard model on a 4 x 4 triangular lattice, including hoppings up to fourth neighbors. In my CBE-DMRG code, I use AC_hamiltonian from MPSKit and eigsolve from KrylovKit@v0.10.3. All runs were performed locally with 8 Julia threads.

I find that, with the default prepare=true path of AC_hamiltonian in MPSKit v0.13.12, the eigensolver section becomes significantly slower in wall time, even though it allocates less memory. Compared with v0.13.10, the full run goes from 138 s to 223 s, and the eigsolve section goes from 115 s to 193 s.
Setting prepare=false in AC_hamiltonian mostly restores the v0.13.10 behavior. The total time becomes 155 s, and the eigsolve time becomes 131 s, with allocations close to v0.13.10.

The run logs are as follow:

using MPSKit@v0.13.10:

CBE-DMRG1 Sweep Started: 14.May 2026 13:11
[1/6] CBE-DMRG1 sweep | duration: 11 seconds, 387 milliseconds
  E₀ = -129.2007929145 | D =   64 | ΔE = 1.479e+02 | ϵp = NaN | err² = 1.530e-03
[2/6] CBE-DMRG1 sweep | duration: 11 seconds, 632 milliseconds
  E₀ = -132.8370208730 | D =   64 | ΔE = 3.636e+00 | ϵp = NaN | err² = 3.569e-03
[3/6] CBE-DMRG1 sweep | duration: 17 seconds, 944 milliseconds
  E₀ = -137.0826193438 | D =  128 | ΔE = 4.246e+00 | ϵp = NaN | err² = 1.223e-03
[4/6] CBE-DMRG1 sweep | duration: 16 seconds, 481 milliseconds
  E₀ = -137.1661414464 | D =  128 | ΔE = 8.352e-02 | ϵp = NaN | err² = 1.436e-03
[5/6] CBE-DMRG1 sweep | duration: 41 seconds, 476 milliseconds
  E₀ = -139.9060810136 | D =  256 | ΔE = 2.740e+00 | ϵp = NaN | err² = 6.283e-04
[6/6] CBE-DMRG1 sweep | duration: 38 seconds, 847 milliseconds
  E₀ = -139.9430690811 | D =  256 | ΔE = 3.699e-02 | ϵp = NaN | err² = 6.392e-04
Ended: 14.May 2026 13:13 | total duration: 2 minutes, 17 seconds, 861 milliseconds
────────────────────────────────────────────────────────────────────────────
                                   Time                    Allocations      
                          ───────────────────────   ────────────────────────
    Tot / % measured:           138s /  97.8%            234GiB /  99.8%    

Section           ncalls     time    %tot     avg     alloc    %tot      avg
────────────────────────────────────────────────────────────────────────────
eigsolve             180     115s   85.1%   640ms    206GiB   88.2%  1.14GiB
CBE expand           180    19.7s   14.5%   109ms   27.4GiB   11.7%   156MiB
  CBE project        100    18.5s   13.7%   185ms   26.6GiB   11.4%   272MiB
  CBE SVD            100    555ms    0.4%  5.55ms    291MiB    0.1%  2.91MiB
  CBE nullspace      100    361ms    0.3%  3.61ms    367MiB    0.2%  3.67MiB
  CBE concat         100    189ms    0.1%  1.89ms    193MiB    0.1%  1.93MiB
SVD trunc            180    441ms    0.3%  2.45ms    145MiB    0.1%   826KiB
────────────────────────────────────────────────────────────────────────────

using MPSKit@v0.13.12, with prepare=true in AC_hamiltonian:

CBE-DMRG1 Sweep Started: 14.May 2026 13:34
[1/6] CBE-DMRG1 sweep | duration: 7 seconds, 800 milliseconds
  E₀ = -129.7361398719 | D =   64 | ΔE = 1.471e+02 | ϵp = NaN | err² = 2.918e-03
[2/6] CBE-DMRG1 sweep | duration: 7 seconds, 835 milliseconds
  E₀ = -132.8724889075 | D =   64 | ΔE = 3.136e+00 | ϵp = NaN | err² = 3.347e-03
[3/6] CBE-DMRG1 sweep | duration: 20 seconds, 169 milliseconds
  E₀ = -137.0878164643 | D =  128 | ΔE = 4.215e+00 | ϵp = NaN | err² = 1.206e-03
[4/6] CBE-DMRG1 sweep | duration: 20 seconds, 209 milliseconds
  E₀ = -137.1738073622 | D =  128 | ΔE = 8.599e-02 | ϵp = NaN | err² = 1.432e-03
[5/6] CBE-DMRG1 sweep | duration: 1 minute, 17 seconds, 516 milliseconds
  E₀ = -139.9022513680 | D =  256 | ΔE = 2.728e+00 | ϵp = NaN | err² = 6.292e-04
[6/6] CBE-DMRG1 sweep | duration: 1 minute, 28 seconds, 957 milliseconds
  E₀ = -139.9440375704 | D =  256 | ΔE = 4.179e-02 | ϵp = NaN | err² = 6.524e-04
Ended: 14.May 2026 13:38 | total duration: 3 minutes, 42 seconds, 644 milliseconds
────────────────────────────────────────────────────────────────────────────
                                   Time                    Allocations      
                          ───────────────────────   ────────────────────────
    Tot / % measured:           223s /  98.2%            187GiB /  99.7%    

Section           ncalls     time    %tot     avg     alloc    %tot      avg
────────────────────────────────────────────────────────────────────────────
eigsolve             180     193s   87.7%   1.07s    159GiB   85.2%   903MiB
CBE expand           180    26.4s   12.0%   147ms   27.5GiB   14.8%   156MiB
  CBE project        100    25.4s   11.6%   254ms   26.7GiB   14.3%   273MiB
  CBE SVD            100    624ms    0.3%  6.24ms    291MiB    0.2%  2.91MiB
  CBE concat         100    188ms    0.1%  1.88ms    191MiB    0.1%  1.91MiB
  CBE nullspace      100    187ms    0.1%  1.87ms    367MiB    0.2%  3.67MiB
SVD trunc            180    486ms    0.2%  2.70ms    146MiB    0.1%   830KiB
────────────────────────────────────────────────────────────────────────────

using MPSKit@v0.13.12 with prepare=false in AC_hamiltonian:

CBE-DMRG1 Sweep Started: 14.May 2026 14:58
[1/6] CBE-DMRG1 sweep | duration: 10 seconds, 490 milliseconds
  E₀ = -129.3572168479 | D =   64 | ΔE = 1.456e+02 | ϵp = NaN | err² = 1.653e-03
[2/6] CBE-DMRG1 sweep | duration: 10 seconds, 831 milliseconds
  E₀ = -132.8055736636 | D =   64 | ΔE = 3.448e+00 | ϵp = NaN | err² = 3.596e-03
[3/6] CBE-DMRG1 sweep | duration: 19 seconds, 3 milliseconds
  E₀ = -137.1022665355 | D =  128 | ΔE = 4.297e+00 | ϵp = NaN | err² = 1.217e-03
[4/6] CBE-DMRG1 sweep | duration: 14 seconds, 751 milliseconds
  E₀ = -137.1893532659 | D =  128 | ΔE = 8.709e-02 | ϵp = NaN | err² = 1.358e-03
[5/6] CBE-DMRG1 sweep | duration: 49 seconds, 2 milliseconds
  E₀ = -139.9088937304 | D =  256 | ΔE = 2.720e+00 | ϵp = NaN | err² = 6.146e-04
[6/6] CBE-DMRG1 sweep | duration: 50 seconds, 507 milliseconds
  E₀ = -139.9518356254 | D =  256 | ΔE = 4.294e-02 | ϵp = NaN | err² = 6.267e-04
Ended: 14.May 2026 15:01 | total duration: 2 minutes, 34 seconds, 712 milliseconds
────────────────────────────────────────────────────────────────────────────
                                   Time                    Allocations      
                          ───────────────────────   ────────────────────────
    Tot / % measured:           155s /  97.4%            235GiB /  99.8%    

Section           ncalls     time    %tot     avg     alloc    %tot      avg
────────────────────────────────────────────────────────────────────────────
eigsolve             180     131s   86.4%   726ms    207GiB   88.2%  1.15GiB
CBE expand           180    20.2s   13.4%   112ms   27.5GiB   11.7%   156MiB
  CBE project        100    19.0s   12.5%   190ms   26.6GiB   11.4%   273MiB
  CBE SVD            100    557ms    0.4%  5.57ms    293MiB    0.1%  2.93MiB
  CBE nullspace      100    348ms    0.2%  3.48ms    371MiB    0.2%  3.71MiB
  CBE concat         100    344ms    0.2%  3.44ms    194MiB    0.1%  1.94MiB
SVD trunc            180    386ms    0.3%  2.14ms    146MiB    0.1%   829KiB
────────────────────────────────────────────────────────────────────────────

[lkdvos]

Thank you for sharing this! I would like to investigate this a bit more in detail, would you maybe be open to sharing this code with me? My email is on my github profile, in case you prefer to not share this publicly. I might not find the time to investigate this week, but next week should be better!

I of course don't know everything that is going on, but it is interesting to me that it seems like all of the calls in the prepare=true case seem to be taking longer (including the SVD trunc, which should be unaffected, I think?) In any case, I'd love to play around a bit more with this, and it's nice to be able to do so with an actual real-world use-case (the Heisenberg tests I ran might just not be representative enough 😢 )

[ZongYongyue]

Thank you very much for being willing to look into this. I will send them to your email later.