Record time at start of every LBFGS iteration

[leburgel]

The starting time t₀ was only recorded once at the start of the LBFGS run, and was kept constant throughout the rest of the iterations such that the printed time was the elapsed time since the start of the algorithm, not the time of the single iteration.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.10%. Comparing base (e476e82) to head (7170ee6).

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #23      +/-   ##
==========================================
+ Coverage   82.06%   82.10%   +0.03%     
==========================================
  Files           5        5              
  Lines         513      514       +1     
==========================================
+ Hits          421      422       +1     
  Misses         92       92              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jutho]

I think this was actually intentional. See also the last code block after the loop:

    if _hasconverged
        verbosity >= 2 &&
            @info @sprintf("GD: converged after %d iterations and time %.2f s: f = %.12f, ‖∇f‖ = %.4e",
                           numiter, t, f, normgrad)
    else
        verbosity >= 1 &&
            @warn @sprintf("GD: not converged to requested tol after %d iterations and time %.2f s: f = %.12f, ‖∇f‖ = %.4e",
                           numiter, t, f, normgrad)
    end

Hence, t is supposed to represent the total time elapsed.

I agree it would make sense to show some Δt per time step instead of total time, but that requires something along the lines of

told = t
t = time() - t₀
Δt = t - told

Can you change PR accordingly, and maybe then make this uniform across LBFGS, CG and GD ?