Feature/mrhs solvers #1489

maddyscientist · 2024-09-05T18:15:23Z

This is PR is a biggie:

Adds MRHS solvers to QUDA (at present CG, MR, SD, GCR, CA-GCR) are all implemented
- Reliable updates performed using 0th RHS
- Do not flag convergence until all RHS are converged
Adds MRHS support to multigrid
- Batched null space finding implemented, exposed with new parameter QudaMultigridParam::n_vec_batch
- MRHS supported in actual solves as well
Expose MRHS solvers using invertMultiSrcQuda interface
- This is compatible with the split-grid interface, batching is used across the number of sources per sub grid
Explicit breaking of the interface
- QudaInvertParam::true_res and QudaInvertParam::true_res_hq are now arrays
Batched deflation implemented
- Batch eigenvalue deflation confirmed working using staggered fermions
- Batch singular-value deflation confirmed working as a coarse grid deflator
Eigenvalue computation is now batch, to take advantage of MRHS
All Dirac::prepare and Dirac::reconstruct functions are now MRHS optimized
Chronological solver is now MRHS optimized
Cast from cvector<T> to T is now explicit instead of implicit
- This makes it much easier to catch bugs when updating code to be MRHS
Tensor-core 3xFP16 DslashCoarse is now robust to underflow
Miscellaneous cleanup and additions to aid all of the above
Fixes some earlier bugs introduced in prior MRHS PRs
Add MMA instantiations for 32 -> 64 coarsening
Since QUDA is now threaded, update to using MPI_THREAD_FUNNELED
Augmentations to the power monitoring

Things left to do

Add real MRHS support to all solvers (or at least to a few outstanding ones: CGNE, CGNR, BiCGStab, etc.)
Verify NVSHMEM MRHS operators are all working as expected
Update QUDA version number due to breakage of interface (QUDA 2.0?)
Fix reporting of true residual per RHS when running split grid

… size to use when generating null-space vectors. This is the parameter used to enable MRHS null space generation. Updated the null-space generation to work on vectors of this width

…-> scalar cast operator explicit instead of implicit

…ch - will be needed for batched CG

…ing solvers

…rator explict

… issues at compile time, the scalar -> vector<scalar> cast operator is now explicit. Apply necessary changes to ensure that stuff doesn't break with this change

…olvers

…ossible

…of invertQuda to new function solve which is MRHS aware

… also now called by the invertMultiSrcQuda interface

lib/inv_cg_quda.cpp

lib/inv_gcr_quda.cpp

weinbe2 · 2024-09-06T00:40:27Z

lib/milc_interface.cpp

@@ -1426,8 +1426,9 @@ void qudaInvertMsrc(int external_precision, int quda_precision, double mass, Qud

  // return the number of iterations taken by the inverter
  *num_iters = invertParam.iter;


I believe MILC expects the aggregate number of iterations as if it wasn't a batch/block solve, but I forget. I'll also scope out how it expects the residuals as part of the work I'm doing on the MILC side and sort this out

Note to self, can confirm that is the case: https://github.com/lattice/milc_qcd/blob/develop/generic_ks/d_congrad5_fn_milc.c#L409-L417

lib/multigrid.cpp

tests/invert_test.cpp

tests/utils/command_line_params.cpp

tests/utils/host_utils.cpp

tests/utils/set_params.cpp

include/invert_quda.h

… have single implementations of these, which use a solver factor to call the correct CG method

weinbe2 · 2024-09-17T18:17:30Z

lib/inv_ca_cg.cpp


        // break-out check if we have reached the limit of the precision
        if (r2 > r2_old) {
          resIncrease++;
          resIncreaseTotal++;
          warningQuda(
            "CA-CG: new reliable residual norm %e is greater than previous reliable residual norm %e (total #inc %i)",
-            sqrt(r2), sqrt(r2_old), resIncreaseTotal);
+            sqrt(r2[0]), sqrt(r2_old[9]), resIncreaseTotal);


r2_old[0] not r2_old[9] :)

weinbe2 · 2024-09-19T22:31:03Z

lib/solver.cpp

    return true;
  }

-  bool Solver::convergenceHQ(double, double hq2, double, double hq_tol)
+  bool Solver::convergenceHQ(cvector<double> &hq2, cvector<double> &hq_tol)


There's a big "gotcha" that happens here: in, for example, inv_cg_quda.cpp, this routine is called as:

convergenceHQ(hq_res, param.tol_hq)

Where param.tol_hq is just a double. Within convergenceHQ (and elsewhere), this is blindly converted to a cvector<double> where the 0th element is param.tol_hq and then the rest are uninitialized. This gives garbage for the other values, messing up convergence.

I think the only way to catch this is to make a conversion from a type T to cvector<T> require an explicit cast, which I think you've imposed on other types as well.

maddyscientist added 30 commits June 21, 2024 10:36

Fix warning in spin taste and minor cleanup

4951f11

Some cleanup of CG interface

7c4793b

Add MRHS interface for all solvers, and mandate source vector is const

9a2190c

Optimize DiracWilson: vectorize the prepare/reconstruct functions

fda4669

Small cleanup to block_transpose.in.cu

44fb98a

Add new parameter: QudaMultigridParam::n_vec_batch which is the batch…

fdd40fb

… size to use when generating null-space vectors. This is the parameter used to enable MRHS null space generation. Updated the null-space generation to work on vectors of this width

Vectorize DiracCoarsePC prepare/reconstruct

fa64adf

Ensure we don't enable large arg support for pre Volta architecture

fef58e8

Create vector variants of create_alias

8b8cd99

Add some more scalar wrappers: this facilitates us making the vector …

ee6fd26

…-> scalar cast operator explicit instead of implicit

Supress annoying warning with Eigen

ac23c73

Add default copy/move constructors/assignment operator for XUpdateBat…

2b0763b

…ch - will be needed for batched CG

Add some useful overloads to vector class to facilitate writing batch…

70a94df

…ing solvers

Add explicit casting to double in anticipation of making the cast ope…

f792a33

…rator explict

First pass at enabling MRHS for CG, MR and SD solvers. To better find…

57ba15e

… issues at compile time, the scalar -> vector<scalar> cast operator is now explicit. Apply necessary changes to ensure that stuff doesn't break with this change

Accelerate MG::verify by using batch blas where applicable

224bdb2

Fix bug in MRE solver

6256391

Apply MRHS optimization to MRE solver

7962dc3

Remove complex.h inclusion

075cfb8

Vectorize all remaining Dirac prepare/reconstruct functions

7cbab27

Fix bug in GammaApply with introduced in #1416

d488607

Fix issue with CG::hq_solve

6d1bafe

Merge branch 'develop' of github.com:lattice/quda into feature/mrhs-s…

d526544

…olvers

Fix bug with Clover Hasenbsusch operator (wrong braces)

902d8ab

Fix bug with DiracCoarsePC::reconstruct when using odd solve

02eecaa

Fix bug with counting bytes with clover operator

8b067d4

Default inner GCR solver to use L2 residual to enable early exit if p…

c682cae

…ossible

Initial work to prepare for multi-rhs solver exposure: move the body …

1c5baef

…of invertQuda to new function solve which is MRHS aware

Fix flops counters for blas and reduce functions

faf4658

Move remainder of invertQuda body into new MRHS solve wrapper that is…

b2f9849

… also now called by the invertMultiSrcQuda interface

weinbe2 reviewed Sep 5, 2024

View reviewed changes

lib/inv_cg_quda.cpp Outdated Show resolved Hide resolved

Fix CI warnings (one of which was a real bug)

0ed1fe9

weinbe2 reviewed Sep 6, 2024

View reviewed changes

lib/inv_gcr_quda.cpp Outdated Show resolved Hide resolved

weinbe2 reviewed Sep 6, 2024

View reviewed changes

lib/inv_gcr_quda.cpp Outdated Show resolved Hide resolved

weinbe2 reviewed Sep 6, 2024

View reviewed changes

lib/multigrid.cpp Show resolved Hide resolved

weinbe2 reviewed Sep 6, 2024

View reviewed changes

tests/invert_test.cpp Show resolved Hide resolved

weinbe2 reviewed Sep 6, 2024

View reviewed changes

tests/utils/command_line_params.cpp Outdated Show resolved Hide resolved

weinbe2 reviewed Sep 6, 2024

View reviewed changes

tests/utils/host_utils.cpp Show resolved Hide resolved

weinbe2 reviewed Sep 6, 2024

View reviewed changes

tests/utils/set_params.cpp Show resolved Hide resolved

maddyscientist added 6 commits September 5, 2024 22:13

More CI warnings

75a3b4c

Fix some outstanding CI issues

37a6ae9

renaming as suggested in CI

fc0762d

Use std::vector iterator constructor

44d8a2a

Revert change made in this branch

e89be7d

Cleanup of DiracCloverHasenbuschTwistPC

dcd0d43

weinbe2 reviewed Sep 6, 2024

View reviewed changes

include/invert_quda.h Outdated Show resolved Hide resolved

maddyscientist added 9 commits September 6, 2024 16:25

Cleanup ero source checking in the solvers

f0d9f3c

CGNR and CGNE are now MRHS

36c9b6d

CG3 is now MRHS

d5c6708

Remove derived CGNR and CGNE specialziations for CG/CA-CG/CG3: we now…

2e52dad

… have single implementations of these, which use a solver factor to call the correct CG method

Optimize HQ in CA-GCR

0803993

CA-CG is now MRHS

e1589e5

BiCGStab is now MRHS

a3186a0

BiCGStab(l) is now multi-RHS

cbf5943

Fix typo. Closes #1492

dce23c6

weinbe2 reviewed Sep 17, 2024

View reviewed changes

maddyscientist added 2 commits September 18, 2024 08:53

Use fine-grain parallelization for CopySpinor

827700d

Ensure that mg_eig_evals_batch_size in test code has sensible default

ade0e16

weinbe2 reviewed Sep 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/mrhs solvers #1489

Feature/mrhs solvers #1489

maddyscientist commented Sep 5, 2024 •

edited

Loading

weinbe2 Sep 6, 2024

weinbe2 Sep 19, 2024

weinbe2 Sep 17, 2024

weinbe2 Sep 19, 2024

		@@ -1426,8 +1426,9 @@ void qudaInvertMsrc(int external_precision, int quda_precision, double mass, Qud

		// return the number of iterations taken by the inverter
		*num_iters = invertParam.iter;

Feature/mrhs solvers #1489

Are you sure you want to change the base?

Feature/mrhs solvers #1489

Conversation

maddyscientist commented Sep 5, 2024 • edited Loading

weinbe2 Sep 6, 2024

Choose a reason for hiding this comment

weinbe2 Sep 19, 2024

Choose a reason for hiding this comment

weinbe2 Sep 17, 2024

Choose a reason for hiding this comment

weinbe2 Sep 19, 2024

Choose a reason for hiding this comment

maddyscientist commented Sep 5, 2024 •

edited

Loading