Investigate GPU Offloading

[huttered40]

Although algorithm (static) class templates should not care about where computation is performed (CPU or GPU), I think there are a few design choices that motivate parameterizing the algorithm itself instead of the matrix class. However, there are still reasons for parameterizing the matrix class, one of which is because polymorphic data containers (kokkos for example) do the same thing, and such data types should be able to plug into distributed-memory algorithms without any pain.

Think about the pros and cons here.

Three policy classes for offloading (just gemm for now) include:

1. NoOffload (default)
2. OffloadKeepDataResident (keep data on GPU as much as possible. Any communication of data on GPU is not a problem, but remember that it still must pass through PCI bus, exploit pinned memory via buffer allocated at the beginning of the program that is used repeatedly).
3. OffloadTransferData (make not attempt to keep data resident on GPUs. Offload for each gemm invocation. Mainly a sanity-check policy class)

• Don't forget to Incorporate into validate class templates as well.
• Modify all test.cpp files to initially allocate memory on device.
• Modify all Makefiles to use nvcc compiler and corresponding flags. Note that anything compiled with nvcc must be separate from anything compiled with MPI
• Update blas directory to allow cuBlas headers.
• Update bench/ files to include Offload policy class
• Replace any syrk or trmm calls with gemm? Will that interfere with algorithm-specific policies (via non-orthogonal policy classes)?

[huttered40]

GPU offloading and algorithm parameterization should be kept separate. GPU Offloading should work with either of CuBLAS's trmm, syrk, or gemm routines, and policy classes parameterizing an algorithm should neither know nor care if the computation is being offloaded. Of course, the user will be aware and thus choose his parametrization carefully.

[huttered40]

GPU offloading and algorithm parameterization should be kept separate. GPU Offloading should work with either of CuBLAS's trmm, syrk, or gemm routines, and policy classes parameterizing an algorithm should neither know nor care if the computation is being offloaded. Of course, the user will be aware and thus choose his parametrization carefully.

This idea is good. I think we can get away with the entire library, besides the blas and lapack directories, not knowing about GPU offloading. I don't think the matrix class needs to know anything about GPU offloading. If we make the user specify this somewhere else, then all that really needs to be changed is which CuBLAS routine is called at the very bottom layer. Wait, but the matrix class does need to know how to allocate the data on the GPU.

Remember that we want to keep data resident on the GPU as much as possible. Is there anything wrong with our current code in that regard? I think we could do this without having to parameterize anything on being GPU resident or not (might be needed for testing, but not later for production).

[huttered40]

No, the matrix class template does definitely need to be parameterized as discussed way above. However, that does not mean that the class that uses the Cuda-specific methods can't be compiled separately using nvcc. That shouldn't be an issue. The default should be no-offloading.

[huttered40]

Places where offloading could be tricky:

• serialize
• Does the Offload type parameter of matrix class template force non-orthogonal policies? For example, I'm concerned that Offload will be related to the methods in Structure class, as it includes things like copy and allocate methods. I don't want to parameterize these structure classes because that also will force many changes to serialize.
• distribute
• Should the lapack routines force a transfer from device to host? I don't think so. Even though they might not be faster, I don't want to incur that overhead
• blas::engine
• lapack::engine
• Pre-compiling blas::engine and lapack::engine calls, is this necessary? We should be calling binaries anyways.
• util::residual_local
• util::blockedToCyclicSpecial
• util::blockedToCyclic
• util::getReferenceMatrix
• util::removeTriangle
• bench
• summa::start2 not as concerned with this one, since we don't use it anyways
• summa::BroadcastPanels we have a new call in here

After investigating these, we can try a simple matmult::summa.