(3) Building COGENT

To build the code on NERSC:

1. Set the following environmental parameters (you can include those lines into your source file, e.g., .cshrc.ext)
   module swap PrgEnv-intel PrgEnv-gnu
   module load cray-hdf5-parallel
   setenv CC cc
   setenv CXX CC
   setenv F77 ftn
   setenv CPP cpp
   

   Note: for Bash, use 'export CC="cc"' etc.

2. Build hypre (see COGENT/hypre/INSTALL for details).
   cd COGENT/hypre/src
   make distclean (if need to clean previous build data)
   ./configure --enable-maxdim=5
   make -j16 install
   

   Note: using more that 16 parallel threads can slow down or even freeze other users who are sharing the same Node, since the total number of threads is 32.

3. After building hypre, make sure that HYPRE_LOC variable in COGENT/exec/GNUmakefile.multidim points to the hypre installation directory.
   If you run hypre configure script without the prefix flag option (as in the instructions above), you should edit manually the file COGENT/exec/GNUmakefile.multidim and set HYPRE_LOC = ../hypre/src/hypre
4. Save the contents of the file linked here Make.defs.nersc as Chombo/lib/mk/Make.defs.local In Make.defs.local, pay attention to the line
   HDF5_DIR = /opt/cray/pe/hdf5-parallel/1.14.3.1/gnu/12.3
   NERSC team updates cray/hdf5-parallel module from time to time, thus, the version 1.14.3.1 might be outdated. Check the folder /opt/cray/pe/hdf5-parallel for the available versions and make changes if needed.
5. To build COGENT along with all necessary Chombo libraries, cd to COGENT/exec and run (j<#> for the number of cores per node, Cori Haswell-32, Cori KNL-68)
   make -j16 MPI=TRUE OPT=TRUE DEBUG=FALSE
6. To clean up COGENT build, cd COGENT/exec and run
   make realclean
7. To clean up Chombo build, cd Chombo/lib and run
   make distclean
8. If everything builds successfully, you should see cogent.Linux.64.CC.ftn.OPT.MPI.ex executable program in COGENT/exec directory. To run the program on 32 cores
   srun -n32 PATH_TO_COGENT/exec/cogent.Linux.64.CC.ftn.OPT.MPI.ex input_file_name
   Sample input files can be found in COGENT/sample_inputs folder.

To build the code on quartz.llnl.gov:

1. Run the following to add the necessary environmental variables
   module load gcc/12.1.1
   module load mvapich2/2.3.7
   
2. Build hypre (see COGENT/hypre/INSTALL for details).
   cd COGENT/hypre/src
   make distclean (if need to clean previous build data)
   ./configure CC=mpicc CXX=mpicxx --enable-maxdim=5
   make install
   OR, for parallel build (since quartz has 36 cores per node) make -j16 install
   
3. After building hypre, make sure that HYPRE_LOC variable in COGENT/exec/GNUmakefile.multidim points to the hypre installation directory.
   If you run hypre configure script without the prefix flag option (as in the instructions above), you should edit manually the file COGENT/exec/GNUmakefile.multidim and set HYPRE_LOC = ../hypre/src/hypre
4. Save the linked here file Make.defs.quartz.new as Make.defs.local and copy it into Chombo/lib/mk directory.
5. To build COGENT along with all necessary Chombo libraries, cd to COGENT/exec and run
   make -j16 MPI=TRUE OPT=TRUE DEBUG=FALSE
6. To clean up COGENT build, cd COGENT/exec and run make realclean
7. To clean up Chombo build, cd Chombo/lib and run make distclean
8. If everything builds successfully, you should see cogent.Linux.64.mpicxx.gfortran.OPT.MPI.ex executable program in COGENT/exec directory. To run the program on 32 cores in debug queue
   srun -n32 -ppdebug PATH_TO_COGENT/exec/cogent.Linux.64.CC.ftn.OPT.MPI.ex input_file_name
   Sample input files can be found in COGENT/sample_inputs folder.
9. To debug use totalview
   totalview srun -a -n16 PATH_TO_COGENT/exec/cogent.Linux.64.mpicxx.gfortran.DEBUG.MPI.ex input_file_name
   

To build the code on singe.llnl.gov:

1. Run the following to add the necessary environmental variables
   module load gcc/9.1.0
   module load hdf5
   module load mpi/mpich
   

   Note: other versions of gcc can be used too, however, make sure that they support mpi.

2. Build hypre (see COGENT/hypre/INSTALL for details).
   cd COGENT/hypre/src
   make distclean (if need to clean previous build data)
   ./configure CC=mpicc CXX=mpicxx --enable-maxdim=5
   make install
   OR, for parallel build (since singe has 32 cores per node) make -j16 install
   
3. After building hypre, make sure that HYPRE_LOC variable in COGENT/exec/GNUmakefile.multidim points to the hypre installation directory.
   If you run hypre configure script without the prefix flag option (as in the instructions above), you should edit manually the file COGENT/exec/GNUmakefile.multidim and set HYPRE_LOC = ../hypre/src/hypre
4. Save the linked here file Make.defs.singe as Make.defs.local and copy it into Chombo/lib/mk directory.
5. To build COGENT along with all necessary Chombo libraries, cd to COGENT/exec and run
   make -j16 MPI=TRUE OPT=TRUE DEBUG=FALSE
6. To clean up COGENT build, cd COGENT/exec and run make realclean
7. To clean up Chombo build, cd Chombo/lib and run make distclean
8. If everything builds successfully, you should see cogent.Linux.64.mpicxx.gfortran.OPT.MPI.ex executable program in COGENT/exec directory. To run the program on 32 cores in debug queue
   srun -n32 -ppdebug PATH_TO_COGENT/exec/cogent.Linux.64.mpicxx.gfortran.OPT.MPI.ex input_file_name
   Sample input files can be found in COGENT/sample_inputs folder.
9. To debug use totalview
   totalview srun -a -n16 PATH_TO_COGENT/exec/cogent.Linux.64.mpicxx.gfortran.DEBUG.MPI.ex input_file_name
   

To build the code on cab.llnl.gov:

1. Run the following to add the necessary environmental variables
   use gcc-4.7.4p
   use mvapich2-gnu-2.1
   use hdf5-gnu-parallel-mvapich2-1.8.10
   
2. Build the hypre library (see COGENT/hypre-2.9.0/INSTALL for details).
   cd to COGENT/hypre-2.9.0b
   run make distclean
   run ./doconfig-cab-mvapich2-gnu-2.1-opt
   run ./doinstall
   
3. Rename Make.defs.cab into Make.defs.local and copy it into Chombo/lib/mk directory.
4. To build COGENT along with all necessary Chombo libraries, cd to COGENT/exec and run
   make -j16 MPI=TRUE OPT=TRUE DEBUG=FALSE
5. To clean up COGENT build, cd COGENT/exec and run make realclean
6. To clean up Chombo build, cd Chombo/lib and run make realclean
7. If everything builds successfully, you should see cogent.Linux.64.mpicxx.gfortran.OPT.MPI.ex executable program in COGENT/exec directory. To run the program on 32 cores in debug queue
   srun -n32 -ppdebug PATH_TO_COGENT/exec/cogent.Linux.64.CC.ftn.OPT.MPI.ex input_file_name
   Sample input files can be found in COGENT/sample_inputs folder.

To build the code on macOS:

Instructions for how to build COGENT on Sonoma macOS with GCC and OpenMPI compilers build from source (does not require sudo privileges) SonomaBuildInstructions (used Make.defs.OSX_Sonoma_gcc for Chombo Make.defs.local)

Instructions for how to build the code on Monterey with GCC MontereyBuildInstructions (used Make.defs.OSX_Monterey_Brew for Chombo Make.defs.local)

Instructions for how to build COGENT on Catalina macOS with GCC and OpenMPI compilers build from source (does not require sudo privileges) CatalinaBuildInstructions (used Make.defs.OSX_Catalina_gcc for Chombo Make.defs.local)

Old instructions on how to build the code on Maverics MavericsBuildInstructions



Succesfull build on Sierra (10.12.6):


Chombo revision 23220; COGENT revision 1315
Fortran: gcc-4.9.2/gfortran
C++: Apple LLVM version 8.1.0 (clang-802.0.41)
MPI: openmpi-1.8.1
HDF5: hdf5-1.8.13
Used Make.defs.sierra




Succesfull build on High Sierra (10.13.3) using MacPorts for dependent libraries:


OS X High Sierra V10.13.3
Xcode Version 9.2
MaxPorts Version: 2.4.2 (installed in /opt/local)


Key software from MacPorts:
mpich-devel-gcc7 @3.3a2_4+fortran+gforker
(This has mpicxx, mpif90, mpiexe)
hdf5 @1.10.2_0+clang50+cxx+hl+mpich_devel+szip
zlib @1.2.11_0


Chombo version (Revision): 23287
Used Make.defs.OSX_HSierra_MacPorts


COGENT version (Revision): 1378
In hypre-2.9.0b, used doconfig-Darwin-gfortran-opt


COGENT build instructions for High Sierra using:


OS X High Sierra V 10.13.6
Xcode Version 10.1
OpenMPI 4.0.0
HDF5-1.10.4
gfortran 8.1.0
zlib 1.2.11, gawk 4.2.1 from MacPorts


Chombo revision 23518, COGENT revision 1507


OpenMPI should be configured and built after gfortran installation, and should be configured with MPI-1 compatibility (--enable-mpi1-compatibility)


HDF5 should be built from gzipped download from HDF5 website, NOT from zipped version (wrong end-of-line markers). It should be built with --enable-fortran and --enable-parallel flags. It is suggested to set a number of environment variables prior to configuring: Set CC, FC, F77, F90, and F9X to the paths to openmpi’s mpicc, mpifort, mpif77, mpif90, and mpifort executables, plus the following (csh/tcsh syntax):
setenv FFLAGS "-I/usr/local/openmpi4/include"
setenv CFLAGS "-I/usr/local/openmpi4/include"
setenv LDFLAGS "-L/usr/local/openmpi4/lib -L/usr/local/lib -L/usr/lib -L/opt/local/lib"
setenv LIBS "-lmpi"
setenv CPPFLAGS "-I/usr/local/openmpi4/include -I/usr/include”


The system path should include macports path /opt/local/bin and /opt/local/sbin as well as paths to the OpenMPI binaries and the gfortran binaries.


To build COGENT:


1. in the file Make.defs.OSX_HSierra_OMPI, which can be found in Chombo/lib/mk/local, modify MCP_DIR, Z_DIR and HDF5_DIR if needed to point to your openmpi, lib and hdf5 directories. Copy as Make.defs.local in Chombo/lib/mk .
   
2. Modify COGENT/hypre-2.9.0b/doconfig-Darwin-gfortran-opt so that mpi_include points to the OpenMPI include directory and so that /usr/bin in the configure line is replaced by the path to the OpenMPI binaries. Configure by running the modified doconfig, then build/install by running deinstall.
   
3. cd to COGENT/exec, then build by running make all -j 4 MPI=TRUE OPT=TRUE DEBUG=FALSE.