EOS models, unit conversion, particle fixups

[brbass]

Summary

Features

Singularity EOS

• Includes GammaLawGas, ExtendedVinet, MieGruneisen, Spiner, and EOSPAC.
• Tested against the ordinary GammaLawGas in spheral.
• Will improve a lot once spack is updated, as the current singularity version is a few years out of date.

Null EOS

• NullEquationOfState is the counterpart of NullStrength.

Unit conversion

• Converts between two instances of PhysicalConstants. Very helpful for things like Singularity above.
• Support for subdirectories in silo files. Currently used for RK corrections.

Improvements for the smoothing parameter

• Adds a node separation and relaxation method that slots into the initial iterateIdealH calculations. To use it, create relaxation = RelaxNodes(...) and then pass it to the controller: SpheralController(initialHExtraPackages = [relaxation]). This separates nodes that are too close and relaxes nodes using the Voronoi, without which the smoothing parameter may not converge for problems with problematic initial node distributions.
• Adds a check for the initial H distribution to limit the number of initial points to some reasonable value. If a problem passes in nodes that have hundreds of thousands of neighbors or whatever, this sharply limits the number before the iterateIdealH calculations get underway.
• Adds a damping factor for the IdealH calculation that helps it converge when struggling.

Verbose time step control

• Adds a verboseStep parameter to the Integrator. Printing the verbose time step info every cycle makes things difficult to read. With the verbose step set to 100, it is reasonable to keep it on all the time.

Bugfixes

• Removed the old, non-functional volume calculation in SPH and requested Voronoi cells instead.
• Removed circular dependencies for Johnson-Cook to let it run. Johnson-Cook Damage Model dependency #303. This will lag the strain calculation.
• Added allReduceLoc to prevent the time step from being printed once per rank when they all calculate the same time step.
• Added a gradientPairs method that is more robust for points with difficult H distributions (yes, more smoothing parameter issues). Also cleaned up the gradient instantiation.
• Added some missing includes.

ToDo :

• Annotate RELEASE_NOTES.md with notable changes.
• Create LLNLSpheral PR pointing at this branch. (PR#)
• LLNLSpheral PR has passed all tests.

[ldowen]

Help me understand what is going on here. It seems like singularity has a ton of dependencies but I don't understand why we have the Spheral Spack package depending on Singularity's dependencies and why we are bringing them into our CMake. Neither of those things should be happening. It is possible to use a newer Spack recipe for Singularity than what is in the current version of Spack we are using, if that will help you. You might have to tweak it a little.

[brbass]

Help me understand what is going on here. It seems like singularity has a ton of dependencies but I don't understand why we have the Spheral Spack package depending on Singularity's dependencies and why we are bringing them into our CMake. Neither of those things should be happening. It is possible to use a newer Spack recipe for Singularity than what is in the current version of Spack we are using, if that will help you. You might have to tweak it a little.

It is onerous to have to include all of these, yes, but I couldn't get it to work otherwise. The dependencies are explicitly listed because the -fPIC flags don't propagate and the code didn't compile for me without them. Likewise, the code would not compile for me without adding the variables to the cmake cache, as the individual packages would not find each other without these.

Do you have any suggestions on how to make this work in a nicer way? I agree that it would be great to get this down to a single singularity-eos statement!

It would also be great to be on a newer version of singularity-eos, but after fighting with spack for a few days, I gave up and figured that it would be easier once Spheral is updated to Spack v1.0.0 or v1.1.0, which supports the latest Singularity version v1.10.0.

[ldowen]

Did you try adding the +kokkos +kokkos-kernels variants to singularity-eos? Is there any particular reason why you need use kokkos? We don't really support GPUs in Spheral yet anyway. I don't want to add another 6 dependencies unless we really need all of them. So if you don't need the Spiner or EOSpac, you should get rid of those too. Similarly, add ~python if you don't need that functionality

You can get a newer version of a TPL than is in the builtin by just adding a package file in our scripts/spack/packages directory. Look at what we we do with Umpire to get an idea of how to do it. You would also add the cflags and cppflags in that package file.

[brbass]

Did you try adding the +kokkos +kokkos-kernels variants to singularity-eos? Is there any particular reason why you need use kokkos? We don't really support GPUs in Spheral yet anyway. I don't want to add another 6 dependencies unless we really need all of them. So if you don't need the Spiner or EOSpac, you should get rid of those too. Similarly, add ~python if you don't need that functionality

You can get a newer version of a TPL than is in the builtin by just adding a package file in our scripts/spack/packages directory. Look at what we we do with Umpire to get an idea of how to do it. You would also add the cflags and cppflags in that package file.

kokkos is one of the required libraries for singularity. It uses it even off of the GPU. The Spiner EOS is the main one of interest here, since it gives another option for tabulated data. I can try pruning the dependencies and using ~python to see if that works, but I recall it being pretty difficult to get to work in the first place, so I'm not holding my breath.

[ldowen]

Based on the singular-eos Spack package recipe, Kokkos is an optional variant. Unless this is a mistake, it implies that Singularity can work without Kokkos. By not including +kokkos +kokkos-kernel on your depends_on('singularity-eos the version of singularity you are building will have Kokkos turned off. Again, unless there is a mistake in the package recipe in Spack.

[brbass]

Based on the singular-eos Spack package recipe, Kokkos is an optional variant. Unless this is a mistake, it implies that Singularity can work without Kokkos. By not including +kokkos +kokkos-kernel on your depends_on('singularity-eos the version of singularity you are building will have Kokkos turned off. Again, unless there is a mistake in the package recipe in Spack.

From what I remember, kokkos is an optional variant, but is required for several of the EOS options that we want. I will try to build without, but it may not work.

[ldowen]

Ok. If you determine you need it, you have to add the +kokkos +kokkos-kernel to variant to your singularity. That might also help with your building issues with CMake.

[brbass]

I have tried all sorts of configurations to get Singularity to build with fewer changes to the build files, but I am not the expert that @ldowen is with spack. Anytime I remove one of the dependencies, the -fPIC flag goes away and it fails to link correctly. It is really a miracle that I got it working the first time.

I would rather not put too much work into it for now since the Singularity spack package changes a lot with the version compatible with the spack v1.0.0 release. Once that comes out, it might be worth revisiting and seeing if some of the pain points have been resolved.