Many of Nalu-Wind reg tests are not actively being run

[mbkuhn]

The dashboard (and general usage of Nalu-Wind) uses hypre solvers instead of trilinos solvers. However, the majority of the Nalu-Wind reg tests still employ trilinos solvers, and these are not being run during the nightly tests.

Let's make an effort to convert these tests to using hypre and thus include them. After converting the trilinos-based input arguments to use hypre, these cases should be checked by either comparing the norms and ensuring that the diffs are reasonably small or by looking at the actual flow physics to ensure things are running properly. Something to be aware of: not all source terms are hypre-compatible, so some coding may be needed to add updated kernels, like in this merged PR: #1274

Below are all the reg tests with tpetra included (grep -linr -m 1 "tpetra" reg_tests/test_files/*, then the ones with "Trilinos" in the test name removed):

• ActLineSimpleFLLC/ActLineSimpleFLLC.yaml
• ActLineSimpleNGP/ActLineSimpleNGP.yaml
• BLTFlatPlateT3A/BLTFlatPlateT3A.yaml
• BoussinesqNonIso/BoussinesqNonIso_R0.yaml
• BoussinesqNonIso/BoussinesqNonIso_R1.yaml
• BoussinesqNonIso/BoussinesqNonIso_R2.yaml
• IDDESPeriodicHillEdge/IDDESPeriodicHillEdge.yaml
• KEChannelEdge/KEChannelEdge.yaml
• KOChannelEdge/KOChannelEdge.yaml
• MeshMotionInterior/MeshMotionInterior.yaml
• SSTAMSChannelEdge/SSTAMSChannelEdge_rst.yaml
• SSTAMSChannelEdge/SSTAMSChannelEdge.yaml
• SSTAMSOversetRotCylinder/SSTAMSOversetRotCylinder_rst.yaml
• SSTAMSOversetRotCylinder/SSTAMSOversetRotCylinder.yaml
• SSTChannelEdge/SSTChannelEdge.yaml
• SSTLRChannelEdge/SSTLRChannelEdge.yaml
• SSTPeriodicHillEdge/SSTPeriodicHillEdge.yaml
• SSTWallHumpEdge/SSTWallHumpEdge.yaml
• VOFDroplet/VOFDroplet.yaml
• VOFInertialDroplet/VOFInertialDroplet.yaml
• VOFZalDisk/VOFZalDisk.yaml
• ablHill3dSymPenalty/ablHill3dSymPenalty.yaml
• ablHill3d_ii/ablHill3d_ii.yaml
• ablHill3d_ip/ablHill3d_ip.yaml
• ablHill3d_pp/ablHill3d_pp.yaml
• ablNeutralEdge/ablNeutralEdge.template.yaml
• ablNeutralEdge/ablNeutralEdge.yaml
• ablNeutralEdgeAMS/ablNeutralEdgeAMS.yaml
• ablNeutralEdgeNoSlip/ablNeutralEdgeNoSlip.yaml
• ablNeutralEdgeSST/ablNeutralEdgeSST.yaml
• ablNeutralEdgeSegregated/ablNeutralEdgeSegregated.yaml
• ablNeutralStat/ablNeutralStat.yaml
• ablStableEdge/ablStableEdge.yaml
• ablUnstableEdge/ablUnstableEdge.yaml
• ablUnstableEdge/ablUnstableEdge_rst.yaml
• ablUnstableEdge_ra/ablUnstableEdge_ra.yaml
• airfoilSSTSUST/airfoilSSTSUST.yaml
• amsChannelEdge/amsChannelEdge_rst.yaml
• amsChannelEdge/amsChannelEdge.yaml
• aslNeutralEdgeSST/aslNeutralEdgeSST.yaml
• conduction_p4/conduction_p4.yaml
• convTaylorVortex/convTaylorVortex.yaml
• dgNonConformalEdgeCylinder/dgNonConformalEdgeCylinder.yaml
• dgNonConformalFluidsEdge/dgNonConformalFluidsEdge.yaml
• drivenCavity_p1/drivenCavity_p1.yaml
• edgeHybridFluids/edgeHybridFluids.yaml
• ekmanSpiral/ekmanSpiral.yaml
• heatedWaterChannelEdge/heatedWaterChannelEdge_rst.yaml
• heatedWaterChannelEdge/heatedWaterChannelEdge.yaml
• karmanVortex/karmanVortex.yaml
• nonIsoEdgeOpenJet/nonIsoEdgeOpenJet.yaml
• oversetHybrid/oversetHybrid.yaml
• oversetMovingCylinder/oversetMovingCylinder.yaml
• oversetMovingCylinder/oversetMovingCylinder_rst.yaml
• oversetRotCylMultiRealm/oversetRotCylMultiRealm.yaml
• oversetRotCylinder/oversetRotCylinder.yaml
• oversetRotCylinderHypre/oversetRotCylinderHypre.yaml
• oversetSphereTIOGA/oversetSphereTIOGA.yaml
• oversetTransformedTwoRotCylinder/oversetTransformedTwoRotCylinder.yaml
• periodic3dEdge/periodic3dEdge.yaml
• vortexOpen/vortexOpen.yaml

[BumseokLee]

Hi

I found that two test cases below employ the strong symmetry boundary condition.

• ablNeutralEdgeAMS/ablNeutralEdgeAMS.yaml
• BLTFlatPlateT3A/BLTFlatPlateT3A.yaml

However, hypre doesn't currently support the strong symmetry boundary conditions.

Probably, I will open a new issue for this.

[mbkuhn]

I will take these:

• VOFDroplet/VOFDroplet.yaml
• VOFInertialDroplet/VOFInertialDroplet.yaml
• VOFZalDisk/VOFZalDisk.yaml

[mbkuhn]

@itopcuoglu, could you take these on?

• oversetMovingCylinder/oversetMovingCylinder.yaml
• oversetMovingCylinder/oversetMovingCylinder_rst.yaml
• oversetTransformedTwoRotCylinder/oversetTransformedTwoRotCylinder.yaml

I'm sticking to ones that say "decoupled_overset_solve: yes" because I'm not sure if the tightly coupled overset pressure solves are compatible with hypre.

[mbkuhn]

@sseraj, could you try converting this case to use hypre?

• taylorGreenVortex_p3/taylorGreenVortex_p3.yaml

[mbkuhn]

I'm going to remove from the list all of the ones with "Trilinos" in the name because I think those are intended for testing trilinos solvers.

[BumseokLee]

@mbkuhn
I think we can delete the BLTFlatPlateT3A case. This case was created for the transition model two years ago. However, the transition model wasn't merged at that time, and for this reason, Jon turned off the case in CTestList.cmake. It has never been used since then. Currently, I am creating a new reg. test for the transition model using an existing airfoil mesh in the repository.

[mbkuhn]

@ndevelder @BumseokLee @sbidadi9

We're thinking of focusing on a subset of these tests to switch them over to hypre, ideally choosing the ones that are most relevant to how nalu-wind is most frequently used currently. If you were picking a handful from this list (in issue description), which would you categorize as including important aspects of nalu-wind for your simulations?

[sseraj]

I tried converting the taylorGreenVortex_p3 test to use hypre but ran into the following error when trying to run the test:

libc++abi: terminating due to uncaught exception of type std::runtime_error: Nalu must be compiled with matrix-free support for promotion

It doesn't look like there is an option in exawind-manager to enable matrix-free support.

Is this something that's worth updating or should we just remove tests for the high-order code? Maybe @rcknaus has opinions on this?

[marchdf]

AFAIK, that's a trilinos solvers only path. You would need ENABLE_MATRIXFREE and ENABLE_TRILINOS_SOLVERS for a successful build. I think we should keep the test for now.

[mbkuhn]

@sseraj thanks for looking into this. I didn't think about the "p3" aspect of this test when I asked for it to be converted. The element-based scheme is rarely used now, but it's probably good to keep the test as-is while the scheme is still present. I'll de-prioritize this test and remove it from the list.

[sseraj]

@mbkuhn Sounds good, feel free to assign me to other tests