Add 3d thermal forcing to OCN-GLC coupling #121

matthewhoffman · 2025-02-04T22:45:00Z

This pull request extends the existing thermal forcing functionality from a single depth to a predefined set of z-levels. The coupler implementation follows that of multiple elevation classes in the LND-GLC coupling. A new coupler variable is added, GLC_NZOC. It has a default value of 0, meaning TF coupling is disabled. The standard value for when the coupling is activated is 30, which follows the TF z-levels used by ISMIP6. Other values are supported, and an additional example with 4 levels is included. A new coupler module, glc_zocnclass_mod.F90, is added which is follows very closely from glc_elevclass_mod.F90. It contains subroutines and functions to initialize the ocean z-level classes and to access the z-level values. It also has subroutines to provide the z-level indices as strings, which the coupling field initialization code and the component import/export routines uses to create GLC_NZOC fields, one for each layer.

The intention is that this 3d TF coupling will be used for facemelting (melting at vertical glacier margins terminating in the ocean), and so is used for a process that we do not anticipate resolving explicitly in MPAS-Ocean. As such, this coupling should be included anytime both OCN and GLC are active, and logic has been added so that this is the case based on compset definition. There are versions of the GLC_NZOC variable in both the MPAS-Ocean and MALI Registries so that it can be set by namelist settings controlled by CIME/build operations.

The MPAS-Ocean mpas_ocn_time_average_coupled.F module has been modified to loop over the z-levels and calculate TF at each one. Then, the time-averaged 3d TF values are passed to the coupler, where they are remapped to the GLC mesh and passed to MALI. MALI assigns the values to the existing 3d TF field, and the uses its bathymetry-aware extrapolation routine to extrapolate the TF values from the edge of the MPAS-Ocean domain to wherever the ice-sheet margin is on the MALI mesh. From there, the TF is used to force the facemelting parameterization.

The facemelting flux in MALI is exported to the previously unused Fogg_rofl flux, which already gets combined in the coupler with river runoff to be imported in MPAS-Ocean in the Foxx_rofl flux. Long term, we may want to separate these runoff fluxes so that the horizontal and vertical distribution of facemelting can be handled differently than river runoff, but this provides a reasonable first approximation.

Note that this 3d TF can also be used to force the ISMIP6 ice-shelf basal melt parameterization in MALI, which provides a less sophisticated alternative to using ice-shelf basal melt rates calculated in the coupler (or MPAS-Ocean). This simpler approach will be used for the few, small ice shelves in Greenland, which are not included in the MPAS-Ocean meshes (and would not be resolved even if they were). This approach could also be used for Antarctica as an alternative that avoids grounding line migration issues in MPAS-Ocean by taking advantage of the TF extrapolation in MALI.

This PR is now functional, but some additional work is needed before this is complete and ready for review at the main E3SM repo:

new OCN2GLC_TF_SMAPNAME mapping files that use bilin method but exclude extrapolation need to be created for all the grids that currently support that mapping
a decision needs to be made whether to continue to support the previous 2d TF coupling in addition to this new 3d version. I don't see a use-case where the 2d version would ever be used over the 3d version, so I advocate for deprecating the 2d version, but discussion should happen before I proceed to do that.
a mask field should be added to the coupling so that MALI knows what locations on its mesh were valid ocean locations in MPAS-Ocean. Right now this is handled in a hacky (but functional) way.
ocean developers may want to modify the way 3d TF is being calculated in MPAS-Ocean
there are some temporary commits to namelist and streams settings to ease testing that should be removed or adjusted.
the existing 2d TF test should be replaced by a new test
add a package for the 3d TF variable in MPAS-Ocean

matthewhoffman · 2025-02-10T04:33:40Z

Testing with:

./create_test --wait --walltime 0:30:00  SMS_D_Ld5.TL319_oQU240wLI_ais8to30.MPAS_FOLISIO_JRA1p5.chrysalis_gnu

Note this is smoketest with debug on for testing purposes. The ais8to30 mesh is chosen as a low-res testing mesh over the gis20 mesh because the gis20 mesh has very little ocean mesh overlap and very few marine-terminating ice-sheet margins. The ais8to30 mesh also allows evaluation of how the TF coupling works for ice-shelf basal melting.

Results look like things are functioning correctly:

TF on the MALI mesh at two different depths:
Surface:

Deeper:

TF cross-section:

Ice-shelf basal melt rate (m/yr) on the MALI mesh calculated from ISMIP6 parameterization inside MALI:
(calculated as floatingBasalMassBalApplied/-910*3.15e7)

matthewhoffman · 2025-02-10T05:02:55Z

@cbegeman , @xylar , I marked you both as reviewers. I think a review from one of you is sufficient, but I wanted this on both of your radars.

Other people who may want to look at this or weigh in on details (but I don't think a review is required): @irenavankova, @darincomeau , @alexolinhager, @trhille

matthewhoffman · 2025-02-10T05:04:31Z

@jonbob , see note above about new OCN2GLC_TF_SMAPNAME mapping files. Let me know if you want me to create those or if you prefer to do so.

matthewhoffman · 2025-02-12T04:37:24Z

GIS Testing

./create_test --wait --walltime 0:30:00  SMS_D_Ld5.TL319_IcoswISC30E3r5_gis1to10kmR2.MPAS_FOLISIO_JRA1p5.chrysalis_intel

Note: need to modify env_mach_pes in the test directory and set all -4 values to -8 in the <entry id="NTASKS"> section to get this case to run. (Followed by ./case.setup --reset; ./case.build; ./case.submit )

Results look wonky, presumably because there is insufficient overlap between the OCN and GLC meshes. The gis1to10kmR2 MALI mesh has a very narrow gutter.

The 20km GIS mesh is also not useful for testing because the coarse resolution fails to resolve most marine outlet glaciers. Proper GIS testing will require a medium to high res mesh with a larger gutter (like the 4km mesh currently being tested).

matthewhoffman · 2025-02-12T20:02:53Z

After talking with @darincomeau , it became clear that there already is an unused coupler variable for ice-sheet runoff flux to the ocean, Fogg_rofl. This gets combined with river runoff to be received by MPAS-Ocean in the Foxx_rofl variable, which is assigned to the riverRunoff variable. While in the long term, we might want to handle river and facemelt runoff fluxes differently in terms of horizontal smoothing and vertical profile, it was easy to attach facemelting to this existing variable and get the FWF passed to MPAS-Ocean. I've added a commit that does that. It includes logic to also include the ice-shelf basal melt flux in Fogg_rofl if MALI is calculating that flux itself using 3d TF in the ISMIP6 parameterization (as opposed to ISMF being calculate by the coupler or MPAS-Ocean).

Here is a figure showing that the facemeling flux in MALI is being received by MPAS-Ocean properly. You have to zoom far in to see the nonzero facemelt flux values on the 8km MALI mesh, so they are highlighted with red circles.

This run dies in sea ice, so more work needs to go into confirming the flux magnitude is reasonable (for reference, the value at the mouth of the Amazon is 2e-4) and if some sort of smoothing needs to be added (which might requires separating this flux from Foxx_rofl).

This commit adds the ability to support a variable number of ocean z-levels for coupling ocean state fields between OCN and GLC. The intended use case is passing profiles of thermal forcing (and possibly eventually temperature and salinity) from OCN to GLC, for MALI to use for driving parameterizations of marine melting that is unresolved in MPAS-Ocean (facemelting or ice shelves that are unresolved on the ocean mesh). The implementation closely mirrors that already existing for GLC multiple elevation classes used for coupling surface mass balance with the LND component. This commit introduces the changes to the coupler and configuration files needed to support multiple z-ocean classes, but no changes are made to introduce them into MPAS-Ocean or MALI.

…C_NZOC

The number of GLC z-levels defaults to 0 (meaning no z-level coupling), but it set to 30 for any compset where both MPAS-Ocean and MALI are active.

The commit that added glc_zocnclass_mod was a near exact copy of glc_elevclass_mod. This commit makes a number of changes for the functionality needed for z-levels: * Make initialization based on the center of a level * Add an array and an subroutine for initializing the bounds of each level based on assumed relationships to the center values * removal of a subroutine that is unlikely to be ever used * other minor adjustments

This routine initializes the z-levels in MALI using the new mct module

These fields are related to the new 3d thermal forcing and are needed to validate it is working properly. They will also be useful for model analysis.

This commit adds Registry options and MPAS-Ocean code to support a time-averaged 3d thermal forcing field. This follows the implementation of a 2d thermal forcing field at a single depth that was added recently. For now, both 2d and 3d TF functionalities are supported. The time-averaged 3d TF values are calculated in a new variable, avgThermalForcingAtZLevels. The z-levels at which to calculate TF are stored in a variable, glcZLevels. The size of glcZLevels is defined by a new dimension, nGlcZLevels, which is set by a new namelist option, config_n_glc_z_levels. The existing namelist option config_glc_thermal_forcing_coupling_mode is modified to support a ‘3d’ option. mpas_ocn_time_average_coupled.F is modified to calculate avgThermalForcingAtZLevels if the 3d calculation is enabled. Note that in this commit, there is no functionality to initialize glcZLevels. This will happen in a subsequent commit through the coupler. Also note that a package should probably be added for the new fields.

This commit makes it so MPAS-Ocean gets the value of glc_nzoc from the value the E3SM bld system assigns it (which comes from the compset definition). This commit modifies the files in the MPAS-Ocean namelist build system, but does not yet alter the way MPAS-Ocean interacts with coupler at runtime.

Have cime_comp determine it based on what components are active rather than basing it off of MPAS-Ocean namelist options

For the 2d thermal forcing, the OCN2GLC_TF_SMAPNAME map files needed to mask out ocean cells shallower than the coupling depth. For the new 3d thermal forcing coupling, standard mapping is appropriate.

This is needed to configure MPAS-Ocean to work with the 3d TF coupling.

Using ocn_c2_glctf to control TF didn't work, so switching to namelist options in each component.

For the extrapolation in MALI to work properly, extrapolation in the OCN2GLC_TF_SMAPNAME mapping file needs to be disabled. As an initial test, this commit switched from the bilin (which includes extrap) to the aave (which does not) mapping file. The correct behavior in MALI now occurs. A later commit will need to add bilin mapping files without extrapolation for the OCN2GLC_TF_SMAPNAME mapping file for each mesh that includes one.

These liquid runoff fluxes are currently added to the river runoff flux in the coupler before being passed to the ocean. If we want to change the horizontal or vertical distribution of how these are handled in the ocean, we'll need to separate them in the future. Also needed is time averaging of fluxes, which has not yet been implemented in MALI.

This way it will also extract latent heat from the ocean. Note that doing it this way, it will be combined with the calving flux. Long term, we will likely want to handle them separately so they can use different horizontal and vertical distributions. But at present MALI's calving flux is not even hooked up, so that will need to be addressed at a later date.

This updates GLC2OCN_LIQ_RMAPNAME and GLC2OCN_ICE_RMAPNAME mapping files to use nearest neighbor with runoff area rescaling. The old mapping files did not have the area-scaling applied. This needs to be updated for other meshes using these mappers once the updated mapping files are generated.

matthewhoffman · 2025-02-19T06:47:59Z

@cbegeman and @xylar , I tried changing the melt flux from MALI to MPAS-Ocean to occur through rofi instead of rofl. I ran into a problem in that ice-shelf basal melt flux from the Jourdain parameterization can sometimes be freeze-on. When that happens I hit this error in the MPAS-Ocean driver:
https://github.com/E3SM-Project/E3SM/blob/master/components/mpas-ocean/driver/ocn_comp_mct.F#L2363-L2365

           if(iceRunoffFlux(n) < 0.0_RKIND) then
               call shr_sys_abort ('Error: incoming rofi_F is negative')
           end if

For the intermediate step of this PR, do you think it is better to switch back to rofl and ignore the heat flux for now, or disable that check in the driver (I'm not sure how dangerous that would be)?

Also, are you sure that rofi / iceRunoffFlux has its corresponding heat flux automatically accounted for? I'm not seeing where that happens.

This commit adds a 3d mask field to MPAS-Ocean for tracking what fraction of time steps in the time average are valid. If the fraction exceeds 0.9, then the driver considers that location valid and passes the value to the coupler along with a coupler mask value of 1. This new coupler mask field matches the dimensionality of the 3d TF field and is passed to MALI. MALI evaluates the mask in its import routine and only uses TF from locations that have at least 50% overlap with MPAS-Ocean cells. In those locations, it uses the TF passed through the coupler, scaled by the area fraction. In location that don't satisfy that criterion, the MALI extrapolation mask is set to 0 and an invalid TF value is inserted in the 3d TF field.

cbegeman · 2025-02-19T15:04:06Z

Also, are you sure that rofi / iceRunoffFlux has its corresponding heat flux automatically accounted for? I'm not seeing where that happens.

Yes, see here:
https://github.com/E3SM-Project/E3SM/blob/b03e6d10557bfe21ac7cff8fd9986e37f7f284fd/components/mpas-ocean/src/shared/mpas_ocn_surface_bulk_forcing.F#L678-L683

cbegeman · 2025-02-19T15:08:17Z

For the intermediate step of this PR, do you think it is better to switch back to rofl and ignore the heat flux for now, or disable that check in the driver (I'm not sure how dangerous that would be)?

I'm fine with switching back to rofl for now, but I would want to come up with a better solution before we devote significant simulation time to this intermediate step. I don't think we would have any problem with disabling the check, it's more of a check for prescribed forcing fields, as I don't think there's a way to get negative ice runoff prognostically.

xylar · 2025-02-19T15:22:18Z

@matthewhoffman, I would feel best about using rofi and disabling the check. Doug Jacobsen put that in 10 years ago without explanation and I don't think it is necessary. It's okay to freeze and thereby warm up the ocean a little -- this seems less dangerous than the current problem we have where positive ice runoff can supercool the ocean, generating frazil.

xylar

@matthewhoffman, I have only partially looked through the code but wanted to flag a significant problem before I continue.

You are using a z-level approach to find the depth that each value in glcZLevels corresponds to. This is not really a safe assumption anywhere in MPAS-Ocean but it is particularly poor for the Antarctic application because the vertical coordinate is very far from z-level inside of and within about a 20-cell ring around ice-shelf cavities. Instead, we need to find zMid that is closest to a given entry in glcZLevels.

xylar · 2025-02-20T12:30:39Z

components/mpas-albany-landice/bld/build-namelist

@@ -55,6 +55,7 @@ OPTIONS
     -ninst_glc            NINST_GLC for this case
     -mali_dynamic         turns on dynamic ice sheet mode
                           Options are: FALSE, TRUE
+     -glc_nzoc             number of z-ocean classes [0 | 4 | 30]


Just wanting to check that these are the only 3 supported values. It sounded in our discussion like the number was flexible.

I see, these are hard-coded in glc_zocnclass_init_default()

xylar · 2025-02-20T12:38:21Z

components/mpas-ocean/bld/build-namelist

@@ -72,6 +72,7 @@ OPTIONS
     -ninst_ocn            NINST_OCN for this case
     -ocn_tidal_mixing     variable for defining if to run with parameterized tidal mixing
                           Options are: false, true. Default is false
+     -glc_nzoc             number of z-ocean classes for indirect glc-ocn coupling [0 | 4 | 30]


Same question as above: is this restricted to only these 3 values?

xylar · 2025-02-20T12:40:26Z

components/mpas-ocean/cime_config/buildnml

-            lines.append('        output_interval="none">')
+            lines.append('        output_interval="00-00-01_00:00:00">')


I don't think we want to make this change.

xylar · 2025-02-20T12:49:35Z