Skip to content

Cleanup _dsl fields#1052

Open
jcanton wants to merge 109 commits intomainfrom
cleanup_dsl_fields
Open

Cleanup _dsl fields#1052
jcanton wants to merge 109 commits intomainfrom
cleanup_dsl_fields

Conversation

@jcanton
Copy link
Contributor

@jcanton jcanton commented Feb 10, 2026

WIP for blueline merging.
sync with https://github.com/C2SM/icon-exclaim/pull/412

@jcanton jcanton requested review from havogt and muellch February 10, 2026 16:13
@havogt
Copy link
Contributor

havogt commented Feb 10, 2026

Briefly looking at the code it seems we can get rid of the mask, here

z_gradh_exner_vp = where(ipeidx_dsl, z_gradh_exner + z_hydro_corr * pg_exdist, z_gradh_exner)

jcanton and others added 10 commits February 11, 2026 10:09
* main:
  Muphys bug fix: drivers run_graupel_only and run_full_muphys (#877)
  Deduplicate RBF test tolerances, use common experiment fixture (#1049)
  Fix tests for MASK_HDIFF, ZD_DIFFCOEF_DSL, ZD_INTCOEF_DSL, and ZD_VERTOFFSET_DSL fields with torus grids (#1045)
* cleanup_pg_edgeidx_dsl:
  CI: disable cuda mps (#1054)
  add missing file
  remove some more...
  fix return type annotation
  add description
  cleanup unused program
  Remove pg_edgeidx_dsl
@jcanton
Copy link
Contributor Author

jcanton commented Feb 23, 2026

cscs-ci run default

@jcanton
Copy link
Contributor Author

jcanton commented Feb 23, 2026

cscs-ci run distributed

@jcanton
Copy link
Contributor Author

jcanton commented Feb 24, 2026

cscs-ci run distributed

@jcanton
Copy link
Contributor Author

jcanton commented Feb 24, 2026

cscs-ci run default

from icon4py.model.common.grid import base, horizontal as h_grid, icon, utils as grid_utils
from icon4py.model.common.states import prognostic_state
from icon4py.model.common.utils import data_allocation as data_alloc
from icon4py.tools.py2fgen.wrappers import common as wrapper_common
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a new package dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@havogt should we move list2field and kflip to icon4py.common?

Copy link
Contributor

@muellch muellch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, looks like a lot of fiddly stuff correctly implemented!

def wgtfacq_c_dsl(self):
return self._get_field("wgtfacq_c_dsl", dims.CellDim, dims.KDim)
ar = self.wgtfacq_c().ndarray
k = ar.shape[1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't. But at least someone does.

dims.KDim: self.theta_ref_mc().domain[dims.KDim].unit_range,
}
)
return wrapper_common.list2field(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see!

@jcanton
Copy link
Contributor Author

jcanton commented Feb 25, 2026

cscs-ci run distributed

@jcanton
Copy link
Contributor Author

jcanton commented Feb 25, 2026

cscs-ci run default

@github-actions
Copy link

Mandatory Tests

Please make sure you run these tests via comment before you merge!

  • cscs-ci run default
  • cscs-ci run distributed

Optional Tests

To run benchmarks you can use:

  • cscs-ci run benchmark-bencher

To run tests and benchmarks with the DaCe backend you can use:

  • cscs-ci run dace

To run test levels ignored by the default test suite (mostly simple datatest for static fields computations) you can use:

  • cscs-ci run extra

For more detailed information please look at CI in the EXCLAIM universe.

Comment on lines +202 to +215
def get_array_namespace(array: NDArray) -> ModuleType:
"""
Returns the array namespace for a given array.
"""
if hasattr(array, "__array_namespace__"):
return array.__array_namespace__()
elif isinstance(array, np.ndarray):
return np
elif isinstance(array, xp.ndarray):
return xp
else:
raise RuntimeError(
f"Unsupported array type '{type(array)}'. Cannot detect the array namespace."
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracted into #1085.

jcanton added a commit that referenced this pull request Feb 27, 2026
<img width="2204" height="1171" alt="image"
src="https://github.com/user-attachments/assets/c8ee017d-ae0f-4b7f-926e-cd3453310a14"
/>


3 stencils update theta_v and exner in the halo / lateral boundary
region.
2 of these can be merged (for performance's sake) as they modify the
same cells with the `mask_prog_halo_c` (ex `bdy_halo_c`) mask

based upon #1063
needs C2SM/icon-exclaim#412 and
#1052 for updated serialized data

---------

Co-authored-by: Hannes Vogt <hannes@havogt.de>
Copy link
Contributor

@vectorflux vectorflux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My objective in reviewing this was to understand the *_dsl removals on the Fortran side, and see how the calculations move from the Fortran side to the Python side. I failed. It is not clear to me when and where the _dsl variables are removed and when they stay on the Python side, and what the difference now is. I'm really impressed that you have followed through all the logic in this complicated PR.

In the interest of moving this all forward by the end of February I approve now, but hope that you will explain it all to me at some point.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Straightforward: removed mask.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed mask.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed mask; change to non-_DSL variables.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed mask.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed mask

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the trade-off between mask_hdiff / zd_vertoffset in the old, and zd_cellidx / zd_vertidx / zd_diffcoef in the new. But no doubt you do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the trade-off between mask_hdiff / zd_vertoffset in the old, and zd_cellidx / zd_vertidx / zd_diffcoef in the new. But no doubt you do.

Comment on lines +397 to +398
horizontal_pressure_gradient = (
horizontal_pressure_gradient + hydrostatic_correction * pg_exdist
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the new pg_exdist is simply to avoid the where( ipeidx_dsl, ...) ?

),
},
fields={"pg_edgeidx_dsl": attrs.PG_EDGEIDX_DSL, "pg_exdist_dsl": attrs.PG_EDGEDIST_DSL},
fields={"pg_exdist_dsl": attrs.PG_EXDIST_DSL},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once again, when is it PG_EXDIST and when PG_EXDIST_DSL ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Struggling to understand when it is *_ref, *_dsl, *_full and *_ndarray.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants