Skip to content

OmegaConf Test Bug Fix#765

Merged
NickGeneva merged 3 commits intoNVIDIA:0.13.0-rcfrom
NickGeneva:ngeneva/rc_omegaconf_fixes
Mar 19, 2026
Merged

OmegaConf Test Bug Fix#765
NickGeneva merged 3 commits intoNVIDIA:0.13.0-rcfrom
NickGeneva:ngeneva/rc_omegaconf_fixes

Conversation

@NickGeneva
Copy link
Collaborator

@NickGeneva NickGeneva commented Mar 19, 2026

Earth2Studio Pull Request

Description

Theres this rare issue with omegaconf when having a full install.

_____________ ERROR at setup of test_stormcast_sda_package[cuda:0] _____________
    @pytest.fixture(scope="function")
    def sda_model() -> StormCastSDA:
        package = StormCastSDA.load_default_package()
>       return StormCastSDA.load_model(package)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
test/models/da/test_da_sda_stormcast.py:492: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.12/site-packages/earth2studio/utils/imports.py:177: in _wrapper
    return obj(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/earth2studio/models/da/sda_stormcast.py:396: in load_model
    config = OmegaConf.load(package.resolve("model.yaml"))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/omegaconf/omegaconf.py:205: in load
    ret = OmegaConf.create(obj)
          ^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/omegaconf/omegaconf.py:178: in create
    return OmegaConf._create_impl(
.venv/lib/python3.12/site-packages/omegaconf/omegaconf.py:900: in _create_impl
    format_and_raise(node=None, key=None, value=None, msg=str(e), cause=e)
.venv/lib/python3.12/site-packages/omegaconf/_utils.py:819: in format_and_raise
    _raise(ex, cause)
.venv/lib/python3.12/site-packages/omegaconf/_utils.py:797: in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/omegaconf/omegaconf.py:861: in _create_impl
    return DictConfig(
.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py:111: in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
.venv/lib/python3.12/site-packages/omegaconf/_utils.py:899: in format_and_raise
    _raise(ex, cause)
.venv/lib/python3.12/site-packages/omegaconf/_utils.py:797: in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py:109: in __init__
    self._set_value(content, flags=flags)
.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py:647: in _set_value
    raise e
.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py:644: in _set_value
    self._set_value_impl(value, flags)
.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py:658: in _set_value_impl
    self._validate_set(key=None, value=value)
.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py:171: in _validate_set
    vk = get_value_kind(value)
         ^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/omegaconf/_utils.py:558: in get_value_kind
    if _is_missing_value(value):
       ^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
value = {'data': {'conditioning_channels': ['u10m', 'v10m', 't2m', 'tcwv', 'sp', 'msl', ...], 'conditioning_dt': 1, 'invariant...2, 2], 'channel_mult_noise': 1, ...}}, 'sampler_args': {'S_churn': 0.0, 'S_max': inf, 'S_min': 0.0, 'S_noise': 1, ...}}
    def _is_missing_value(value: Any) -> bool:
        from omegaconf import Node
    
>       if isinstance(value, Node):
           ^^^^^^^^^^^^^^^^^^^^^^^
E       omegaconf.errors.ConfigTypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
E           full_key: 
E           object_type=None
.venv/lib/python3.12/site-packages/omegaconf/_utils.py:511: ConfigTypeError
---------------------------- Captured stderr setup -----------------------------
Downloading config.json: 100%|██████████| 40.0/40.0 [00:00<00:00, 175B/s]
Downloading model.yaml: 100%|██████████| 2.64k/2.64k [00:00<00:00, 14.0kB/s]
_________________ ERROR at setup of test_aifs_package[cuda:0] __________________
    @pytest.fixture(scope="function")
    def model() -> AIFS:
        # Test only on cuda device
        package = AIFS.load_default_package()
>       p = AIFS.load_model(package)
            ^^^^^^^^^^^^^^^^^^^^^^^^
test/models/px/test_aifs.py:815: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.12/site-packages/earth2studio/utils/imports.py:177: in _wrapper
    return obj(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/earth2studio/models/px/aifs.py:365: in load_model
    model = torch.load(
.venv/lib/python3.12/site-packages/torch/serialization.py:1530: in load
    return _load(
.venv/lib/python3.12/site-packages/torch/serialization.py:2122: in _load
    result = unpickler.load()
             ^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <[KeyError('_content') raised in repr()] ListConfig object at 0x7c0b101c4050>
d = {'_content': ['tp', 'cp', 'sf', 'ro', 'tcw', 'ssrd', ...], '_metadata': ContainerMetadata(ref_type=typing.Any, object_...'>, element_type=typing.Any), '_parent': <[KeyError('_content') raised in repr()] DictConfig object at 0x7c0b101c6ed0>}
    def __setstate__(self, d: Dict[str, Any]) -> None:
        from omegaconf import DictConfig
        from omegaconf._utils import is_generic_dict, is_generic_list
    
>       if isinstance(self, DictConfig):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
.venv/lib/python3.12/site-packages/omegaconf/basecontainer.py:148: TypeError

The issue was because some mocks in the serve test suite which were marked as dangerous were infact... dangerous resulting in breaking hydra for the entire package.

This fixes the test to no mock the import...

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.
  • The CHANGELOG.md is up to date with these changes.
  • An issue is linked to this pull request.
  • Assess and address Greptile feedback (AI code review bot for guidance; use discretion, addressing all feedback is not required).

Dependencies

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 19, 2026

Greptile Summary

This PR fixes a subtle but impactful test-isolation bug: the serve test suite was directly injecting MagicMock objects into sys.modules for hydra and omegaconf, and those mocks bled into other test modules in the same process. When other tests (e.g. test_da_sda_stormcast, test_aifs) tried to use the real omegaconf — e.g. calling OmegaConf.load() or unpickling ListConfig — they encountered the mock instead of the real class, causing isinstance() to fail with ConfigTypeError or TypeError.

The fix replaces the sys.modules patching approach with the idiomatic pattern:

  • A try/except ImportError block sets a _SERVE_AVAILABLE flag
  • pytestmark = pytest.mark.skipif(not _SERVE_AVAILABLE, ...) skips all tests in the file when the optional dependencies are absent
  • The module-level import of earth2studio.serve.server.config is safe to leave unconditional because config.py itself already handles missing hydra/omegaconf with its own try/except guard

No issues were found with the new approach.

Confidence Score: 5/5

  • This PR is safe to merge — it is a targeted, well-understood test isolation fix with no changes to production code.
  • Only one test file is changed. The fix removes a self-described "dangerous" sys.modules mock and replaces it with the standard pytest skip pattern. The underlying config module already handles missing dependencies gracefully, so the new approach works correctly both when hydra/omegaconf are installed and when they are not. No logic changes to the library itself.
  • No files require special attention.

Important Files Changed

Filename Overview
test/serve/server/test_server_config.py Removes dangerous sys.modules mocking of hydra/omegaconf (which leaked across the whole test suite) and replaces it with a proper try/except availability check + pytest.mark.skipif guard. The fix is clean — earth2studio/serve/server/config.py already guards its own hydra/omegaconf imports with try/except, so the module-level import in the test file succeeds even when the dependencies are absent.

Last reviewed commit: "Real fix"

@NickGeneva NickGeneva changed the title Hydra and OmegaConf Bumps OmegaConf Test Bug Fix Mar 19, 2026
@NickGeneva
Copy link
Collaborator Author

@greptile-ai

@NickGeneva
Copy link
Collaborator Author

/blossom-ci

1 similar comment
@NickGeneva
Copy link
Collaborator Author

/blossom-ci

Copy link
Collaborator

@shrek shrek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@NickGeneva NickGeneva merged commit b30eb61 into NVIDIA:0.13.0-rc Mar 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants