OpenEO Processes Dask: Machine Learning

openeo-processes-dask-ml is a Python package that implements generic machine learning (ML) processes for openEO. It is built to work alongside and integrate with openeo-processes-dask, extending it by machine learning capabilities.

Warning

This package is Work-In-Progress, and everything is experimental. You will likely encounter many NotImplementedError.
At the moment, it is rather a proof-of-concept than something to be used in production.

Currently supported

• Loading pre-trained ML models using the load_ml_model process
• Use the loaded ML model to make predictions from the datacube using ml_predict
• Restructure ML model output back into a datacube structure.

Installation

This package is not published on PyPI yet. It can only be used from source

Development environment

Due to gdal dependency, installation can be quite tricky (on Linux at least...) and we have to use a combination of conda and uv for installation.

Installation on Linux:

1. Clone the repository
2. Create a conda environment: conda create -n gdal python=3.14 gdal=3.13.1
3. Activate conda environment: conda acitvate gdal
4. Install it using uv from the conda env: uv sync --python "$CONDA_PREFIX/bin/python" --extra scikit-learn --extra torch-cuda
5. Deactivate conda env: conda deactivate
6. Run the test suite (Errors may occur if optional dependency groups were not installed): uv run pytest

(As I do not have a Windows or MacOS system, I can not provide and test installation instructions for these systems. Please test and contribute them with a PR.)

Optional dependencies: Runtime packages for a specific ML framework are provided in optional dependencies:

• torch-cuda and torch-cpu: For pytorch ML models with and without GPU support. Can not be installed at the same time.
• scikit-learn: For scikit-learn based ML models

Extensibility

This package is made to be easily extensible (e.g. adding support for new ML frameworks) by inheriting from openeo_processes_dask_ml.process_implementations.data_model.data_model.MLModel and implementing the abstract methods.

Pre-commit hooks

This repo makes use of pre-commit hooks to enforce linting & a few sanity checks. In a fresh development setup, install the hooks using poetry run pre-commit install. These will then automatically be checked against your changes before making the commit.

Structure

• minibackend has a minimal backend implementation for executing process graphs
• opd-ml-dev-utils has some scripts that are helpful during development
• openeo-processes-dask-ml the actuall ML process specs and implementations
• ml_datacube_bridge contains glue code to convert between datacube arrays and ML model input and output
• tests for pytest

Acknowledgement

Development within this repository are carried out as part of the Embed2Scale project and is cofunded by the EU Horizon Europe program under Grant Agreement 101131841. Additional funding for this project has been provided by the Swiss State Secretariat for Education, Research and Innovation and UK Research and Innovation.