Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
bb0fb1b
Add cdsapi dependency
rosielickorish Feb 10, 2026
5bc42dd
Update docs and API Key instructions for CDS
rosielickorish Feb 10, 2026
f481bb5
Update lock file
rosielickorish Feb 10, 2026
3fb50b1
Update check_header script to accommodate new year
rosielickorish Feb 10, 2026
c71e6bb
Add climate_data_store connector type
rosielickorish Feb 10, 2026
842e316
Add projections-cordex-domains-single-levels to list_collections
rosielickorish Feb 10, 2026
612a5f9
Ensure connector_template has template connect name in docstrings
rosielickorish Feb 10, 2026
4a71608
Ensure CDS API Key is present
rosielickorish Feb 10, 2026
8410295
find cds data given collection constraints
rosielickorish Feb 11, 2026
c513f27
Update docs to include cordex domain details
rosielickorish Feb 11, 2026
7667cdf
Include API docs for CDS
rosielickorish Feb 11, 2026
345e514
Implement CDS get_data for CORDEX and ERA5 datasets
rosielickorish Feb 11, 2026
593369a
Fix test data path and ensure CDS API mock is only in conftest.py
rosielickorish Feb 11, 2026
347a711
Add documentation for CDS collection domains
rosielickorish Feb 11, 2026
eef7895
Improve docstrings for get_data for other data connectors
rosielickorish Feb 11, 2026
551db53
Script to generate test data for CDS data connector
rosielickorish Feb 11, 2026
ab227b6
Merge branch 'main' of github.com:terrastackai/terrakit into feature/…
rosielickorish Mar 16, 2026
01aaafa
Improve handling of era5 get_data
rosielickorish Mar 22, 2026
7e7b75b
Improve ERA5 data processing
rosielickorish Mar 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env_template
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
SH_CLIENT_ID=""
SH_CLIENT_SECRET=""
NASA_EARTH_BEARER_TOKEN=""
CDSAPI_KEY=""
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ dmypy.json
*.tortilla
*metadata.json
*.nc

*.zip
# Test data
test_wildfire_vector

Expand Down
65 changes: 51 additions & 14 deletions .secrets.baseline
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"files": "^.secrets.baseline$",
"lines": null
},
"generated_at": "2025-11-19T10:10:18Z",
"generated_at": "2026-03-24T13:55:12Z",
"plugins_used": [
{
"name": "AWSKeyDetector"
Expand Down Expand Up @@ -85,54 +85,68 @@
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "365a32402f2062038d01718da7de099f94555775",
"is_verified": false,
"line_number": 98,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "d3270f852a922a83e8e2dabb8535a68464ec5165",
"is_verified": false,
"line_number": 113,
"line_number": 121,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "4e1e76c221c6dace8f9a2c943e00d3328366366a",
"is_verified": false,
"line_number": 122,
"line_number": 130,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "54a38e82cf07a4f36900b855ca7dd0584ca9f3ff",
"is_verified": false,
"line_number": 124,
"line_number": 132,
"type": "Secret Keyword",
"verified_result": null
}
],
"docs/download_data.md": [
"docs/data_connectors.md": [
{
"hashed_secret": "5204df45fc8c724684bbc61cd4107a726a6b9204",
"is_verified": false,
"line_number": 175,
"line_number": 65,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "365a32402f2062038d01718da7de099f94555775",
"is_verified": false,
"line_number": 67,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "d401c389d24671398ced85b259e17975a4fcb65e",
"is_verified": false,
"line_number": 192,
"line_number": 90,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "4e1e76c221c6dace8f9a2c943e00d3328366366a",
"is_verified": false,
"line_number": 201,
"line_number": 99,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "54a38e82cf07a4f36900b855ca7dd0584ca9f3ff",
"is_verified": false,
"line_number": 203,
"line_number": 101,
"type": "Secret Keyword",
"verified_result": null
}
Expand Down Expand Up @@ -213,32 +227,55 @@
{
"hashed_secret": "22d3fe8cf3013398e16de38c7d1ab68a5ed77e37",
"is_verified": false,
"line_number": 174,
"line_number": 176,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "97f8a44eec24e0a90c4217c103e88b3ddfb7f2ba",
"is_verified": false,
"line_number": 381,
"line_number": 383,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "7a77237e10ab85b114a83affb9567c10eb4eda32",
"is_verified": false,
"line_number": 383,
"line_number": 385,
"type": "Secret Keyword",
"verified_result": null
},
{
"hashed_secret": "156d45846d67877099a7a846d3cc9c16e7dc784a",
"is_verified": false,
"line_number": 475,
"line_number": 477,
"type": "Secret Keyword",
"verified_result": null
}
],
"docs/examples/terrakit_download_cds.ipynb": [
{
"hashed_secret": "45e9f10b3b6aa19c8124c22a9ff2723498779f62",
"is_verified": false,
"line_number": 220,
"type": "Hex High Entropy String",
"verified_result": null
},
{
"hashed_secret": "f7034486e49d0ed60b3bded84f7b05288be9283d",
"is_verified": false,
"line_number": 390,
"type": "Base64 High Entropy String",
"verified_result": null
},
{
"hashed_secret": "b084221739b62e38c35cba68dfd407485d9b8d0c",
"is_verified": false,
"line_number": 425,
"type": "Base64 High Entropy String",
"verified_result": null
}
],
"terrakit/general_utils/labels_downloader.py": [
{
"hashed_secret": "34fd2a7f5faa004cd1b9e4a22aa09b16d521661b",
Expand Down Expand Up @@ -268,7 +305,7 @@
{
"hashed_secret": "20adce7ba26cdb1b654e8a53d42ef471580b92ad",
"is_verified": false,
"line_number": 69,
"line_number": 73,
"type": "Secret Keyword",
"verified_result": null
}
Expand Down
17 changes: 13 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,15 @@ Data connectors are classes which enable a user to search for data and query dat
* get_data()


### Available data connectors
### Available collections
The following data connectors and associated collections are available:

| Connectors | Collections |
| ----------------- | ----------- |
| sentinelhub | s2_l1c, dem, s1_grd, hls_l30, s2_l2a, hls_s30 |
| nasa_earthdata | HLSL30_2.0, HLSS30_2.0 |
| sentinel_aws | sentinel-2-l2a |
| climate_data_store| derived-era5-single-levels-daily-statistics, projections-cordex-domains-single-levels|
| IBMResearchSTAC | ukcp18-land-cpm-uk-2.2km, ch4, sentinel-5p-l3grd-ch4-wfmd |
| TheWeatherCompany | weathercompany-daily-forecast |

Expand All @@ -94,19 +95,27 @@ Each data connector has a different access requirements. For connecting to Senti
SH_CLIENT_ID="<SentinelHub Client ID>"
SH_CLIENT_SECRET="<SentinelHub Client Secret>"
NASA_EARTH_BEARER_TOKEN="<NASA EarthData Bearer Token>"
CDSAPI_KEY="<Climate Data Store API Key>"
```

### NASA Earthdata
To access NASA Earthdata, register for an Earthdata Login profile and requests a bearer token. https://urs.earthdata.nasa.gov/profile
To access NASA Earthdata, register for an Earthdata Login profile and requests a bearer token. [https://urs.earthdata.nasa.gov/profile](https://urs.earthdata.nasa.gov/profile)

### Sentinel Hub
To access sentinel hub, register for an account and requests an OAuth client using the Sentinel Hub dashboard https://www.planet.com
To access sentinel hub, register for an account and requests an OAuth client using the Sentinel Hub dashboard [https://www.planet.com](https://www.planet.com)

### Sentinel AWS
Access sentinel AWS data is open and does not require any credentials.

### Climate Data Store
Create an account at [https://cds.climate.copernicus.eu/](https://cds.climate.copernicus.eu/). Once created, find your API key under the `Profile` section and add to your `.env` file. Each dataset may also require accepting the licence agreement. If this is the case, the first time a request is made, an error will be returned with the url to visit to accept the terms.

Available collections include:
- [ERA5 post-processed daily statistics on single levels from 1940 to present](https://cds.climate.copernicus.eu/datasets/derived-era5-single-levels-daily-statistics?tab=overview)
- [CORDEX regional climate model data on single levels](https://cds.climate.copernicus.eu/datasets/projections-cordex-domains-single-levels?tab=overview)

### The Weather Company
To access The Weather Company, register for an account and requests an API Key https://www.weathercompany.com/weather-data-apis/. Once you have an API key, set the following environment variable:
To access The Weather Company, register for an account and requests an API Key [https://www.weathercompany.com/weather-data-apis/](https://www.weathercompany.com/weather-data-apis/). Once you have an API key, set the following environment variable:

```
THE_WEATHER_COMPANY_API_KEY="<>"
Expand Down
16 changes: 16 additions & 0 deletions docs/api/data_connectors/climate_data_store.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@

# Climate Data Store Data Connector Documentation

Documentation for the `terrakit.download.data_connectors.climate_data_store` data connector module.


::: terrakit.download.data_connectors.climate_data_store
handler: python
options:
members:
- CDS
show_root_heading: True
show_source: true
show_object_full_path: false
show_signature: false

Loading