-
Notifications
You must be signed in to change notification settings - Fork 24
Implement a rex backend for xarray (rexarray)
#192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 62 commits
Commits
Show all changes
74 commits
Select commit
Hold shift + click to select a range
1ccd031
[WIP] Crude first pass of backend implementation
ppinchuk 8417942
Add data
ppinchuk 6fc9507
Add init module
ppinchuk 8b9da95
Register backend
ppinchuk f551a05
[WIP] Add tests
ppinchuk 1aae855
Add `xarray` req
ppinchuk faa9a23
Add package data
ppinchuk de4151b
Add min version to xarray req
ppinchuk f36bb7c
Update version string
ppinchuk 0f899d8
Add dask as a req
ppinchuk 95a9edf
Fix segfault
ppinchuk 2b6a7f6
Better detect dimensions
ppinchuk ef5824f
Add test for dims
ppinchuk b8a019b
Add `rex_unscale` function
ppinchuk f3050a7
Use `rex_unscale`
ppinchuk fdafb69
Python 3.9 compat
ppinchuk 458c0b5
Update reqs
ppinchuk 1b1054a
Python 3.9 fix
ppinchuk 1c2022f
Custom ds loading so that close method is set properly
ppinchuk 952200b
Add tests for closing mf ds
ppinchuk 9d86aad
Minor updates
ppinchuk 61a923f
Add test
ppinchuk 228a445
Add docstrings
ppinchuk 370d095
Add xarray tests
ppinchuk 348677e
Add fsspec (S3) support
ppinchuk 1184111
Remove deprecated arg
ppinchuk 5e204e6
Use `open_mfdataset`
ppinchuk 6f6c77c
Minor formatting
ppinchuk b2961c4
Fix s3 tests
ppinchuk 350d0d0
Fix tests
ppinchuk 95b7f67
Python 3.9 fix
ppinchuk 3306ad1
Use existing ds
ppinchuk fa7492e
Remove unused code
ppinchuk a45c808
Support hsds
ppinchuk bc7d31f
Add xarray tests
ppinchuk 8dda21f
Don't need to store hsds var
ppinchuk 308d7dd
coordinates not take precedent over meta
ppinchuk 5717511
Add coords test
ppinchuk 80d2c2d
Bump version
ppinchuk 66a82f7
Merge remote-tracking branch 'origin/main' into pp/rexarray
ppinchuk 99a61cc
Use backward-compatible syntax
ppinchuk ffe2e73
Support data trees
ppinchuk e55b058
tests for data trees
ppinchuk 362f4d6
Add logger
ppinchuk b39d952
Python 3.9 compat
ppinchuk 19d12f3
Fix skip conditions
ppinchuk df15362
Add some utility funcs
ppinchuk 62b019d
More utility funcs
ppinchuk f54b247
Use new utility functions
ppinchuk 1df3009
Cleanup part 1
ppinchuk 2d72d85
More cleanup
ppinchuk bb1edc0
More cleanup
ppinchuk b958526
More cleanup
ppinchuk 9321598
More cleanup
ppinchuk d1174e2
Add test for overwriting attrs
ppinchuk 85e1392
Linter fix
ppinchuk 490573d
Formatting
ppinchuk 233fb1a
Reformat
ppinchuk b0a6687
Linter fixes
ppinchuk 305f15e
Add nsrdb attrs
ppinchuk d31116f
Fix test
ppinchuk 24c501b
Fix typo
ppinchuk 4378b31
Remove bad name
ppinchuk 2ef9fdc
Add guards for dask req
ppinchuk 9a8911b
Merge functions
ppinchuk 10e882c
Add `filename_from_h5pyd` function
ppinchuk eb80b8d
Add test for multiple files
ppinchuk b2bd8ed
Add `open_mfdataset_hsds` implementation
ppinchuk 9dd880c
Rename dimension
ppinchuk 94039ef
`fsspec` now a core req
ppinchuk bc02b6a
Fix for windows
ppinchuk 8c9f953
ensure that rexarray can open mf hsds with list input
grantbuster b057a12
docstrings
grantbuster 47ff8c1
Merge pull request #204 from NREL/gb/xr_hsds_list
grantbuster File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| { | ||
| "air_temperature": { | ||
| "dtype": "i2", | ||
| "scale_factor": 0.1 | ||
| }, | ||
| "alpha": { | ||
| "dtype": "u1", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "aod": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.001 | ||
| }, | ||
| "asymmetry": { | ||
| "dtype": "i1", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "cld_opd_dcomp": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "cld_press_acha": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "cld_reff_dcomp": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "clearsky_dhi": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "clearsky_dni": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "clearsky_ghi": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "cloud_fill_flag": { | ||
| "dtype": "u1", | ||
| "scale_factor": 1 | ||
| }, | ||
| "cloud_type": { | ||
| "dtype": "i1", | ||
| "scale_factor": 1 | ||
| }, | ||
| "dew_point": { | ||
| "dtype": "i2", | ||
| "scale_factor": 0.1 | ||
| }, | ||
| "dhi": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "dni": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "fill_flag": { | ||
| "dtype": "u1", | ||
| "scale_factor": 1 | ||
| }, | ||
| "ghi": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "inversemoninobukhovlength": { | ||
| "dtype": "i2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "ozone": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.001 | ||
| }, | ||
| "precipitationrate": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.1 | ||
| }, | ||
| "pressure": { | ||
| "dtype": "u2", | ||
| "scale_factor": 10 | ||
| }, | ||
| "relativehumidity": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "relative_humidity": { | ||
| "dtype": "i1", | ||
| "scale_factor": 0.01 | ||
| }, | ||
|
|
||
| "solar_zenith_angle": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "ssa": { | ||
| "dtype": "u1", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "surface_albedo": { | ||
| "dtype": "u1", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "surface_pressure": { | ||
| "dtype": "u2", | ||
| "scale_factor": 1 | ||
| }, | ||
| "temperature": { | ||
| "dtype": "i2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "total_precipitable_water": { | ||
| "dtype": "u1", | ||
| "scale_factor": 0.1 | ||
| }, | ||
| "winddirection": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "wind_direction": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "windspeed": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| }, | ||
| "wind_speed": { | ||
| "dtype": "u2", | ||
| "scale_factor": 0.01 | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.