-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
46 implement actris ebas reader #48
Conversation
…is-ebas-reader # Conflicts: # setup.cfg
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The implementation is rather complicated to understand since it downloads data on use instead of acting on pre-downloaded files. I wish the download part could be separated from making the data accessible for pyaerocom
def metadata(self): | ||
return self._metadata | ||
|
||
def _read( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is eager and does all the reading of all variables at once, where as I think we would like to have the reading done when calling data(var_to_read)
, for minimising the amount of data kept per iteration. Just a note as refactoring at this point might be a lot of work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is that in pyaerocom there's a check against supported variables.
For that info to get, pyro has to know the variable names AFTER the variable name changing reader and is calling data(var_to_read)
for that. And that is the reason the data is read early on (if I understood the code correctly).
In any case, this reader needs an variable name filter anyway.
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
Co-authored-by: Magnus Ulimoen <[email protected]>
The nature of this reader is to work online. The data can change at any time and it would too computational heavy to download all data from Nilu (for Nilu). Therefore the online reading is intended and part of the contract with ACTRIS. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving to include this reader as a prototype for the release of this month
1st iteration of an ACTRIS-EBAS reader (1st because neither the API nor the netcdf file's format is stable yet).
Done due to the need to test and use what NILU does to find errors on their side
closes #46