Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding R nc to DwC notebook #13

Merged
merged 33 commits into from
May 16, 2024
Merged

adding R nc to DwC notebook #13

merged 33 commits into from
May 16, 2024

Conversation

MathewBiddle
Copy link
Contributor

Not complete yet

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@MathewBiddle
Copy link
Contributor Author

# Conflicts:
#	jupyterbook/content/code_gallery/data_management_notebooks/DRAFT-R-netCDF2DwC.ipynb
@MathewBiddle
Copy link
Contributor Author

xref: inbo/etn#256

@MathewBiddle
Copy link
Contributor Author

The notebook is unfinished but it already has a lot of information. Does it make sense to break it down into smaller steps?

Agreed. There is a lot of printing of the contents of variables and netCDF stuff. Maybe that should be reduced and could help with the overwhelming amount of information?

Are there some small nuggets we can pull out of this notebook to add to the CodeLab? Initially I wanted to add this notebook here because I wanted to highlight using netCDF in the R language (I heard some comments that they didn't play nicely together).

I propose that this notebook, in it's entirety, gets moved over to bio_data_guide as an R markdown notebook. Then, we simplify the notebook here to reading netCDF data from R and making a map (cells 1-11).

@ocefpaf
Copy link
Member

ocefpaf commented Nov 16, 2023

Agreed. There is a lot of printing of the contents of variables and netCDF stuff. Maybe that should be reduced and could help with the overwhelming amount of information?

If outputting just a few lines for some tables still conveys the message, that should help a bit.

Are there some small nuggets we can pull out of this notebook to add to the CodeLab? Initially I wanted to add this notebook here because I wanted to highlight using netCDF in the R language (I heard some comments that they didn't play nicely together).

netCDF problems are always hard to understand without more context. Is it installation? Are folks compiling from source? Or maybe v3 vs v4? netCDF does plays with R but the devil is in the details and folks who are not used to it may hit a wall.

I propose that this notebook, in it's entirety, gets moved over to bio_data_guide as an R markdown notebook. Then, we simplify the notebook here to reading netCDF data from R and making a map (cells 1-11).

+1, and we can link them to help direct traffic from the short version to the full one and vice-versa.

removing data files
@MathewBiddle
Copy link
Contributor Author

Okay, I simplified the notebook and removed all the DwC formatting and metadata generation stuff.

I'm thinking the map would be nice to include some depiction of the quality of each point to highlight the odd appearance off Hawaii.

Maybe we could use ERDDAPs interpolate feature to show what water temperatures were during this animals movement?

@MathewBiddle
Copy link
Contributor Author

I feel better about the notebook now. Using the animal time, lat, and lon, we ping coastwatch ERDDAP for SST data using the interpolate feature.

Now I think the notebook should be moved to data_analysis_and_visualizations? Maybe data_access? I'm not sure.

Comments are welcome!

@ocefpaf
Copy link
Member

ocefpaf commented Nov 21, 2023

Now I think the notebook should be moved to data_analysis_and_visualizations? Maybe data_access? I'm not sure.

Both are good places but data_access is "overloaded" and it would be nice to add more notebooks in another category.

@MathewBiddle
Copy link
Contributor Author

@ocefpaf I moved it, renamed it, added some context. When you get a chance, mind taking a look so we can finish it out? Feel free to put on your oceanographer hat for the last bit in using the MURSST data. I'd like to show something more meaningful than a timeseries of temperature data...

@ocefpaf
Copy link
Member

ocefpaf commented Feb 5, 2024

Some (very minor) nitpicks:

  • I would remove comments that can be inferred from the code, like #Load libraries , #includes stringr, #fname = str_split_i(url, "/", i=-1) and others.
  • the maps package is triggering a deprecation warning. I'm not a R users so I don't know if it is a matter of just wainting for an update or if we must find a substitute for it.
  • cell[3]: prints a lot of metadata! That makes it a bit hard to read the notebook. Maybe we should print only the summary or the title?
  • cel[10]: I'm not sure that plot is useful.
  • cell[11]: Maybe overlaid on the SST image?
  • cell[11]: The markdown says with the colors dependent on time but I don't see any colors, right?

My oceanographic hat is stored someplace I cannot find but... What if you plot the sharks position with a color coded SST circle? Similar to what we did in https://ioos.github.io/ioos_code_lab/content/code_gallery/data_analysis_and_visualization_notebooks/2021-10-25-ERDDAP-interpolate.html ?

@MathewBiddle
Copy link
Contributor Author

Thanks @ocefpaf! I think I addressed all your comments. We could probably make the mapping a function which might clean up the code.

I tried to replicate the map in the notebook you referenced.

@ocefpaf
Copy link
Member

ocefpaf commented Feb 5, 2024

I'm not an R expert but it LGTM!

@MathewBiddle
Copy link
Contributor Author

@laurabrenskelle I just ran through this notebook and the NCEI data access url changed from nodc -> ncei. The notebook has been updated to reflect the change. Do you mind having a look at it?

@laurabrenskelle
Copy link
Contributor

I ran through this and it looks good to me.

@MathewBiddle
Copy link
Contributor Author

Thanks @laurabrenskelle. @ocefpaf let's merge this and move on! :)

@ocefpaf ocefpaf merged commit 9ee7dd8 into ioos:main May 16, 2024
10 of 11 checks passed
@ocefpaf
Copy link
Member

ocefpaf commented May 16, 2024

I'll create the gallery entry for it in a bit.

ocefpaf added a commit to ocefpaf/ioos_code_lab that referenced this pull request May 16, 2024
ocefpaf added a commit to ocefpaf/ioos_code_lab that referenced this pull request May 16, 2024
ocefpaf added a commit to ocefpaf/ioos_code_lab that referenced this pull request May 16, 2024
MathewBiddle added a commit that referenced this pull request May 16, 2024
@MathewBiddle MathewBiddle deleted the r_nc2dwc branch June 7, 2024 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Announcement
Development

Successfully merging this pull request may close these issues.

4 participants