diff --git a/docs/content/tutorials/fundamentals/0_project_overview.ipynb b/docs/content/tutorials/fundamentals/0_project_overview.ipynb new file mode 100644 index 0000000..cafda18 --- /dev/null +++ b/docs/content/tutorials/fundamentals/0_project_overview.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a2e2e354-9dcf-4af5-84d8-51142b01f1b2", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "# Project Overview\n", + "\n", + "This project demonstrates a typical project workflow in GeoLab. The objective is to instruct how to download data from EarthScope web services, and how to access data in the cloud. The difference between the two methods is that web services are file download services. In contrast, cloud services stream data directly to memory, which is more efficient and enables processing more data.\n", + "\n", + "This exercise covers:\n", + "\n", + "- how to initialize a project using git\n", + "- how to create a Python environment and manage packages\n", + "- explains how to get an EarthScope token to access data\n", + "- how to download seismic and geodetic data using EarthScope web services\n", + "- how access seismic and geodetic data in the cloud\n", + " \n", + "A section can include quizzes, working in the terminal, or a coding exercise. The project map below shows how the sections correspond to notebooks.\n", + "\n", + "![](images/project_work_flow.png)\n", + "\n", + "## Using Git\n", + "\n", + "Git is used to store and track code changes. We strongly recommend that you start a project by creating a git repository. First, by creating a git repository, you can rollback any changes made in your code. For example, adding a new feature. If it fails to work, you can revert your changes to the point where the code was working. Git also enables the creation of branches, which let you add new features without disrupting working code. If you create a GitHub account, you can keep a copy of the repository online and share it with others. There are many benefits to using git to manage project code.\n", + "\n", + "The notebook covers the basics of creating a git repository and managing code.\n", + "\n", + "## Python Environments and Package Management\n", + "\n", + "Creating a Python environment for a project is a best practice. Environments determine the version of Python used and the software packages available for a project. GeoLab's default environment contains many packages useful for scientific computing. However, you may want to add a package, and this module explains how to add packages and the basics of managing a Python environment.\n", + "\n", + "## Getting Authorization to Access EarthScope Services\n", + "\n", + "To download or access EarthScope SAGE and GAGE data, you will need an EarthScope account. Tracking data usage is a requirement of EarthScope's funders. Usage is tracked by a issuing an 0Auth token. You can request a token using EarthScope's CLI (Command Line Interface) or programmatically with EarthScope's SDK (Software Development Kit). This section demonstrates both methods for requesting a token.\n", + "\n", + "In the future, EarthScope will provide access to data in AWS S3 buckets. Direct data access is briefly discussed and will be expanded when the service becomes available.\n", + "\n", + "## Getting Seismic Data\n", + "\n", + "This task demonstrates how to download files from SAGE and GAGE web services using Python. How to formulate a request with the correct parameters and authorization is shown. In addition, this section demonstrates how to access data from an AWS S3 bucket that does not require authorization.\n", + "\n", + "## Seismic Data Exercise\n", + "\n", + "Using the skills from the previous notebook. You will complete a Python script that will download data from EarthScope web services.\n", + "\n", + "## Getting Geodetic Data\n", + "\n", + "This notebook demonstrates how to get cloud native seismic and geodetice data using the `boto3` Python and the EarthScope SDK. This section demonstrates how to access data from an AWS S3 bucket without authorization, and how the EarthScope SDK manages authorization.\n", + "\n", + "## Geodetic Data Exercise\n", + "\n", + "Using the skills from the previous notebook. You will complete a Python script that pull seismic and geodetic data from the cloud." + ] + }, + { + "cell_type": "markdown", + "id": "d3c7e78d-5791-4c1a-a59c-8af6467921a7", + "metadata": {}, + "source": [ + "## [Next >](./1_git_intro.ipynb)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d28597e-bba5-4d3b-aa4a-98524e741a31", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/1_git_intro.ipynb b/docs/content/tutorials/fundamentals/1_git_intro.ipynb new file mode 100644 index 0000000..81e0f51 --- /dev/null +++ b/docs/content/tutorials/fundamentals/1_git_intro.ipynb @@ -0,0 +1,466 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b6d2d667", + "metadata": {}, + "source": [ + "# Intro to Git \n", + "\n", + "## Keeping track of code\n", + "\n", + "Git is a free and open-source distributed version control system (VCS). Software developers use it to track changes to their code over time, enabling collaboration and efficient management of project versions. Think of it as a sophisticated \"undo\" button and a way to manage different versions of files, which is especially useful when multiple people are working on the same project. You can also use git to save your work outside of GeoLab.\n", + "\n", + "## Installation\n", + "\n", + "GeoLab includes the git client. However, to check if git is installed on a computer, use the version flag, which returns the version of git.\n", + "\n", + "```\n", + "git --version\n", + "```\n", + "\n", + "## Setting up Git\n", + "\n", + "Git is a distributed system, meaning you can have a version of code on your computer and access it through a Git service provider. By configuring git with your user name and email, you'll be able to make changes on the remote copy of your code. To configure git in GeoLab, use the following commands in the terminal.\n", + "\n", + "```\n", + "git config --global user.name anu\n", + "git config --global user.email anu@gmail.com\n", + "```\n", + "\n", + "## How Git Works\n", + "\n", + "Git manages changes to your projects. It uses repositories, which contain all your project files and their complete history. Git keeps track of every modification you make, allowing you to save different versions of your work in a branch and merge changes back into the main project. You can collaborate with others by sharing your project online with a remote repository. You can get feedback through pull requests from other project users and maintainers, and integrate their contributions seamlessly. Git keeps track of changes and lets you collaborate with other project users, making it easier to manage and improve your projects without losing any work.\n", + "\n", + "## Getting Started with Git\n", + "\n", + "There are two ways to start using git. The first way is to `clone` or copy a repository from a online git hosting, such as GitLab or GitHub. Cloning a repository copies all the files, code and tracking data, from an online resource to your computer. \n", + "\n", + "The second method is to create a local git repository on your computer. You can create a remote repository in GitHub or GitLab and copy, or `push`, the local repository into the online, or `remote`, repository.\n", + "\n", + "We'll start with cloning a repository, and follow with creating a local repository." + ] + }, + { + "cell_type": "markdown", + "id": "55f11155", + "metadata": {}, + "source": [ + "## Cloning a Repository\n", + "\n", + "Cloning repositories enables us to collaborate on projects with other developers. Cloning a repository creates a complete local copy, including all branches and commit history. You can make changes to the project and push your changes back to the remote repository for others to review and merge. To clone the remote repository, use the following command.\n", + "\n", + "```\n", + "git clone \n", + "```\n", + "\n", + "There are two types of repository URLs used when cloning a project.\n", + "\n", + "- HTTPS URL: Commonly used for cloning and pushing code.\n", + "\n", + "> Example: https://github.com/username/repository.git\n", + "\n", + "- SSH URL: More secure and used for authentication without entering a username or password.\n", + "\n", + "> Example: git@github.com:username/repository.git\n", + "\n", + "`git clone` copies the main branch by default. If you want to clone a specific branch of the repository, you can use the -b option.\n", + "\n", + "```\n", + "git clone -b \n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "e05eb452", + "metadata": {}, + "source": [ + "\n", + "## Creating a Repository\n", + "\n", + "Let's create a repository in GeoLab. Open a terminal, and go to your home directory.\n", + "\n", + "```\n", + "cd ~\n", + "```\n", + "\n", + "Create a new directory named `my_repo` and then use the `cd` command to switch to it.\n", + "\n", + "```\n", + "mkdir my_repo\n", + "cd my_repo\n", + "```\n", + "\n", + "To create a new repository, use the `init` command. The repository has a hidden directory, `.git`, which contains the files for tracking and managing changes to the repository.\n", + "\n", + "```\n", + "git init\n", + "```\n", + "\n", + "If we want to check if there are changes to repository, use the `status` command.\n", + "\n", + "```\n", + "git status\n", + "```\n", + "\n", + "This shows the status of all the files in the repository. Since the repository is empty, git will respond with:\n", + "\n", + "```\n", + "On branch main\n", + "\n", + "No commits yet\n", + "\n", + "nothing to commit (create/copy files and use \"git add\" to track)\n", + "```\n", + "\n", + "## Adding Files\n", + "\n", + "Let's add a Markdown and Python file and add them to the repository.\n", + "\n", + "```\n", + "echo \"# My Repository\" > README.md\n", + "echo \"print(\"Hello GeoLab!\") > hello.py\n", + "```\n", + "\n", + "Although the files are in the directory, git isn't tracking them. When you modify an existing file, create a new file, or delete a file in your working directory, Git considers changes as \"untracked\" or \"modified.\" The `git add` command adds changes from the working directory to the staging area, also known as the index. The staging area acts as a temporary holding area where you can carefully select which changes you want to include when want to save or make a snapshot of the repository. To add all the changes to the staging area, follow the `git add` command with a period, e.g.:\n", + "\n", + "```\n", + "git add .\n", + "```\n", + "\n", + "You can also add specific files:\n", + "\n", + "\n", + "`README.md` and `hello.py` files are added to the repository. You can also add single files by their name. \n", + "\n", + "```\n", + "git add hello.py\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "492a41d8", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "## Committing Files\n", + "\n", + "Committing files takes a snapshot of your code at that moment. That means you can always go back to that point. Use the `commit` command to create a snapshot. The `-m` flag adds a message about the change.\n", + "\n", + "```\n", + "git commit -m \"first commit\"\n", + "```\n", + "\n", + "You can see all your commits with the `log` command. A commit hash references each commit.\n", + "\n", + "```\n", + "git log\n", + "\n", + "commit 10814a1c1cd6d8696ccc1a98eb65720ec95607c1 (HEAD -> main)\n", + "Author: spara-earthscope \n", + "Date: Mon Jul 7 21:29:56 2025 -0600\n", + "\n", + " first commit\n", + "```\n", + "\n", + "We've create a repository snapshot with the two files. Let's see what happens when you delete a file with the `git rm` command which removes the file from staging. We want to update our snapshot with `git commit` and check the commit with `git log`.\n", + "\n", + "\n", + "```\n", + "git rm --cached hello.py\n", + "git commit -m 'deleted hello.py from staging'\n", + "git log\n", + "```\n", + "\n", + "commit 77f2e485703918f938a285f5aac012ab0899d1bf\n", + "Author: spara-earthscope \n", + "Date: Mon Jul 7 21:26:11 2025 -0600\n", + "\n", + " deleted hello.py from staging\n", + "```\n", + "\n", + "At this point you have two snapshots of your code. The first snapshot contains the README.md and hello.py files. The second and current snapshot contains only the README.md file." + ] + }, + { + "cell_type": "markdown", + "id": "e6f3604e", + "metadata": {}, + "source": [ + "## Push\n", + "\n", + "Sometimes you want to save or share your work outside of GeoLab. You can save or publish your code to GitHub by creating a new GitHub repository with these [instructions](https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-new-repository). \n", + "\n", + "Don't initialize the repository with README, license, or gitignore files to avoid errors when pushing you code. \n", + "\n", + "![](images/github_push.png)\n", + "\n", + "GitHub provides the remote repository URL when the remote repository is available. Copy the URL. In your terminal, go to the project directory and set the alias for the remote repository as `origin`.\n", + "\n", + "```\n", + "git remote add origin REMOTE-URL\n", + "```\n", + "\n", + "Verify the remote URL:\n", + "\n", + "```\n", + "git remote -v\n", + "```\n", + "\n", + "To push your project and changes to GitHub:\n", + "\n", + "```\n", + "git push -u origin main\n", + "\n", + "```\n", + "\n", + "Refresh the browser window to see the files you've pushed to the repository." + ] + }, + { + "cell_type": "markdown", + "id": "dacbf116", + "metadata": {}, + "source": [ + "## Pull\n", + "\n", + "If the GitHub repository has been updated, the `git pull` command updates the local version of a repository from a remote repository on GitHub with the changes. It does two things.\n", + "\n", + "- Updates the current local working branch (currently checked out branch)\n", + "- Updates the remote tracking branches for all other branches.\n", + "\n", + "`git pull` fetches (`git fetch`) new commits and merges (`git merge`) these into your local branch. In addition, the remote repository is set (`git remote -v`), then it will pull from the remote branch.\n", + "\n", + "```\n", + "git pull \n", + "```\n", + "\n", + "If the remote repository, or `origin`, is not set, use:\n", + "\n", + "```\n", + "git pull origin \n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "1ac3da09", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "Git manages your code by saving snapshots of the project at various points. You can revert changes to previous versions of the project code. A typical git work flow looks like this:\n", + "\n", + "1. Clone a repository from GitHub or GitLab, or initialize a new repository for a project on your computer.\n", + "2. [Optional] Create a branch for project changes and to preserve the main, or primary branch.\n", + "3. Edit or add files to the repository.\n", + "4. Use the `git add` command to stage the files to save in the repository.\n", + "5. Save a snapshot of the changes with `git commit`. Committing saves the code at that point and lets you go back to if if needed.\n", + "6. `git commit` saves the snapshot on your computer. If you want to publish changes to the remote repository on GitHub or GitLab, use the `git push` command.\n", + "7. Use the `git merge` if you want to update the remote main branch with your changes. If you are woking on the main branch, this isn't necessary and a `push` will merge your changes to main.\n", + "\n", + "Repeat the `add` => `commit` => `push` cycle when you've made a changes to the repository. It can prevent loss of work and let you recover changes throughout the project's life cycle." + ] + }, + { + "cell_type": "markdown", + "id": "1a533357", + "metadata": {}, + "source": [ + "## More Git Resources\n", + "\n", + "Want to learn more about git?\n", + "\n", + "- Git basics in [GitHub](https://docs.github.com/en/get-started/start-your-journey/hello-world)\n", + "- If you want to understand how to collaborate with others, [Git Flow](https://docs.github.com/en/get-started/using-github/github-flow) provides an overview.\n", + "- A deep dive into git with the [Git Pro Book](https://git-scm.com/book/en/v2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e123d2df", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "6687320f", + "metadata": {}, + "source": [ + "## Exercise\n", + "\n", + "Using the information above, complete the exercise.\n", + "\n", + "1. In the terminal, make a directory called `first_project` and change to that directory.\n", + "\n", + " ```\n", + " mkdir first_project\n", + " cd first_project\n", + " ```\n", + " \n", + "2. Create a project by initializing git.\n", + "\n", + "3. Create a README file.\n", + "\n", + " ```\n", + " echo \"# First GeoLab Project\" > README.md\n", + " ```\n", + " \n", + "4. Add the README to the repository.\n", + "\n", + "5. Look at the status of the repository.\n", + "\n", + "6. Commit the README to the repository.\n", + "\n", + "7. Look at the repository log. What is in the log?\n", + "\n", + "In the next section, we'll learn about Python package managers and environments." + ] + }, + { + "cell_type": "markdown", + "id": "b2cf015d", + "metadata": {}, + "source": [ + "# OPTIONAL: Additional Git Commands \n", + "\n", + "## Fixing Mistakes\n", + "\n", + "Git lets you fix mistakes by reverting commits. There are three ways to fix a mistake.\n", + "\n", + "1. `Checkout` is a safe option because you can view a version without changing any other versions and return to the current version. To check out a commit, the first eight to ten characters of the hash are sufficient for uniqueness. \n", + "\n", + "```\n", + "git checkout 10814a1c\n", + "git checkout main\n", + "```\n", + "\n", + "2. `Revert` changes commits. For example, if we want to change a commit using `revert`, it opens a [vim](https://github.com/vim/vim) editor. To save the change, add a new message, then type `:wq`. Revert does not delete a commit; it creates a new commit that shows that the commit was changed.\n", + "\n", + "```\n", + "git revert 10814a1c\n", + "git log\n", + "\n", + "commit 8aa48904a09d826118569227bfaa34fb7a1ff2e7 (HEAD -> main)\n", + "Author: spara-earthscope \n", + "Date: Mon Jul 7 23:06:13 2025 -0600\n", + "\n", + " Revert \"update README\"\n", + "\n", + " This reverts commit 10814a1c1cd6d8696ccc1a98eb65720ec95607c1.\n", + "\n", + " Removed change to README\n", + "```\n", + "3. `Reset` deletes commits from a certain point, e.g.:\n", + "\n", + "```\n", + "git reset 10814a1c\n", + "```\n", + "\n", + "All commits after and including 2aa211 will be deleted and recovered.\n", + "\n", + "## Branches\n", + "\n", + "When you initialize a git repository, it starts at the main branch. The main branch is typically the stable branch, i.e., the branch with working code that you can share. If we want to add a new feature without changing the published code, we add a branch, like this:\n", + "\n", + "```\n", + "git branch new_feature\n", + "```\n", + "\n", + "You can see all the branches in a repository by:\n", + "\n", + "```\n", + "git branch\n", + "```\n", + "\n", + "or by:\n", + "\n", + "```\n", + "git branch -a\n", + "```\n", + "\n", + "It shows `main` and `new_feature` branches. The branch with an asterisk, `*`, is the current branch. To switch to the `new_feature` use the `checkout` command, e.g.:\n", + "\n", + "```\n", + "git checkout new_feature\n", + "```\n", + "\n", + "Note that you can also use `checkout` to create a new branch:\n", + "\n", + "```\n", + "git checkout -b new_feature\n", + "```\n", + "\n", + "You can safely delete a branch if it has been merged into main or another branch.\n", + "\n", + "```\n", + "git branch -d new_feature\n", + "```\n", + "\n", + "To `delete` an unmerged branch, use the force option:\n", + "\n", + "```\n", + "git branch -D new_feature\n", + "```\n", + "\n", + "## Merging Branches\n", + "\n", + "If the new_feature branch is complete, we can merge it into the main branch. First, check out the main branch, then merge the new_feature branch.\n", + "\n", + "\n", + "```\n", + "git checkout main\n", + "git merge new_feature.\n", + "```\n", + "\n", + "If you encounter conflicts, ensure you have updated the branch to be merged by adding and committing the code.\n", + "\n", + "```\n", + "git add .\n", + "git commit -m 'updating code'\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "id": "9e475784-2c57-42c4-992f-50b7ef392297", + "metadata": {}, + "source": [ + "## [< Previous](./0_project_overview.ipynb)             [Next >](./2_package_management.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "geolab", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.14.0" + }, + "toc": { + "base_numbering": 0 + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/2_package_management.ipynb b/docs/content/tutorials/fundamentals/2_package_management.ipynb new file mode 100644 index 0000000..382db07 --- /dev/null +++ b/docs/content/tutorials/fundamentals/2_package_management.ipynb @@ -0,0 +1,186 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "7495572e", + "metadata": {}, + "source": [ + "# Python Environments and Package Management\n", + "\n", + "Python packages are collections of modules that provide additional functionality and tools to Python programs. These packages can contain functions, classes, and variables that are designed to perform specific tasks, ranging from data manipulation and scientific computing to web development and machine learning. Developers can leverage pre-written, tested code to save time and reduce the potential for errors.\n", + "\n", + "Python package managers are tools for installing, updating, configuring, and removing Python packages. Package managers handle dependencies and ensure that software projects have the necessary libraries and supporting packages to run correctly. They fetch and install packages, simplifying the process of integrating external libraries into projects.\n", + "\n", + "## Pip and Conda\n", + "\n", + "[Pip](https://github.com/pypa/pip) and [conda](https://docs.conda.io/en/latest/) are popular package managers. Pip is the default package manager for Python and is commonly used because of its integration with PyPI (Python Package Index), a third-party repository for Python packages. Conda is a package manager that comes with the [Anaconda distribution](https://www.anaconda.com/docs/getting-started/anaconda/main). Conda can handle packages from various languages and is useful in data science and scientific computing because it can manage non-Python dependencies and complex binary packages. Pip is sufficient for pure Python projects, but conda provides additional functionality and ease of use for more complex environments, making it a preferred choice for many data scientists and researchers.\n", + "\n", + "GeoLab is based on the [PanGeo](https://github.com/pangeo-data/pangeo-docker-images) computing environment and uses conda as the package manager. \n", + "\n", + "## Virtual Environments\n", + "\n", + "Virtual environments are a best practice and an integral part of Python development because they provide isolated environments for project development. Isolation ensures that the dependencies of one project do not conflict with those of another. Developers can manage project-specific dependencies effectively and avoid potential conflicts between package versions. The result is an easy-to-manage, clean, and organized development environment.\n", + "\n", + "To list the virtual environments in GeoLab, type the following command in the terminal.\n", + "\n", + "```\n", + "conda env list\n", + "```\n", + "\n", + "The environment with an asterisk is the current environment. To list the installed packages, type:\n", + "\n", + "```\n", + "conda list\n", + "```\n", + "\n", + "The list has four columns: Name, Version, Build, and Channel. The Channel is the repository for a package. If you scroll through the list, [conda-forge](https://conda-forge.org/) is the predominant channel, and [ PyPI or the Python Package Index](https://pypi.org/) is occasionally listed. You can install packages from PyPI in a conda environment, and in general, it will work with the existing dependencies. \n", + "\n", + "To use or activate a specific environment, for example, `notebook`, use the activate command.\n", + "\n", + "```\n", + "conda activate notebook\n", + "```\n", + "\n", + "To leave an environment, use the `deactivate` command \n", + "\n", + "```\n", + "conda deactivate\n", + "```\n", + "\n", + "### Creating Environments\n", + "\n", + "You can create an environment with the `conda create` command.\n", + "\n", + "```\n", + "conda create --name \n", + "```\n", + "\n", + "You can also clone the base environment if you want to keep the existing packages and install additional packages.\n", + "\n", + "```\n", + "conda create --name myenv --clone notebook\n", + "```\n", + "\n", + "More information about creating and managing environments is available in the [documentation](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html). Don't forget to activate the environment after creating it.\n", + "\n", + "### Installing Packages\n", + "\n", + "To install Python packages with conda, type:\n", + "\n", + "```\n", + "conda install \n", + "```\n", + "\n", + "This command installs packages from anaconda.org. If you want to install packages from conda-forge, which has the latest version, add the channel parameter. Either syntax shown below installs a package from a channel.\n", + "\n", + "```\n", + "conda install --channel conda-forge \n", + "conda install conda-forge::\n", + "```\n", + "\n", + "You can also specify package versions, including Python. For example, a package may require Python 3.10.\n", + "\n", + "```\n", + "conda install conda-forge:python=3.10\n", + "```\n", + "\n", + "You can also install packages with pip.\n", + "\n", + "```\n", + "pip install \n", + "```\n", + "\n", + "GeoLab instances are ephemeral. That means if you install a package in the terminal, it will not be there the next time you open a stopped GeoLab. The best practice for installing packages is to install them in a notebook with a [magic command](https://ipython.readthedocs.io/en/stable/interactive/magics.html).\n", + "\n", + "```\n", + "%pip install \n", + "%conda install \n", + "```\n", + "\n", + "Alternatively, you can create a custom environment or clone the base environment and install the required packages. The custom environment will persist and can be reused. In this example, we will clone the default environment and add a beautifulsoup package for parsing web pages. We can list packages and see beautifulsoup4 and bs4 are installed.\n", + "\n", + "```\n", + "conda create --name myenv --clone notebook\n", + "conda activate myenv\n", + "conda install bs4\n", + "conda list\n", + "```\n", + "\n", + "If you leave GeoLab, `myenv` will persist, along with the added packages.\n", + "\n", + "\n", + "## Summary\n", + "\n", + "Python projects use libraries of software, or packages. Building packages with different versions of Python can introduce dependencies on a Python version and other packages as well. Virtual environments manage dependencies by creating a repeatable working space. Conda, the Python package manager, takes this one step further and checks for dependencies and finds, or solves, for packages without dependency issues.\n", + "\n", + "Conda can install many common packages, however, there are also geoscience packages distributed through the Python Package Index (PyPI). These packages can be installed in a conda environment.\n", + "\n", + "These are the common commands to manage a python environment.\n", + "\n", + "1. List the conda environments with `conda env list`.\n", + "2. Create conda environment with `conda create` environment_name.\n", + "3. Use the environment you created with `conda activate` environment_name.\n", + "4. Add conda packages with `conda install` package_name or from the CondaForge repository, `conda install --channel conda-forge` package_name.\n", + "5. Add packages from PyPI with `pip install` package_name.\n", + "6. To leave an environment use `conda deactivate`.\n", + "\n", + "## Exercise\n", + "\n", + "For this exercise, we will create an environment and add packages. Follow these instructions and type the commands in the terminal.\n", + "\n", + "1. In the terminal, change into the geolab directory and create an environment called `first_project`.\n", + "\n", + " ```\n", + " conda create --name first-project --clone base\n", + " ```\n", + "\n", + "3. List the environments; you should see `first_project` in the list.\n", + "\n", + " ```\n", + " conda list\n", + " ```\n", + "\n", + "4. Activate the geolab-intro environment.\n", + "\n", + " ```\n", + " conda activate first_project\n", + " ```\n", + "\n", + "5. List the packages; it should have the same packages as the base environment.\n", + "\n", + " ```\n", + " conda list\n", + " ```" + ] + }, + { + "cell_type": "markdown", + "id": "b87b0908-0428-4673-b2ea-645073191d0a", + "metadata": {}, + "source": [ + "## [< Previous](./1_git_intro.ipynb)             [Next >](./3_authorization.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/3_authorization.ipynb b/docs/content/tutorials/fundamentals/3_authorization.ipynb new file mode 100644 index 0000000..2239f3a --- /dev/null +++ b/docs/content/tutorials/fundamentals/3_authorization.ipynb @@ -0,0 +1,130 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2f74efac-4dfd-4e00-8362-2a748d5a4740", + "metadata": {}, + "source": [ + "# Getting a Token to Access EarthScope Services\n", + "\n", + "EarthScope currently offers data through web services and, in the future, direct access to seismic and geodetic data in the cloud. In this module, we'll learn how to request an authorization token for downloading or accessing data from EarthScope.\n", + "\n", + "## Getting an EarthScope Token\n", + "\n", + "EarthScope's funders require tracking data usage. To this end, EarthScope issues a token to access resources. Tokens are tied to your EarthScope account (which you used to log into GeoLab). Tokens are generated using either the EarthScope SDK (Software Development Kit) or CLI (Command Line Interface). Both packages are installed in GeoLab. The simplest way to generate a token is to use the EarthScope CLI. Copy and paste `es login` into a terminal, then follow the instructions to get a token.\n", + "\n", + "```bash\n", + "es login\n", + "\n", + "Attempting to automatically open the SSO authorization page in your default browser.\n", + "If the browser does not open or you wish to use a different device to authorize \n", + "this request, open the following URL:\n", + "\n", + " https://login.earthscope.org/activate?user_code=GFFX-WRZB\n", + "```\n", + "\n", + "Select the link to open a browser tab, and Confirm.\n", + "\n", + "![](images/login.png)\n", + "\n", + "Tokens are stored in your HOME directory under `.earthscope/default/tokens.json`. When you run an application, earthscope-sdk will find your credentials.\n", + "\n", + "Check to see if a token is present in your home directory.\n", + "\n", + "```bash\n", + "cat $HOME/.earthscope/default/tokens.json\n", + "```\n", + "\n", + "Creating an environment variable for the token makes it easier to use, e.g.:\n", + "\n", + "```bash\n", + "export EARTHSCOPE_TOKEN=\"$HOME/.earthscope/default/tokens.json\"\n", + "cat $EARTHSCOPE_TOKEN\n", + "```\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "04626615-c678-472e-8708-eb2b6ce51805", + "metadata": {}, + "source": [ + "Alternatively, you can get a token programmatically with Python and the EarthScope SDK." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "640f69ba-0199-44ef-b5e1-abe75f88efc4", + "metadata": {}, + "outputs": [], + "source": [ + "from earthscope_sdk import EarthScopeClient\n", + "\n", + "client = EarthScopeClient()\n", + "\n", + "def get_token(token_path='./'):\n", + "\n", + " # refresh the token if it has expired\n", + " client.ctx.auth_flow.refresh_if_necessary()\n", + "\n", + " token = client.ctx.auth_flow.access_token\n", + " \n", + " return token\n", + "\n", + "print(get_token())" + ] + }, + { + "cell_type": "markdown", + "id": "8468d7f4-3dd4-4e90-8fe8-94ba5e703dc2", + "metadata": {}, + "source": [ + "The token enables you to download miniSEED or GNSS data by including it in the web service request. The following [**notebook**](4_data_access.ipynb) demonstrates how to request, save, and stream data in GeoLab.\n", + "\n", + "More information about authorization at EarthScope is [**available**](https://www.earthscope.org/data/authentication/)." + ] + }, + { + "cell_type": "markdown", + "id": "bdde6d81-3c7f-42d1-8fdc-8463a7efc534", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "An authorization token is required to download or access EarthScope seismic and geodetic data. There are two ways to get a token. The first way is to use the EarthScope CLI. Entering `es login` in a GeoLab terminal returns a link that opens a browser window with a button to confirm the token request.\n", + "\n", + "The second method to get a token is to use the EarthScope SDK to request a token programmatically with Python. Note that both the EarthScope CLI and SDK are installed in GeoLab." + ] + }, + { + "cell_type": "markdown", + "id": "8165b249", + "metadata": {}, + "source": [ + "## [< Previous](./2_package_management.ipynb)             [Next >](./4_seismic_data.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/4_seismic_data.ipynb b/docs/content/tutorials/fundamentals/4_seismic_data.ipynb new file mode 100644 index 0000000..05fff63 --- /dev/null +++ b/docs/content/tutorials/fundamentals/4_seismic_data.ipynb @@ -0,0 +1,279 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "fe38ad28-c3c2-48be-946b-5352eae82ded", + "metadata": {}, + "source": [ + "# Getting Seismic Data" + ] + }, + { + "cell_type": "markdown", + "id": "0c1c2659-11c1-4f59-9903-9681a4bb8434", + "metadata": {}, + "source": [ + "This section details how to get seismic data from EarthScope. This section requires a working knowledge of Python. You should have an understanding of data types and string formatting, creating file paths and file naming, web requests, and creating Python functions. In addition to Python, you should have an understanding of miniSEED data distributed through the International Federation of Digital Seismograph Networks (FDSN). \n", + "\n", + "The notebook material is informational and is useful for completing the seismic and geodetic exercises." + ] + }, + { + "cell_type": "markdown", + "id": "6705f359", + "metadata": {}, + "source": [ + "## Getting Seismic Data from EarthScope\n", + "\n", + "Seismic data is available through third party packages such as [obspy](https://docs.obspy.org/) or through EarthScope's `dataselect` web service.\n", + "\n", + "![](images/Web_Services_Data_Flow.png)\n", + "\n", + "Currently, the simplest way to access EarthScope's seismic data is to use a third party package such as obspy. If you want to work directly with miniSEED files, EarthScope's dataselect service supports selecting sorting event data and returns data in the miniSEED format. \n", + "\n", + "![](images/cloud_native_data_access.png)\n", + "\n", + "In the future, data will be available from EarthScopes cloud services hosted in Amazon's S3 storage service. We provide an overview of how data is stored in S3 and an example of how to access data from a public S3 bucket. " + ] + }, + { + "cell_type": "markdown", + "id": "54fbff8a", + "metadata": {}, + "source": [ + "### Getting Data from Obspy and Third-Party Packages\n", + "\n", + "Obspy is a Python framework (or package) for processing seismic data. It provides parsers for common file formats, clients to access data centers and seismological signal processing routines which allow the manipulation of seismological time series.\n", + "\n", + "We'll start by importing modules from the obspy package. It's not necessary to import the entire package, just the function from the module, e.g., obspy.clients.fdsn. Next we create a client that connects to EarthScope's services and send a query based on start and end time, and the minimum magnitude of an event.\n", + "\n", + "```python\n", + "from obspy.clients.fdsn import Client\n", + "from obspy.clients.fdsn.header import URL_MAPPINGS\n", + "from obspy import UTCDateTime\n", + "import warnings\n", + "import cartopy\n", + "\n", + "warnings.filterwarnings('ignore')\n", + "warnings.simplefilter('ignore')\n", + "\n", + "# creates a client that connects to the IRIS data center\n", + "client = Client(\"IRIS\")\n", + "\n", + "starttime = UTCDateTime(\"2020-01-01\")\n", + "endtime = UTCDateTime(\"2025-12-31\")\n", + "catalog = client.get_events(starttime=starttime, endtime=endtime, minmagnitude=7)\n", + "```\n", + "\n", + "The data is returned in a catalog, which is a list-like container for events. To learn more about working with obspy, see the documentation and tutorials on the obspy [web site](chttps://docs.obspy.org/) " + ] + }, + { + "cell_type": "markdown", + "id": "be0ab70a", + "metadata": {}, + "source": [ + "\n", + "### Getting Seismic Data from DataSelect\n", + "\n", + "The `fdsnws-dataselect` service provides access to time series data for specified channels and time ranges. Dataselect implements the [FDSN web service specification](https://www.fdsn.org/webservices/).\n", + "\n", + "Data queries use SEED time series identifiers (network, station, location & channel) in addition to time ranges. Returned data formats include miniSEED, SAC zip, and GeoCSV.\n", + "\n", + "To create a request, the dataselect API takes these parameters at a minimum:\n", + "\n", + "| parameters | examples | discussion | default |type |\n", + "| ---------- | -------- | ---------- | ------- |-----|\n", + "| start[time] |\t2010-02-27T06:30:00\t| Specifies the desired start-time for miniSEED data | | day/time |\n", + "| end[time]\t| 2010-02-27T10:30:00 | Specify the end-time for the miniSEED data | | day/time | \n", + "|net[work] | IU | Select one or more network codes. Accepts wildcards and lists. Can be SEED codes or data center-defined codes. | any | string |\n", + "| sta[tion] | ANMO | Select one or more SEED station codes. Accepts wildcards and lists. | any| string |\n", + "|loc[ation] |00 | Select one or more SEED location identifiers. Accepts wildcards and lists. Use -- for “Blank” location IDs (ID’s containing 2 spaces). | any | string |\n", + "| cha[nnel] | BHZ | Select one or more SEED channel codes. Accepts wildcards and lists. | any | string |\n", + "\n", + "To download a file, we can use the `requests` package to send the HTTP request to the dataselect web service. As discussed in the previous section, the request must include an authorization token using the `get_token` function.\n", + "\n", + "The `download_data` function requires the query parameters required by the FDSN Web Service specification, and where to write the data. The function does several things. First, it requests an authorization token. Next, it creates a file name for the data. Finally, it sends the request to dataselect and writes the data to a file in a directory." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d36e1f63-bc44-49fd-8876-1e501731ab1b", + "metadata": {}, + "outputs": [], + "source": [ + "import requests, os\n", + "from pathlib import Path\n", + "from datetime import datetime\n", + "from earthscope_sdk import EarthScopeClient\n", + "\n", + "# SAGE archive\n", + "URL = \"http://service.iris.edu/fdsnws/dataselect/1/query?\"\n", + "\n", + "# function to get authorization token \n", + "def get_token():\n", + " \n", + " # refresh the token if it has expired\n", + " client.ctx.auth_flow.refresh_if_necessary()\n", + "\n", + " token = client.ctx.auth_flow.access_token\n", + " \n", + " return token\n", + "\n", + "def download_data(params, data_directory):\n", + "\n", + " # get authorization Bearer token\n", + " token = get_token()\n", + "\n", + " # get year and day from string start time\n", + " start_date = datetime.strptime(params['start'], '%Y-%m-%dT%H:%M:%S')\n", + " year = start_date.year\n", + " day = start_date.day\n", + " \n", + " \n", + " # file name format: STATION.NETWORK.YEAR.DAYOFYEAR\n", + " file_name = \".\".join([params[\"sta\"], params[\"net\"],params['loc'],params['cha'], str(year), \"{:03d}\".format(day),'mseed'])\n", + " \n", + " \n", + " r = requests.get(URL, params=params, headers={\"authorization\": f\"Bearer {token}\"}, stream=True)\n", + " if r.status_code == requests.codes.ok:\n", + " # save the file\n", + " with open(Path(Path(data_directory) / file_name), 'wb') as f:\n", + " for data in r:\n", + " f.write(data)\n", + " else:\n", + " #problem occured\n", + " print(f\"failure: {r.status_code}, {r.reason}\")\n", + " \n", + "\n", + "# create client to get token\n", + "client = EarthScopeClient()\n", + "\n", + "# create directory for data\n", + "data_directory = \"./miniseed_data\"\n", + "os.makedirs(data_directory, exist_ok=True)\n", + "\n", + "# parameters specifying the miniSEED file\n", + "params = {\"net\" : 'IU',\n", + " \"sta\" : 'ANMO',\n", + " \"loc\" : '00',\n", + " \"cha\" : 'BHZ',\n", + " \"start\": '2010-02-27T06:30:00',\n", + " \"end\": '2010-02-27T10:30:00'}\n", + "\n", + "download_data(params, data_directory)\n" + ] + }, + { + "cell_type": "markdown", + "id": "729eab32", + "metadata": {}, + "source": [ + "### Seismic Data in AWS S3\n", + "\n", + "Object storage in the cloud is a cost-effective way to hold and distribute very large collections of data. Objects consist of the data, metadata, and a unique identifier and are accessible through an application programming interface, or API. EarthScope uses Amazon Web Services' (AWS) Simple Storage Service, or S3, to store and distribute seismic and geodetic data.\n", + "\n", + "AWS S3 supports streaming data directly into memory. Streaming data is a significant advantage when analyzing large amounts of data because writing and reading data to and from a drive consumes the majority of time when performing an analysis. When data is streamed directly into memory, it is immediately available for processing.\n", + "\n", + "#### Buckets and Keys\n", + "\n", + "Objects in S3 are stored in containers called `buckets`. Each object is identified by a unique object identifier, or `keys`. Objects are accessed using a combination of the web service endpoint, a bucket name, and a key. The combination is called an ARN or an Amazon Resource Name. Unlike a hierarchical file system on your computer, S3 doesn't have directories; instead, it has prefixes, which act as filters that logically group data. Consider the following example, we can decipher the key:\n", + "\n", + "> s3:ncedc-pds/continuous_waveforms/BK/2022/2022.231/MERC.BK.HNZ.00.D.2022.231\n", + "\n", + "- s3 - service name\n", + "- ncedc-pds - bucket name\n", + "- continuous_waveforms - prefix\n", + "- BK - (prefix) seismic network name \n", + "- 2022 - (prefix) year \n", + "- 2022.231 - (prefix) year and day of year\n", + "- MERC.BK.HNZ.00.D.2022.231 - (key) station.network.channel.location.year.day of year\n", + "\n", + "Similar to a web service file URL, the ARN is used to request data." + ] + }, + { + "cell_type": "markdown", + "id": "3e373cc8", + "metadata": {}, + "source": [ + "### S3 Buckets with Public Read Access\n", + "\n", + "S3 buckets can be configured for public read access, and you can access objects without providing credentials. The [`boto3`](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) Python package provides libraries for working with AWS services, including S3. Boto3 provides two methods for interacting with AWS services. The `client` method is a low-level and fine-grained interface that closely follows the AWS API for a service. The 'resource` method is a high-level interface that wraps the 'client` interface. AWS stopped development on the resource interface in `boto3` in 2023; for this reason, we will use the `client` interface when working with S3 resources.\n", + "\n", + "The following example reads a miniSEED file from the Northern California Earthquake Data Center (NCEDC). The trace data is for the 2014 Napa earthquake. GeoLab's default environment includes both `boto3` and `obspy` packages, and we can import them without installation. We establish the connection to S3 by creating a client that specifies that requests are unsigned. This means that the S3 bucket allows public access and does not require credentials. The client calls the `get_object` method with the bucket name and key for the miniSEED object. \n", + "\n", + "Instead of writing the data to a file (as we did using web services), the S3 client sends it to an in-memory binary stream that can be read by [`obspy`](https://docs.obspy.org/). Streaming the data to in-memory objects is more efficient than downloading and reading files for analysis." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "71065adf", + "metadata": {}, + "outputs": [], + "source": [ + "import boto3\n", + "from botocore import UNSIGNED\n", + "from botocore.config import Config\n", + "# from io import BytesIO\n", + "import io\n", + "from obspy import read\n", + "\n", + "s3 = boto3.client('s3', config = Config(signature_version = UNSIGNED), region_name='us-west-2')\n", + "\n", + "BUCKET_NAME = 'ncedc-pds'\n", + "KEY = 'continuous_waveforms/BK/2014/2014.236/PACP.BK.HHN.00.D.2014.236'\n", + "\n", + "response = s3.get_object(Bucket=BUCKET_NAME, Key=KEY)\n", + "data_stream = io.BytesIO(response['Body'].read())\n", + "\n", + "# Parse with ObsPy\n", + "st = read(data_stream)\n", + "\n", + "# Print the ObsPy Streams\n", + "print(st)" + ] + }, + { + "cell_type": "markdown", + "id": "53c43c58-49e0-43f2-b9bd-49b2456c5983", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "There are currently two options to get seismic data from EarthScope. You can use obspy or another third-party package, or use the dataselect service to select and sort event data in miniSEED. In the future, EarthScope will provide access to seismic data through AWS cloud services." + ] + }, + { + "cell_type": "markdown", + "id": "b8c41243-75e2-41a0-99a8-339c4ad0613a", + "metadata": {}, + "source": [ + "## [< Previous](./3_authorization.ipynb)             [Next >](./5_seismic_exercise.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/5_seismic_exercise.ipynb b/docs/content/tutorials/fundamentals/5_seismic_exercise.ipynb new file mode 100644 index 0000000..5550dcb --- /dev/null +++ b/docs/content/tutorials/fundamentals/5_seismic_exercise.ipynb @@ -0,0 +1,391 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "aa63515a-b00e-46a0-a509-749f04c0ca5b", + "metadata": {}, + "source": [ + "# Seismic Data Exercise" + ] + }, + { + "cell_type": "markdown", + "id": "f3cd3e28-a424-46c6-8f09-ba1009490c5e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "## Exercise A: Download miniSEED files from dataselect\n", + "\n", + "This exercise is a script for downloading seismic data from EarthScope in the miniSEED file format. All the functions are covered in the previous `4_seismic.ipynb` notebook. The download function builds a SAGE web service request. Fill in the blanks to complete the script. Download earthquake event data by the query parameters for the SAGE web service. You can use the example in the previous notebook or use another event and station. \n", + "\n", + "If the script is correctly completed you will have a new directory called `/data` containing a miniSEED file." + ] + }, + { + "cell_type": "raw", + "id": "e55fbf74-162c-4bd0-9a17-7cbf21e00772", + "metadata": { + "editable": true, + "raw_mimetype": "", + "slideshow": { + "slide_type": "" + }, + "tags": [ + "hide-cell" + ] + }, + "source": [ + "!pip install jupyterlab-myst" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64c5363c-7739-43ad-961d-3d0f101c7f29", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# ---------- Fill in the package imports for this script ----------\n", + "import ____ \n", + "from datetime import datetime\n", + "from pathlib import Path\n", + "import ____\n", + "\n", + "# ---------- Create an EarthScope client to get token ----------\n", + "client = ____\n", + "\n", + "\n", + "# Fill this with the service endpoint for SAGE data\n", + "SAGE_URL = ____\n", + "\n", + "DATA_DIR = Path(\"./data\")\n", + "DATA_DIR.mkdir(parents=True, exist_ok=True)\n", + "\n", + "# ---------- Get an EarthScope token ----------\n", + "def get_token():\n", + " \"\"\"\n", + " Use the SDK to refresh (if needed) and return an access token.\n", + " \"\"\"\n", + " # Refresh if necessary\n", + " ____\n", + "\n", + " # Retrieve the token string\n", + " token = ____\n", + " return token\n", + "\n", + "\n", + "# ---------- Helper: common auth header ----------\n", + "def auth_header(token):\n", + " \"\"\"\n", + " Helper function to create header with token\n", + " \"\"\"\n", + " # Use a standard Bearer token header (lowercase 'authorization' is fine)\n", + " return ____\n", + "\n", + "\n", + "# ---------- Download a file from EarthScope's web services ----------\n", + "# --------------------------------------------------------------------\n", + "def download_data(url, data_directory, params={}): # params is an optional query parameter\n", + " \"\"\"\n", + " Sends GET with query parameters to the SAGE web service and saves the response \n", + " body to a file. Expected params include: net, sta, loc, cha, start (ISO), end (ISO), etc.\n", + " \"\"\"\n", + " # get authorization token\n", + " token = get_token()\n", + "\n", + " # create the authorization header \n", + " header = auth_header(token) \n", + "\n", + "\n", + " # This creates a file name for a miniSEED file by\n", + " # parsing the start datetime to build filename parts\n", + " start_dt = datetime.strptime(params[\"start\"], \"%Y-%m-%dT%H:%M:%S\")\n", + " year = str(start_dt.year) # Day-of-year (001-366) for file naming\n", + " doy = \"{:03d}\".format(____) # hint: see how year is extracted from the start\n", + " \n", + " # create a file name for a miniSEED\n", + " file_name = \".\".join([\n", + " params[\"sta\"],\n", + " params[\"net\"],\n", + " params[\"loc\"],\n", + " params[\"cha\"],\n", + " year,\n", + " doy,\n", + " \"mseed\"\n", + " ])\n", + " \n", + " # create a variable with the path to the data directory and file\n", + " out_path = Path(Path(data_directory) / ____)\n", + "\n", + " # Make the request to the web service with params and bearer auth\n", + " r = requests.____(\n", + " url,\n", + " params=____,\n", + " headers=____,\n", + " stream=True\n", + " ) \n", + "\n", + " if r.status_code == requests.codes.____:\n", + " with open(out_path, \"wb\") as f:\n", + " for data in r:\n", + " f.write(data)\n", + " else:\n", + " #problem occured\n", + " print(f\"failure: {r.status_code}, {r.reason}\")\n", + " return None\n", + "\n", + "# Download a miniSEED file for an event\n", + "# \n", + "params = {\"net\" : 'IU',\n", + " \"sta\" : 'ANMO',\n", + " \"loc\" : '00',\n", + " \"cha\" : 'BHZ',\n", + " \"start\": '2010-02-27T06:30:00',\n", + " \"end\": '2010-02-27T10:30:00'}\n", + "\n", + "# Try the parameterized request\n", + "download_data(____, DATA_DIR, ___)\n", + "\n", + "# Download a RINEX file\n", + "year = 2025\n", + "day = 1\n", + "station = 'p034'\n", + "doy = '%03d'.format(day)\n", + "compression = 'd.Z' # or \".Z\" / \"\" depending on the archive\n", + "url = create_url(year, ____, station, compression)\n", + "download_data(url, ____)" + ] + }, + { + "cell_type": "markdown", + "id": "e03de38b-f8b2-4f94-8f39-9a59ad45ad41", + "metadata": {}, + "source": [ + "## Exercise B: Access Seismic Data from an AWS S3 Bucket\n", + "\n", + "This exercise shows how to get data from an AWS S3 bucket, stream the data to obspy, and print out the metadata for a trace. Fill in the blanks as needed." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a83e564f-03c3-4385-b933-2a0881245390", + "metadata": {}, + "outputs": [], + "source": [ + "# --- Fill in the blanks (marked ____ ) ---\n", + "import boto3\n", + "from botocore import UNSIGNED\n", + "from botocore.config import Config\n", + "import io\n", + "from obspy import read\n", + "\n", + "# 1) Create an anonymous S3 client in the us-west-2 region\n", + "s3 = boto3.client('s3',\n", + " config=Config(signature_version=____),\n", + " region_name='____')\n", + "\n", + "# 2) Identify what to fetch\n", + "BUCKET_NAME = '____' # public bucket name\n", + "KEY = '____' # object key/path inside the bucket\n", + "\n", + "# 3) Download the object and put bytes into a buffer\n", + "response = s3.get_object(Bucket=____, Key=____)\n", + "data_stream = io.BytesIO(response['Body'].read())\n", + "\n", + "# 4) Parse the miniSEED bytes with ObsPy\n", + "st = read(____)\n", + "\n", + "# 5) Print a summary of the Stream\n", + "print(st)\n", + "\n", + "# 6) (bonus) Print the first Trace's stats: network, station, channel, starttime, sampling_rate\n", + "tr = st[0]\n", + "print(tr.stats.network, tr.stats.station, tr.stats.channel, tr.stats.starttime, tr.stats.sampling_rate)\n" + ] + }, + { + "cell_type": "markdown", + "id": "f5e97140-ba62-4609-8b31-d7e14565354f", + "metadata": {}, + "source": [ + "## Answer Key for Exercise A\n", + "\n", + "```{admonition} Click to see answer\n", + ":class: dropdown\n", + "\n", + "
\n",
+    "# ---------- Imports ----------\n",
+    "import requests\n",
+    "import os\n",
+    "from datetime import datetime\n",
+    "from pathlib import Path\n",
+    "from earthscope_sdk import EarthScopeClient\n",
+    "\n",
+    "# ---------- Create an EarthScope client to get token ----------\n",
+    "client = EarthScopeClient()\n",
+    "\n",
+    "# Fill this with the service endpoint for SAGE data\n",
+    "SAGE_URL = \"http://service.iris.edu/fdsnws/dataselect/1/query?\"\n",
+    "\n",
+    "# create a directory for rinex data\n",
+    "DATA_DIR = \"./data\"\n",
+    "os.makedirs(DATA_DIR, exist_ok=True)\n",
+    "\n",
+    "# ---------- Get an EarthScope token ----------\n",
+    "def get_token():\n",
+    "    \"\"\"\n",
+    "    Use the SDK to refresh (if needed) and return an access token.\n",
+    "    \"\"\"\n",
+    "    # Refresh if necessary\n",
+    "    client.ctx.auth_flow.refresh_if_necessary()\n",
+    "\n",
+    "    # Retrieve the token string\n",
+    "    token = client.ctx.auth_flow.access_token\n",
+    "    return token\n",
+    "\n",
+    "\n",
+    "# ---------- Helper: common auth header ----------\n",
+    "def auth_header(token):\n",
+    "    # Use a standard Bearer token header (lowercase 'authorization' is fine)\n",
+    "    return {\"authorization\": f\"Bearer {token}\"}\n",
+    "\n",
+    "\n",
+    "# ---------- Download a file from EarthScope's web services ----------\n",
+    "# --------------------------------------------------------------------\n",
+    "def download_data(url, data_directory, params={}):\n",
+    "    \"\"\"\n",
+    "    Sends GET with query parameters and saves the response body to a file.\n",
+    "    Expected params include: net, sta, loc, cha, start (ISO), end (ISO), etc.\n",
+    "    \"\"\"\n",
+    "    # get authorization token\n",
+    "    token = get_token()\n",
+    "\n",
+    "    # create the authorization header \n",
+    "    header = auth_header(token) \n",
+    "    \n",
+    "    # Example filename: STA.NET.LOC.CHA.YEAR.DOY.mseed\n",
+    "    start_dt = datetime.strptime(params[\"start\"], \"%Y-%m-%dT%H:%M:%S\")\n",
+    "    year = str(start_dt.year) # Day-of-year (001-366) for file naming\n",
+    "    doy =  \"{:03d}\".format(start_dt.day)  # hint: see how year is extracted from the start\n",
+    "    file_name = \".\".join([\n",
+    "        params[\"sta\"],\n",
+    "        params[\"net\"],\n",
+    "        params[\"loc\"],\n",
+    "        params[\"cha\"],\n",
+    "        year,\n",
+    "        doy,\n",
+    "        \"mseed\"\n",
+    "    ])\n",
+    "\n",
+    "    out_path = Path(Path(data_directory) / file_name)\n",
+    "\n",
+    "    # Make the request with params and bearer auth\n",
+    "    r = requests.get(\n",
+    "        url,\n",
+    "        params=params,\n",
+    "        headers=header,\n",
+    "        stream=True\n",
+    "    ) \n",
+    "\n",
+    "    if r.status_code == requests.codes.ok:\n",
+    "        with open(out_path, \"wb\") as f:\n",
+    "            for data in r:\n",
+    "                f.write(data)\n",
+    "    else:\n",
+    "        #problem occured\n",
+    "        print(f\"failure: {r.status_code}, {r.reason}\")\n",
+    "        return None\n",
+    "\n",
+    "# Example query parameters (adjust to a valid service)\n",
+    "params = {\"net\" : 'IU',\n",
+    "          \"sta\" : 'ANMO',\n",
+    "          \"loc\" : '00',\n",
+    "          \"cha\" : 'BHZ',\n",
+    "          \"start\": '2010-02-27T06:30:00',\n",
+    "          \"end\": '2010-02-27T10:30:00'}\n",
+    "\n",
+    "# Try the parameterized request\n",
+    "download_data(SAGE_URL, DATA_DIR, params)\n",
+    "
\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "09b956bf-6833-462c-9905-20aba01f6dd5", + "metadata": {}, + "source": [ + "## Answer Key for Exercise B\n", + "\n", + "```{admonition} Click to see answer\n", + ":class: dropdown\n", + "\n", + "
\n",
+    "import boto3\n",
+    "from botocore import UNSIGNED\n",
+    "from botocore.config import Config\n",
+    "import io\n",
+    "from obspy import read\n",
+    "\n",
+    "s3 = boto3.client('s3',\n",
+    "                  config=Config(signature_version=UNSIGNED),\n",
+    "                  region_name='us-west-2')\n",
+    "\n",
+    "BUCKET_NAME = 'ncedc-pds'\n",
+    "KEY = 'continuous_waveforms/BK/2014/2014.236/PACP.BK.HHN.00.D.2014.236'\n",
+    "\n",
+    "response = s3.get_object(Bucket=BUCKET_NAME, Key=KEY)\n",
+    "data_stream = io.BytesIO(response['Body'].read())\n",
+    "\n",
+    "st = read(data_stream)\n",
+    "print(st)\n",
+    "\n",
+    "tr = st[0]\n",
+    "print(tr.stats.network, tr.stats.station, tr.stats.channel, tr.stats.starttime, tr.stats.sampling_rate)\n",
+    "
\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "0273e3c7-fcf7-4b42-9d72-981034d3f816", + "metadata": {}, + "source": [ + "## [< Previous](./4_seismic_data.ipynb)             [Next >](./6_geodetic_data.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/6_geodetic_data.ipynb b/docs/content/tutorials/fundamentals/6_geodetic_data.ipynb new file mode 100644 index 0000000..493f949 --- /dev/null +++ b/docs/content/tutorials/fundamentals/6_geodetic_data.ipynb @@ -0,0 +1,297 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "fe38ad28-c3c2-48be-946b-5352eae82ded", + "metadata": {}, + "source": [ + "# Getting Geodetic Data" + ] + }, + { + "cell_type": "markdown", + "id": "0c1c2659-11c1-4f59-9903-9681a4bb8434", + "metadata": {}, + "source": [ + "This section details how to get geodetic data from EarthScope and requires a working knowledge of Python. You should have an understanding of data types and string formatting, creating file paths and file naming, web requests, and creating Python functions. In addition to Python, you should have an understanding of GNSS data. \n", + "\n", + "The notebook material is informational and is useful for completing the seismic and geodetic exercises." + ] + }, + { + "cell_type": "markdown", + "id": "98ab6e5a", + "metadata": {}, + "source": [ + "## Getting Geodetic Data from EarthScope\n", + "\n", + "![](images/cloud_native_data_access.png)\n", + "\n", + "The recommended method to access EarthScope's geodetic data is to use the [EarthScope SDK](). The SDK supports requesting just the GNSS observations data you need instead of a single RINEX file. It reduces the size of data and returns the data in Apache [arrow](https://arrow.apache.org/), which is a memory efficient format that can used by analytic data formats such as Pandas [dataframes](https://pandas.pydata.org/). In the future other GNSS data types will be added to the SDK.\n", + "\n", + "![](images/Web_Services_Data_Flow.png)\n", + "\n", + "In addition to GNSS observations, EarthScope distributes RINEX files and other GNSS data such as navigation and meteorological data. These data can be downloaded as files in their respective formats. An example for downloading files is provided below." + ] + }, + { + "cell_type": "markdown", + "id": "47eb63d8-74e7-41d3-8d3d-4916c466cd5f", + "metadata": {}, + "source": [ + "## Getting RINEX Observations in Apache Arrow from EarthScope\n", + "\n", + "Apache Arrow is a memory based data format and framework that makes large-scale data analysis faster, more efficient, and compatible across different tools. Arrow is a columnar format, which means that data is held in columns instead of rows. File formats, such as RINEX, are row-based formats. This example compares calculating the average L1 signal strength between a RINEX file and an Arrow response.\n", + "\n", + "**Row-based format (RINEX)**: \n", + ">Each observation epoch has: (timestamp, satellite, obs_code, range, phase, snr, slip, flags, fcn, system, igs). To get snr, you would need to read every row and skip the other details, such as phase, slip, system, etc.\n", + " \n", + "**Columnar format (Arrow)**: \n", + ">Data is stored by columns: one column for snr, satellite, phase, etc. Since we're only interested in SNR, you can proceed directly to the SNR column without reviewing the other data.\n", + "\n", + "Another feature of arrow is that it is a memory format that supports zero-copy sharing. This means that other data formats, such as Pandas, xarray, and numpy, can use Arrow data directly in memory without translation. Analyzing data is faster and more efficient without the overhead of translation.\n", + "\n", + "An important feature of arrow is its vectorized operations, which process many values at the same time. Filtering, aggregating, and joining data are done quickly and in memory. Using the previous example, arrow, can filter snr over a specified period of time and aggregate into 15-minute intervals, reducing the amount of data to transfer and analyze.\n", + "\n", + "These features make arrow an ideal way to deliver large amounts of data efficiently.\n", + "\n", + "### Requesting Data in Arrow\n", + "\n", + "The EarthScope SDK supports requesting GNSS observations in arrow. An EarthScope client can request RINEX observations in arrow. Because arrow is an in-memory format, it's commonly converted to a Python data structure for analysis. \n", + "\n", + "[Pandas](https://pandas.pydata.org/) is popular package for working with tabular, or table, data. It enables loading, cleaning, exploring, transforming, and analyzing data efficiently. Arrow tables integrate with other analyis and visualization Python libraries and is commonly used for data exploration.\n", + "\n", + "The following code demonstrates how to use the SDK to request data. As with other methods, an authorization token is needed to make the request. The EarthScope client checks for a token when making a request, which means that it isn't necessary to include it in the request. If you do not have a token, you can use the EarthScope CLI to get a token, see the [**Getting a Token to Access EarthScope Services**](./3_authorization.ipynb) notebook.\n", + "\n", + "The client has a `data.gnss_observation` method for requesting GNSS observation data. The method takes a number of parameters but at minimum, the start and end datetime and station is required. \n", + "\n", + "Note that the request is for 30 hours of data with three hours of data before and after. If we used a web service request, we would have to download two files, extract the data for the time period from each file and join the two files to get the dataset. This method does that automatically and returns only the data you need.\n", + "\n", + "Pandas can convert an arrow table directly into a [dataframe](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html), which is a two-dimensional tabular data structure. The `df` displays the first and last five rows of the table." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e7a166dd-d98c-48ec-8dd4-dc421516b001", + "metadata": {}, + "outputs": [], + "source": [ + "import datetime as dt\n", + "import pandas as pd\n", + "from earthscope_sdk import EarthScopeClient\n", + "\n", + "es = EarthScopeClient()\n", + "\n", + "# Request 30 hours of data (1 day + 3 hour arcs on either side)\n", + "arrow_table = es.data.gnss_observations(\n", + " start_datetime=dt.datetime(2025, 7, 20, 21),\n", + " end_datetime=dt.datetime(2025, 7, 22, 3),\n", + " station_name=\"AC60\",\n", + ").fetch()\n", + "\n", + "df = arrow_table.to_pandas()\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "bdd01ab4-ee05-43a5-9df3-5e56ba7f36e9", + "metadata": {}, + "source": [ + "By specifying values for the columns, you can filter the data so it returns only the data you need. After the arrow table has been converted to a dataframe, you can apply functions, such as sort, to make it easier to work with." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "59cae22f-c85f-43f7-a066-e5cd8ab599ab", + "metadata": {}, + "outputs": [], + "source": [ + "arrow_table = es.data.gnss_observations(\n", + " start_datetime=dt.datetime(2025, 7, 20),\n", + " end_datetime=dt.datetime(2025, 9, 20),\n", + " station_name=\"AC60\",\n", + " session_name=\"A\",\n", + " system=\"G\",\n", + " obs_code=\"1C\",\n", + " satellite=\"7\",\n", + " field=\"snr\",\n", + ").fetch()\n", + "df = arrow_table.to_pandas()\n", + "df.sort_values(by=\"timestamp\")\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "de0b8f0e-523a-4ec9-b3bf-3499310ba9bf", + "metadata": {}, + "source": [ + "The `data.gnss_observations` method is the first of new functions that support cloud-native methods planned for the EarthScope SDK. These functions are more efficient than web services and can support large scale research efforts." + ] + }, + { + "cell_type": "markdown", + "id": "525448ca", + "metadata": {}, + "source": [ + "## Downloading Geodetic Data from GAGE\n", + "\n", + "The GAGE archive holds many types of data ranging from GPS/GNSS data to borehole strain data. We will focus on GPS/GNSS data. Each type of data has API interfaces specific to the data. Unlike dataselect, the API calls return information about data or processed data. The collected data is distributed by a file server and can be programmatically downloaded if you know the URL to the file.\n", + "\n", + "In this example, we will download GNSS data in RINEX. GAGE data is located on a file server and data cab be downloaded with a properly formatted URL. The script downloads the stations by providing the parameters that make up the URL to the data. \n", + "\n", + "### Downloading RINEX files\n", + "\n", + "The GAGE base URL for gnss data in RINEX is `https://gage-data.earthscope.org/archive/gnss/rinex/obs/`. \n", + "\n", + "Files are organized by year and the day of the year, e.g., `/2025/001/`. File names use this pattern: \n", + "\n", + "| station | day of year | 0. | two digit year | o.Z or d.Z |\n", + "|---------|-------------|----|----------------|-----|\n", + "| p034 | 001 |0. | 25 | d.Z |\n", + "| p034 | 001 |0. | 25 | o.Z |\n", + "\n", + "The complete URL for this RINEX file:\n", + "\n", + "`https://gage-data.earthscope.org/archive/gnss/rinex/obs/2025/001/p0340010.25d.Z`\n", + "\n", + "> Note: files ending with `d.Z` are [hatanaka compressed files](https://www.unavco.org/data/gps-gnss/hatanaka/hatanaka.html) and files ending with `o.Z` are not hatanaka compressed. Hatanaka compressed files are much smaller but require software to read the data.\n", + "\n", + "The same method for downloading SAGE data can be used to download GAGE data once URL is properly constructed." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "930d394b", + "metadata": {}, + "outputs": [], + "source": [ + "import requests, os\n", + "from pathlib import Path\n", + "from earthscope_sdk import EarthScopeClient\n", + "\n", + "client = EarthScopeClient()\n", + "\n", + "BASE_URL= 'https://gage-data.earthscope.org/archive/gnss/rinex/obs/'\n", + "\n", + "# function to get authorization token \n", + "def get_token():\n", + " \n", + " # refresh the token if it has expired\n", + " client.ctx.auth_flow.refresh_if_necessary()\n", + "\n", + " token = client.ctx.auth_flow.access_token\n", + " \n", + " return token\n", + "\n", + "# function to download data from GAGE archive\n", + "def download_file(url, data_directory):\n", + " \n", + " # get authorization Bearer token\n", + " token = get_token()\n", + "\n", + " # the pathlib package (https://docs.python.org/3/library/pathlib.html#accessing-individual-parts) \n", + " # supports extracting the file name from the end of a path\n", + " file_name = Path(url).name\n", + " \n", + " # request a file and provide the token in the Authorization header\n", + " r = requests.get(url, headers={\"authorization\": f\"Bearer {token}\"}, stream=True)\n", + " if r.status_code == requests.codes.ok:\n", + " # save the file\n", + " with open(Path(Path(data_directory) / file_name), 'wb') as f:\n", + " for data in r:\n", + " f.write(data)\n", + " else:\n", + " #problem occured\n", + " print(f\"failure: {r.status_code}, {r.reason}\")\n", + "\n", + "# function to creat URL to download data\n", + "def create_url(year, day, station, compression):\n", + " # using Python string formatting and slicing\n", + " doy = \"{:03d}\".format(day) # converts day to a three character zero padded string , '001'\n", + " two_digit_year = str(year)[2:] # converts integer to string and slices the last characters\n", + "\n", + " # using the Python join method to concatenate an array or list of strings\n", + " file_path = '/'.join([str(year), doy]) # integer year converted to string for string join\n", + " file_name = ''.join(['/', station, doy, '0.', two_digit_year, compression])\n", + " url = ''.join([BASE_URL,file_path,file_name])\n", + "\n", + " return url\n", + "\n", + "# create a directory for rinex data\n", + "directory_path = \"./rinex_data\"\n", + "os.makedirs(directory_path, exist_ok=True)\n", + "\n", + "# data requested from station p034 on January 1, 2025 hatanaka compressed\n", + "year = 2025\n", + "day = 1\n", + "station = 'p034'\n", + "compression = 'd.Z'\n", + "\n", + "# download the RINEX file\n", + "url = create_url(year, day, station, compression)\n", + "download_file(url, directory_path)" + ] + }, + { + "cell_type": "markdown", + "id": "7297d085", + "metadata": {}, + "source": [ + "In this example, we’ve added a function to create a URL to the data. The URL can be constructed more succinctly. However, the example demonstrates how to format parameters using Python string functions and how to join strings to form a URL.\n", + "A more succinct method for building the URL would be to use string formatting and add the following code to the download_file function, along with the required parameters. However, this is less explicit and illustrative.\n", + "\n", + "```python\n", + "doy = '{%03d}'.format(day)\n", + "two_digit_year = str(year)[2:]\n", + "url='https://gage-data.earthscope.org/archive/gnss/rinex/obs/{}/{}/{}{}.{}d.Z'.format(year,doy,station,doy,two_digit_year)\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3c1a43fc-6aa3-4f73-b2f5-9860b44085da", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "The EarthScope SDK supports selectively pulling GNSS observation data instead of pulling all the data from RINEX that covers the requested time period. The SDK returns the data in Apache Arrow, which is an efficient method for transporting data and converting it to a scientific computing data format such as Pandas dataframes. \n", + "\n", + "Other geodetic data are available by downloading their respective files." + ] + }, + { + "cell_type": "markdown", + "id": "52369657-336d-4d13-b9dc-508f8fb146fc", + "metadata": {}, + "source": [ + "## [< Previous](./5_seismic_exercises.ipynb)             [Next >](./7_geodetic_exercise.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "geolab", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.14.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/7_geodetic_exercise.ipynb b/docs/content/tutorials/fundamentals/7_geodetic_exercise.ipynb new file mode 100644 index 0000000..5da029d --- /dev/null +++ b/docs/content/tutorials/fundamentals/7_geodetic_exercise.ipynb @@ -0,0 +1,340 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5c92db8c-ae5a-4ce8-85cb-d89a62a5ded7", + "metadata": {}, + "source": [ + "# Geodetic Data Exercise" + ] + }, + { + "cell_type": "markdown", + "id": "1fd237c1-2f6c-4026-a747-4334f3d2a2aa", + "metadata": {}, + "source": [ + "## Exercise A: Get Geodetic Data with the EarthScope SDK\n", + "\n", + "Using the EarthScope SDK, stream data GNSS observations data to a Pandas dataframe and show the observables." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1235e5f8", + "metadata": {}, + "outputs": [], + "source": [ + "# --- Fill in the blanks (marked ____ ) ---\n", + "import datetime as dt\n", + "import pandas as pd\n", + "from earthscope_sdk import EarthScopeClient\n", + "\n", + "# 1) Create the client\n", + "es = ____()\n", + "\n", + "# 2) Define a start and end datetime (UTC)\n", + "start_dt = dt.datetime(____, ____, ____, ____) # year, month, day, hour\n", + "end_dt = dt.datetime(____, ____, ____, ____) # ~30 hours later\n", + "\n", + "# 3) Choose a station (4-character ID)\n", + "station = \"____\"\n", + "\n", + "# 4) Make the request for GNSS observations and fetch as an Arrow table\n", + "arrow_table = es.data.gnss_observations(\n", + " start_datetime=____,\n", + " end_datetime=____,\n", + " station_name=____,\n", + ").____() # method that actually runs the request\n", + "\n", + "# 5) Convert Arrow ➜ pandas DataFrame and view the first few rows\n", + "df = arrow_table.____()\n", + "print(df.head())\n", + "\n", + "# 6) (bonus) How many rows did we get?\n", + "print(\"rows:\", len(df))\n", + "\n", + "# 7) (bonus) Show unique observation types if present (e.g., L1, L2, C1...)\n", + "if \"observable\" in df.columns:\n", + " print(\"observables:\", sorted(df[\"observable\"].unique()))" + ] + }, + { + "cell_type": "markdown", + "id": "05aeec36", + "metadata": {}, + "source": [ + "## Exercise B: Download a RINEX file" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e6a85520", + "metadata": {}, + "outputs": [], + "source": [ + "# ---------- Fill in the package imports for this script ----------\n", + "import ____ \n", + "from datetime import datetime\n", + "from pathlib import Path\n", + "import ____\n", + "\n", + "# ---------- Create an EarthScope client to get token ----------\n", + "client = ____\n", + "\n", + "\n", + "# Fill this with the service endpoint for GAGE data\n", + "GAGE_URL = ____\n", + "\n", + "DATA_DIR = Path(\"./data\")\n", + "DATA_DIR.mkdir(parents=True, exist_ok=True)\n", + "\n", + "# ---------- Get an EarthScope token ----------\n", + "def get_token():\n", + " \"\"\"\n", + " Use the SDK to refresh (if needed) and return an access token.\n", + " \"\"\"\n", + " # Refresh if necessary\n", + " ____\n", + "\n", + " # Retrieve the token string\n", + " token = ____\n", + " return token\n", + "\n", + "\n", + "# ---------- Helper: common auth header ----------\n", + "def auth_header(token):\n", + " \"\"\"\n", + " Helper function to create header with token\n", + " \"\"\"\n", + " # Use a standard Bearer token header (lowercase 'authorization' is fine)\n", + " return ____\n", + "\n", + "\n", + "# ---------- Download a file from EarthScope's web services ----------\n", + "# --------------------------------------------------------------------\n", + "def download_data(url, data_directory): # params is an optional query parameter\n", + " \"\"\"\n", + " Sends GET request to GAGE web service by construcing the URL to the RINEX file.\n", + " \"\"\"\n", + " # get authorization token\n", + " token = get_token()\n", + "\n", + " # create the authorization header \n", + " header = auth_header(token) \n", + "\n", + "\n", + " # This creates a file name for a RINEX file.\n", + " file_name = Path(url).name\n", + "\n", + " # create a variable with the path to the data directory and file\n", + " out_path = Path(Path(data_directory) / ____)\n", + "\n", + " # Make the request to the web service with params and bearer auth\n", + " r = requests.____(\n", + " url,\n", + " params=____,\n", + " headers=____,\n", + " stream=True\n", + " ) \n", + "\n", + " if r.status_code == requests.codes.____:\n", + " with open(out_path, \"wb\") as f:\n", + " for data in r:\n", + " f.write(data)\n", + " else:\n", + " #problem occured\n", + " print(f\"failure: {r.status_code}, {r.reason}\")\n", + " return None\n", + "\n", + "\n", + "# ---------- Creates URL to download data from the EarthScope GAGE web service ----------\n", + "# ---------------------------------------------------------------------------------------\n", + "def create_url(year, day, station, compression):\n", + " \"\"\"\n", + " Construct a URL to a file in an archive given year, day-of-year, station, etc.\n", + " Example output:\n", + " {BASE_URL}{year}/{DOY}/{/STATIONDOY0.YY}{compression}\n", + " \"\"\"\n", + " doy = \"{:03d}\".____(day)\n", + " two_digit_year = ____ # Hint: use the string slice function in the previous notebook\n", + "\n", + " file_path = \"/\".join([str(year), doy])\n", + " file_name = \"\".join([\"/\", station, doy, \"0.\", two_digit_year, compression])\n", + " url = \"\".join([GAGE_URL, file_path, file_name])\n", + " return url\n", + "\n", + "# Download a RINEX file\n", + "#\n", + "year = 2025\n", + "day = 1\n", + "station = 'p034'\n", + "doy = '%03d'.format(day)\n", + "compression = 'd.Z' # or \".Z\" / \"\" depending on the archive\n", + "url = create_url(year, ____, station, compression)\n", + "download_data(url, ____)" + ] + }, + { + "cell_type": "markdown", + "id": "d27ee485-f1c9-48ea-8cc3-819df2d1ecf9", + "metadata": {}, + "source": [ + "## Answer Key for Exercise A\n", + "\n", + "```{admonition} Click to see answer\n", + ":class: dropdown\n", + "\n", + "
\n",
+    "import datetime as dt\n",
+    "import pandas as pd\n",
+    "from earthscope_sdk import EarthScopeClient\n",
+    "\n",
+    "es = EarthScopeClient()\n",
+    "\n",
+    "start_dt = dt.datetime(2025, 7, 20, 21)\n",
+    "end_dt   = dt.datetime(2025, 7, 22, 3)\n",
+    "station = \"AC60\"\n",
+    "\n",
+    "arrow_table = es.data.gnss_observations(\n",
+    "    start_datetime=start_dt,\n",
+    "    end_datetime=end_dt,\n",
+    "    station_name=station,\n",
+    ").fetch()\n",
+    "\n",
+    "df = arrow_table.to_pandas()\n",
+    "print(df.head())\n",
+    "print(\"rows:\", len(df))\n",
+    "\n",
+    "if \"observable\" in df.columns:\n",
+    "    print(\"observables:\", sorted(df[\"observable\"].unique()))\n",
+    "\n",
+    "
\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "id": "9113f9a0", + "metadata": {}, + "source": [ + "## Answer Key for Exercise B\n", + "\n", + "```{admonition} Click to see answer\n", + ":class: dropdown\n", + "\n", + "
\n",
+    "# ---------- Imports ----------\n",
+    "import requests\n",
+    "import os\n",
+    "from datetime import datetime\n",
+    "from pathlib import Path\n",
+    "from earthscope_sdk import EarthScopeClient\n",
+    "\n",
+    "# ---------- Create an EarthScope client to get token ----------\n",
+    "client = EarthScopeClient()\n",
+    "\n",
+    "# Fill this with the service endpoint for SAGE data\n",
+    "GAGE_URL = \"https://gage-data.earthscope.org/archive/gnss/rinex/obs/?\"\n",
+    "\n",
+    "# create a directory for RINEX data\n",
+    "DATA_DIR = \"./data\"\n",
+    "os.makedirs(DATA_DIR, exist_ok=True)\n",
+    "\n",
+    "# ---------- Get an EarthScope token ----------\n",
+    "def get_token():\n",
+    "    \"\"\"\n",
+    "    Use the SDK to refresh (if needed) and return an access token.\n",
+    "    \"\"\"\n",
+    "    # Refresh if necessary\n",
+    "    client.ctx.auth_flow.refresh_if_necessary()\n",
+    "\n",
+    "    # Retrieve the token string\n",
+    "    token = client.ctx.auth_flow.access_token\n",
+    "    return token\n",
+    "\n",
+    "\n",
+    "# ---------- Helper: common auth header ----------\n",
+    "def auth_header(token):\n",
+    "    # Use a standard Bearer token header (lowercase 'authorization' is fine)\n",
+    "    return {\"authorization\": f\"Bearer {token}\"}\n",
+    "\n",
+    "\n",
+    "# ---------- Download a file from EarthScope's web services ----------\n",
+    "# --------------------------------------------------------------------\n",
+    "def download_data(url, data_directory):\n",
+    "    \"\"\"\n",
+    "    Sends GET with query parameters and saves the response body to a file.\n",
+    "    \"\"\"\n",
+    "    # get authorization token\n",
+    "    token = get_token()\n",
+    "\n",
+    "    # create the authorization header \n",
+    "    header = auth_header(token) \n",
+    "    \n",
+    "    # Create file name \n",
+    "    file_name = Path(url).name\n",
+    "\n",
+    "    out_path = Path(Path(data_directory) / file_name)\n",
+    "\n",
+    "    # Make the request with params and bearer auth\n",
+    "    r = requests.get(\n",
+    "        url,\n",
+    "        params=params,\n",
+    "        headers=header,\n",
+    "        stream=True\n",
+    "    ) \n",
+    "\n",
+    "    if r.status_code == requests.codes.ok:\n",
+    "        with open(out_path, \"wb\") as f:\n",
+    "            for data in r:\n",
+    "                f.write(data)\n",
+    "    else:\n",
+    "        #problem occurred\n",
+    "        print(f\"failure: {r.status_code}, {r.reason}\")\n",
+    "        return None\n",
+    "\n",
+    "# Try the URL-building flow\n",
+    "year = 2025\n",
+    "day = 1\n",
+    "station = 'p034'\n",
+    "doy = '%03d'.format(day)\n",
+    "compression = 'd.Z'  # or \".Z\" / \"\" depending on the archive\n",
+    "url = create_url(year, day, station, compression)\n",
+    "download_data(url, DATA_DIR)\n",
+    "
\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "55674a9d-5814-4a6a-beab-98dc0547ded7", + "metadata": {}, + "source": [ + "## [< Previous](./6_geodetic_data.ipynb)             [Next >](./8_wrap_up.ipynb)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/8_wrap_up.ipynb b/docs/content/tutorials/fundamentals/8_wrap_up.ipynb new file mode 100644 index 0000000..a0e7129 --- /dev/null +++ b/docs/content/tutorials/fundamentals/8_wrap_up.ipynb @@ -0,0 +1,59 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3154392e-b584-445c-a6cb-b9dfed447a7f", + "metadata": {}, + "source": [ + "# Putting It All Together\n", + "\n", + "This project covers how to create a project in GeoLab. It's best practice to structure your project using git for several reasons. First, git tracks your code and saves it at different states. It enables you to roll back to working code. Second, you can create branches to test new features. Finally, you can publish and share your project code by publishing it to a public repository, such as GitHub.\n", + "\n", + "GeoLab is a Python environment that contains many tools for geoscience. However, it's another best practice to create a project-specific virtual environment. GeoLab uses conda as a package manager, and you can clone the base GeoLab virtual environment and add project-specific packages as needed. Conda creates virtual environments and resolves package dependencies and conflicts, ensuring that projects can be deployed on a variety of platforms.\n", + "\n", + "You will need an EarthScope account to access GeoLab. Once in GeoLab, accessing EarthScope data requires an authorization token, which can be requested with the EarthScope CLI or programmatically with the EarthScope SDK. The token enables requesting data from EarthScope.\n", + "\n", + "There are several ways to acquire data from EarthScope. Data is available through web services and through cloud services. The primary distinction between web and cloud services is that web services are used to download data as files in GeoLab. In addition to the computer I/O overhead of writing a file, there is additional overhead to read files for processing and analysis. In contrast, cloud services stream data directly to memory, where it is immediately available for processing and analysis.\n", + "\n", + "The future of GeoLab and EarthScope data distribution is cloud services. Web services will remain available as EarthScope makes more data available through the cloud." + ] + }, + { + "cell_type": "markdown", + "id": "2a7a6216-0058-40ca-9586-448c71a4c6d4", + "metadata": {}, + "source": [ + "## [< Previous](./7_cloud_native_exercise.ipynb)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2d626393-6414-4f45-b275-f84b090b7cdd", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/content/tutorials/fundamentals/images/Web_Services_Data_Flow.png b/docs/content/tutorials/fundamentals/images/Web_Services_Data_Flow.png new file mode 100644 index 0000000..aba035a Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/Web_Services_Data_Flow.png differ diff --git a/docs/content/tutorials/fundamentals/images/cloud_native_data_access.png b/docs/content/tutorials/fundamentals/images/cloud_native_data_access.png new file mode 100644 index 0000000..6883ef6 Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/cloud_native_data_access.png differ diff --git a/docs/content/tutorials/fundamentals/images/github_push.png b/docs/content/tutorials/fundamentals/images/github_push.png new file mode 100644 index 0000000..6878c6c Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/github_push.png differ diff --git a/docs/content/tutorials/fundamentals/images/login.png b/docs/content/tutorials/fundamentals/images/login.png new file mode 100644 index 0000000..82e74a4 Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/login.png differ diff --git a/docs/content/tutorials/fundamentals/images/menubar.png b/docs/content/tutorials/fundamentals/images/menubar.png new file mode 100644 index 0000000..c3f9c3b Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/menubar.png differ diff --git a/docs/content/tutorials/fundamentals/images/project_work_flow.png b/docs/content/tutorials/fundamentals/images/project_work_flow.png new file mode 100644 index 0000000..2eaf913 Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/project_work_flow.png differ diff --git a/docs/content/tutorials/fundamentals/images/toolbar.png b/docs/content/tutorials/fundamentals/images/toolbar.png new file mode 100644 index 0000000..b0ddcfa Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/toolbar.png differ diff --git a/docs/content/tutorials/fundamentals/images/workspace.png b/docs/content/tutorials/fundamentals/images/workspace.png new file mode 100644 index 0000000..55042d7 Binary files /dev/null and b/docs/content/tutorials/fundamentals/images/workspace.png differ diff --git a/docs/content/tutorials/fundamentals/questions/authorization.json b/docs/content/tutorials/fundamentals/questions/authorization.json new file mode 100644 index 0000000..74b6537 --- /dev/null +++ b/docs/content/tutorials/fundamentals/questions/authorization.json @@ -0,0 +1,69 @@ +[ + { + "question": "What are the ways to get an authorization token?", + "type": "many_choice", + "answers": [ + { + "answer": "Use the EarthScope CLI.", + "correct": true, + "feedback": "Correct, you can use the EarthScope CLI can request an authorization token." + }, + { + "answer": "Programatically with the EarthScope SDK.", + "correct": true, + "feedback": "Correct, the Python package, earthscope-sdk, can request an authorization token." + }, + { + "answer": "Email EarthScope to request an token.", + "correct": false, + "feedback": "Incorrect, EarthScope does not issue tokens by email." + }, + { + "answer": "The token is optional.", + "correct": false, + "feedback": "Incorrect, an authorization token is needed to request data from EarthScope services." + }, + { + "answer": "You can generate a token using programs, such as ssh-keygen.", + "correct": false, + "feedback": "Incorrect, the token must come from EarthScope and cannot be create locally." + } + ] + }, + { + "question": "Where is the token stored in GeoLab?", + "type": "string", + "answers": [ + { + "answer": "/home/jovyan/.earthscope/default", + "correct": true, + "feedback": "Correct. The token is stored in a hidden directory in the home directory.", + "match_case": true, + "fuzzy_threshold": 1.0 + }, + { + "answer": "$HOME/.earthscope/default", + "correct": true, + "feedback": "Correct. The token is stored in a hidden directory. You can use the HOME environment variable instead of the full path.", + "match_case": true, + "fuzzy_threshold": 1.0 + } + ] + }, + { + "question": "For all current and future EarthScope data services, an authorization token is necessary", + "type": "multiple_choice", + "answers": [ + { + "answer": "True", + "correct": false, + "feedback": "Incorrect, the AWS Open Data program does not require a token." + }, + { + "answer": "False", + "correct": true, + "feedback": "Correct, the AWS Open Data program does not require a token. " + } + ] + } +] \ No newline at end of file diff --git a/docs/content/tutorials/fundamentals/questions/branches.json b/docs/content/tutorials/fundamentals/questions/branches.json new file mode 100644 index 0000000..b9e709a --- /dev/null +++ b/docs/content/tutorials/fundamentals/questions/branches.json @@ -0,0 +1,98 @@ +[ + { + "question": "Select all the ways to create a new branch.", + "type": "many_choice", + "answers": [ + { + "answer": "git branch new_feature", + "correct": true, + "feedback": "Correct, a new branch is created but you remain in the current branch." + }, + { + "answer": "git checkout -b new_feature", + "correct": true, + "feedback": "Correct, creates an switches to a new branch." + }, + { + "answer": "git branch --name new_feature", + "correct": false, + "feedback": "Incorrect, --name is not an option." + }, + { + "answer": "git checkout new_feature", + "correct": false, + "feedback": "Incorrect, this switches to an existing branch." + }, + { + "answer": "git branch main ", + "correct": false, + "feedback": "Incorrect, the main branch is created after the first commit." + } + ] + }, + { + "question": "This command, git branch -d new_feature, deletes an unmerged branch.", + "type": "multiple_choice", + "answers": [ + { + "answer": "False", + "correct": false, + "feedback": "Incorrect, -d is a safe delete that works on a branch that has been merged." + }, + { + "answer": "True", + "correct": true, + "feedback": "Correct, -d is a safe delete that works on a branch that has been merged." + } + ] + }, + { + "question": "When merging the new_feature branch into main, which branch should you check out?", + "type": "multiple_choice", + "answers": [ + { + "answer": "main", + "correct": true, + "feedback": "Correct, git merges into the target branch which is main in this example." + }, + { + "answer": "new_feature", + "correct": false, + "feedback": "Incorrect, git merges into the targe branch." + } + ] + }, + { + "question": "How do you list all the branches in a project", + "type": "multiple_choice", + "answers": [ + { + "answer": "git branch -a", + "correct": true, + "feedback": "Correct, -a is an optional flag for listing all branches." + }, + { + "answer": "git branches -a", + "correct": false, + "feedback": "Incorrect, branches is not an option." + }, + { + "answer": "git branches", + "correct": false, + "feedback": "Incorrect, branches is not an option." + }, + { + "answer": "git branch", + "correct": true, + "feedback": "Correct, all branches are listed. " + }, + { + "answer": "git list branch", + "correct": false, + "feedback": "Incorrect, list is not command." + } + ] + } + + +] \ No newline at end of file diff --git a/docs/content/tutorials/fundamentals/questions/create_clone.json b/docs/content/tutorials/fundamentals/questions/create_clone.json new file mode 100644 index 0000000..cfc4943 --- /dev/null +++ b/docs/content/tutorials/fundamentals/questions/create_clone.json @@ -0,0 +1,127 @@ +[ + { + "question": "How do you copy a remote repository to work on it locally." , + "type": "many_choice", + "answers": [ + { + "answer": "git clone https://github.com/anu/gnss_tools.git", + "correct": true, + "feedback": "Correct, clones a repository using the HTTPS protocol." + }, + { + "answer": "git clone git@github.com:anu/gnss_tools.git", + "correct": true, + "feedback": "Correct, clones a repository using SSH protocol." + }, + { + "answer": "git clone -b new_feature https://github.com/anu/gnss_tools.git", + "correct": true, + "feedback": "Correct, the -b flag clones a specific branch." + }, + { + "answer": "git clone https://github.com/anu/gnss_tools", + "correct": false, + "feedback": "Incorrect, the URL is incomplete without a .git extension." + }, + { + "answer": "git clone https://github.com/anu/gnss_tools.git -b latest", + "correct": false, + "feedback": "Incorrect, command flags, -b, go before the URL." + } + ] + }, + { + "question": "What is the first thing to do when creating a repository?" , + "type": "multiple_choice", + "answers": [ + { + "answer": "mkdir project_name", + "correct": true, + "feedback": "Correct." + }, + { + "answer": "git init", + "correct": false, + "feedback": "You must be in the project directory to create a repository." + }, + { + "answer": "git status", + "correct": false, + "feedback": "The repository has to be created before checking status." + }, + { + "answer": "git add .", + "correct": false, + "feedback": "The repository must be created before addin files." + }, + { + "answer": "cd project_name", + "correct": false, + "feedback": "You must create the project directory first." + } + ] + + }, + { + "question": "Select the commands to add files to the repository.", + "type": "many_choice", + "answers": [ + { + "answer": "git add .", + "correct": true, + "feedback": "Correct, you can add multiple files." + }, + { + "answer": "git add file_name", + "correct": true, + "feedback": "Correct, you can add a single file." + }, + { + "answer": "git put file_name", + "correct": false, + "feedback": "Incorrect, put is not a command." + }, + { + "answer": "git install file_name", + "correct": false, + "feedback": "Incorrect, install is not a command." + }, + { + "answer": "git put .", + "correct": false, + "feedback": "Incorrect, put is not a command." + } + ] + }, + { + "question": "What does `git init` do? Select all that apply.", + "type" : "many_choice", + "answers": [ + { + "answer": "Create a .git directory", + "correct": true, + "feedback": "Correct." + }, + { + "answer": "Create files and directories for tracking project files.", + "correct": true, + "feedback": "Correct." + }, + { + "answer": "Creates a remote repository on GitHub.", + "correct": false, + "feedback": "It creates a local repository on your computer." + }, + { + "answer": "Configures your user name and email.", + "correct": false, + "feedback": "git config sets these parameters" + }, + { + "answer": "Adds existing files to the repository.", + "correct": false, + "feedback": "git add is the command to add files to the repository." + } + ] + } +] \ No newline at end of file diff --git a/docs/content/tutorials/fundamentals/questions/push_pull.json b/docs/content/tutorials/fundamentals/questions/push_pull.json new file mode 100644 index 0000000..629bda6 --- /dev/null +++ b/docs/content/tutorials/fundamentals/questions/push_pull.json @@ -0,0 +1,50 @@ +[ + { + "question": "When you pull from a repository, what two actions happen?", + "type": "many_choice", + "answers": [ + { + "answer": "git fetch", + "correct": true, + "feedback": "Correct, pull fetches the files from the remote repository." + }, + { + "answer": "git merge", + "correct": true, + "feedback": "Correct, files are merged into the local repository when fetched from the remote." + }, + { + "answer": "git branch", + "correct": false, + "feedback": "Incorrect, a pull request remains in the same branch," + }, + { + "answer": "git commit", + "correct": false, + "feedback": "Incorrect, local files are only commited with the commit command." + }, + { + "answer": "git add", + "correct": false, + "feedback": "Incorrect, local files are added only with the add command." + } + + ] + }, + { + "question": "git push -u origin main, makes code on a public repository available to others.", + "type": "multiple_choice", + "answers": [ + { + "answer": "true", + "correct": true, + "feedback": "Correct, push copies commits from the local repository to the remote repository." + }, + { + "answer": "false", + "correct": false, + "feedback": "Incorrect, code is available on public repository. " + } + ] + } +] \ No newline at end of file diff --git a/docs/content/tutorials/fundamentals/questions/working_with_files.json b/docs/content/tutorials/fundamentals/questions/working_with_files.json new file mode 100644 index 0000000..b844cbc --- /dev/null +++ b/docs/content/tutorials/fundamentals/questions/working_with_files.json @@ -0,0 +1,89 @@ +[ + { + "question": "How many characters should be used to refer to a commit?" , + "type": "numeric", + "answers": [ + { + "type": "range", + "range": [8, 10], + "correct": true, + "feedback": "A minimum of the first 8 to 10 characters is sufficiently unique." + }, + { + "type": "range", + "range": [1, 7], + "correct": false, + "feedback": "Too few characters to guarantee uniqueness." + }, + { + "type": "range", + "range": [11, 100000000], + "correct": true, + "feedback": "This is will work but it is unnecessarily large." + } + ] + + }, + { + "question": "What happens when you commit?" , + "type": "multiple_choice", + "answers": [ + { + "answer": "The project is saved at the point in time.", + "correct": true, + "feedback": "Correct." + }, + { + "answer": "Code is added to the repository and tracked.", + "correct": false, + "feedback": "git add tracks files in the repository." + }, + { + "answer": "The project is saved in GitHub.", + "correct": false, + "feedback": "git push saves a project to GitHub." + }, + { + "answer": "Previous code is deleted.", + "correct": false, + "feedback": "Commits save code at a point in time, previous code is not deleted." + }, + { + "answer": "All deleted files are permanently removed from the repository.", + "correct": false, + "feedback": "Deleted files are retained in previous commits." + } + ] + }, + { + "question": "After adding to the README file, you decide that the change was unnecessary. What do you do to return to the previous README?", + "type": "multiple_choice", + "answers": [ + { + "answer": "git revert add27a78368", + "correct": true, + "feedback": "git revert returns a file to previous commit." + }, + { + "answer": "git rm README.md", + "correct": false, + "feedback": "This deletes the README file." + }, + { + "answer": "Edit the README and git add and commit.", + "correct": true, + "feedback": "This is works but is inefficient." + }, + { + "answer": "git fix add27a78368", + "correct": false, + "feedback": "git fix is not a command." + }, + { + "answer": "git reset add27a78368", + "correct": false, + "feedback": "git reset deletes commits after and including the commit " + } + ] + } +] \ No newline at end of file diff --git a/docs/content/tutorials/tutorials.md b/docs/content/tutorials/tutorials.md new file mode 100644 index 0000000..753169d --- /dev/null +++ b/docs/content/tutorials/tutorials.md @@ -0,0 +1,25 @@ +# GeoLab Tutorial Index + +## Fundamentals + +[Project Overview](./fundamentals/0_project_overview.ipynb) + +[Introduction to Git](./fundamentals/1_git_intro.ipynb) + +[Python Environment and Package Management](./fundamentals/2_package_management.ipynb) + +[EarthScope Authorization](./fundamentals/3_authorization.ipynb) + +[EarthScope Web Services](./fundamentals/4_web_services_data.ipynb) + +[Web Services Exercise](./fundamentals/5_web_services_exercise.ipynb) + +[Cloud Native Data Services](./fundamentals/6_cloud_native_data.ipynb) + +[Cloud Services Exercise](./fundamentals/7_cloud_native_exercise.ipynb) + +[Summary and Wrap Up](./fundamentals/8_wrap_up.ipynb) + +## Advanced Topics + +TBD \ No newline at end of file