accessibility and updates for containers workshop

content/notes/containers-for-hpc/_index.md

@@ -1,14 +1,10 @@
 ---
 title: "Software Containers for HPC"
-summary: "An Introduction to using and building software containers."
 authors: [rs]
-categories: ["Containers","HPC"]
-tags: [Containers, HPC]
 toc: true
 type: docs
 date: "2025-05-15T00:00:00"
 weight: 1
-
 menu:
     containers-for-hpc:
         name: Software Containers for HPC

content/notes/containers-for-hpc/building-apptainer.md

@@ -4,9 +4,10 @@ title: "Building Containers on HPC [Apptainer]"
 toc: true
 type: docs
 weight: 4
+
 menu:
     containers-for-hpc:
-        parent: Software Containers for HPC
+       parent: Software Containers for HPC
 
 ---
 
@@ -114,7 +115,7 @@ This is the very first entry. It defines the bootstrap agent:
 - `docker`
 - `library`
 - `shub`
-- and [many more](https://apptainer.org/docs/user/latest/definition_files.html#preferred-bootstrap-agents)
+- and [many more (opens in new tab)](https://apptainer.org/docs/user/latest/definition_files.html#preferred-bootstrap-agents)
 
 #### `From` (mandatory)
 Define the base container.
@@ -342,7 +343,7 @@ Save this as `lolcow_2.def`.
 - You may need to specify extra packages
     - `fortune` itself provides the executable without the message database
     - `fortunes-min` contains the message database
-- See [how Ubuntu reduced image size by 60%](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends)
+- See [how Ubuntu reduced image size by 60% (opens in new tab)](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends)
 
 ### Image size comparison
 
@@ -447,21 +448,21 @@ $ apptainer push lolcow_0.sif oras://myname/lolcow:0
 ```
 
 Check:
-`https://hub.docker.com/r/myname/lolcow/tags`
+[Docker Hub tags for `myname/lolcow` (opens in new tab)](https://hub.docker.com/r/myname/lolcow/tags)
 
-{{< info >}}Best practice: Sign your containers; see [here](https://apptainer.org/docs/user/latest/signNverify.html).{{< /info >}}
+{{< info >}}Best practice: Sign your containers; see [Apptainer signing and verification documentation (opens in new tab)](https://apptainer.org/docs/user/latest/signNverify.html).{{< /info >}}
 
 {{< warning >}}While Apptainer can convert a Docker image into SIF, you cannot run SIF with Docker. You are simply using Docker Hub to host your SIF - it is not converted into Docker.{{< /warning >}}
 
 #### UVA Research Computing container resources
 
-[Docker Hub account](https://hub.docker.com/u/uvarc)
+[Docker Hub account (opens in new tab)](https://hub.docker.com/u/uvarc)
 
-[Repository of Dockerfiles and definition files](https://github.com/uvarc/rivanna-docker)
+[Repository of Dockerfiles and definition files (opens in new tab)](https://github.com/uvarc/rivanna-docker)
 
 ### GitHub Packages
 
-1. [Create a personal access token](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic)
+1. [Create a personal access token (opens in new tab)](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic)
 1. Login
     ```bash
     apptainer remote login --username myname docker://ghcr.io
@@ -471,7 +472,7 @@ Check:
     ```bash
     apptainer push lolcow_0.sif oras://ghcr.io/myname/lolcow
     ```
-1. Check `https://github.com/users/myname/packages/container/package/lolcow`
+1. Check [your GitHub package page (opens in new tab)](https://github.com/users/myname/packages/container/package/lolcow)
 
 ## Case Studies
 
@@ -487,16 +488,16 @@ While you can create a conda environment locally, you cannot directly migrate it
 Your project requires PyTorch, Numpy, Seaborn, and Pandas. Write the corresponding Apptainer definition file.
 
 Hints:
-- Find the appropriate base image from [here](https://hub.docker.com/r/pytorch/pytorch/tags).
+- Find the appropriate base image from [PyTorch Docker image tags on Docker Hub (opens in new tab)](https://hub.docker.com/r/pytorch/pytorch/tags).
 - Pull the base image and examine it first. Does it already provide some packages?
 
 {{< info >}}While PyTorch runs on a GPU, you do not need to build the container on a GPU.{{< /info >}}
 
-{{< info >}}You will likely run out of memory when building large containers (over a few GBs). Request an [interactive job](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#submitting-an-interactive-job) with say `--mem=50G` to build on a compute node with 50 GB memory.{{< /info >}}
+{{< info >}}You will likely run out of memory when building large containers (over a few GBs). Request an [interactive job (opens in new tab)](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#submitting-an-interactive-job) with say `--mem=50G` to build on a compute node with 50 GB memory.{{< /info >}}
 
 ### R
 
-Rocker provides many base images for all R versions (see [here](https://rocker-project.org/images/)):
+Rocker provides many base images for all R versions (see [Rocker image catalog (opens in new tab)](https://rocker-project.org/images/)):
 - `rocker/r-ver`: basic R installation
 - `rocker/rstudio`: with RStudio Server
 - `rocker/tidyverse`: plus tidyverse and dependencies
@@ -515,14 +516,14 @@ Hints:
 
 ## Multistage Build
 
-By distinguishing between buildtime-dependencies vs runtime-dependencies, it is possible to reduce the image size drastically via a [multistage build](https://apptainer.org/docs/user/latest/definition_files.html#multi-stage-builds). (My experience with some extreme cases is that only 1% is needed at runtime.)
+By distinguishing between buildtime-dependencies vs runtime-dependencies, it is possible to reduce the image size drastically via a [multistage build (opens in new tab)](https://apptainer.org/docs/user/latest/definition_files.html#multi-stage-builds). (My experience with some extreme cases is that only 1% is needed at runtime.)
 
 This is beyond the scope of the workshop, but you are welcome to browse the Apptainer documentation and Appendix 2 on Minimal Containers.
 
 ---
 
 ## References
-- [Apptainer User Guide](https://apptainer.org/docs/user/latest)
-- [Definition File](https://apptainer.org/docs/user/latest/definition_files.html)
-- [Definition File vs Dockerfile](https://apptainer.org/docs/user/latest/docker_and_oci.html#apptainer-definition-file-vs-dockerfile)
-- [GitHub Packages](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry)
+- [Apptainer User Guide (opens in new tab)](https://apptainer.org/docs/user/latest)
+- [Definition File (opens in new tab)](https://apptainer.org/docs/user/latest/definition_files.html)
+- [Definition File vs Dockerfile (opens in new tab)](https://apptainer.org/docs/user/latest/docker_and_oci.html#apptainer-definition-file-vs-dockerfile)
+- [GitHub Packages (opens in new tab)](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry)

content/notes/containers-for-hpc/building-docker.md

@@ -21,7 +21,7 @@ Starting from a simple Dockerfile, we will adopt best practices sequentially and
 
 2. Install Docker
 
-    To follow along on your own computer, please install Docker Desktop and register for a free account on Docker Hub. Both can be found [here](https://www.docker.com/get-started).
+    To follow along on your own computer, please install Docker Desktop and register for a free account on Docker Hub. Both are available from [Docker Get Started (opens in new tab)](https://www.docker.com/get-started).
 
     After the installation, open a terminal ("cmd" on Windows) and make sure you can execute the command `docker run hello-world` successfully.
 
@@ -38,7 +38,7 @@ Starting from a simple Dockerfile, we will adopt best practices sequentially and
 - Singularity
     - Users cannot build on Rivanna (needs `sudo` privilege)
     - Singularity Library/Hub (more limitations)
-    - Refer to [workshop](https://learning.rc.virginia.edu/workshops/singularity/) in Spring 2020
+    - Refer to [workshop (opens in new tab)](https://learning.rc.virginia.edu/workshops/singularity/) in Spring 2020
 
 ## Intro to Dockerfile: lolcow
 
@@ -65,7 +65,7 @@ FROM ubuntu:22.04
 - Doesn't have to be a bare OS
     - `python`, `continuumio/miniconda3`, `node`, `nvidia/cuda`, etc.
 
-[Dockerfile reference](https://docs.docker.com/engine/reference/builder/#format)
+[Dockerfile reference (opens in new tab)](https://docs.docker.com/engine/reference/builder/#format)
 
 ### Steps 2 & 3: Install software
 
@@ -237,14 +237,14 @@ ENTRYPOINT fortune | cowsay | lolcat
 - You may need to specify extra packages
     - `fortune` itself provides the executable without the message database
     - `fortunes-min` contains the message database
-- See [how Ubuntu reduced image size by 60%](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends)
+- See [how Ubuntu reduced image size by 60% (opens in new tab)](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends)
 
 ### 3. Use a smaller base image
 
 For installation of common packages, you may consider Alpine.
 
 - BusyBox + package manager + musl libc (beware of compatibility issues)
-- [Presentation](https://youtu.be/sIG2P9k6EjA) on Alpine Linux from DockerCon EU 17 
+- [Presentation (opens in new tab)](https://youtu.be/sIG2P9k6EjA) on Alpine Linux from DockerCon EU 17 
 
 Look for `slim` variants (e.g. `debian:buster-slim`) of a base image, if any.
 
@@ -281,7 +281,7 @@ $ docker images | grep lolcow | sort -nk 2 | awk '{print $1, $2, $NF}'
 |2  |Combination of previous two |53 | 26 |
 |3  |Alpine base image           |161| 78 |
 
-Reference: [_Best practices for writing Dockerfiles_](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/)
+Reference: [_Best practices for writing Dockerfiles_ (opens in new tab)](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/)
 
 ### Summary
 
@@ -309,11 +309,11 @@ docker push <user>/lolcow:latest
 
 ### Docker Hub interface
 
-In your browser, go to `https://hub.docker.com/r/<user>/lolcow`.
+In your browser, go to [Docker Hub for `<user>/lolcow` (opens in new tab)](https://hub.docker.com/r/<user>/lolcow).
 
 - Overview: 
     - Sync with GitHub to update `README.md`; or
-    - Use [docker-pushrm](https://github.com/christian-korneck/docker-pushrm)
+    - Use [docker-pushrm (opens in new tab)](https://github.com/christian-korneck/docker-pushrm)
 - Tags:
     - List all versions
     - View image history if Dockerfile not provided
@@ -327,31 +327,31 @@ By now, we know how to write a simple Dockerfile to install software using the d
 
 ### Compiled language (C++) 
 
-https://github.com/lilab-bcb/cumulus_feature_barcoding
+[cumulus_feature_barcoding repository (opens in new tab)](https://github.com/lilab-bcb/cumulus_feature_barcoding)
 
 Hints:
 - You do not have to start from a bare OS. Search for `gcc` on Docker Hub.
 - Install `build-essential` if you are starting from a bare OS.
-- Version pinning - to choose a specific version, download from https://github.com/lilab-bcb/cumulus_feature_barcoding/releases (you will need `wget`).
+- Version pinning - to choose a specific version, download from [cumulus_feature_barcoding releases (opens in new tab)](https://github.com/lilab-bcb/cumulus_feature_barcoding/releases) (you will need `wget`).
 
 ### Interpreted language (Python)
 
-https://docs.qiime2.org/2022.8/install/native/#install-qiime-2-within-a-conda-environment
+[QIIME 2 native install guide (opens in new tab)](https://docs.qiime2.org/2022.8/install/native/#install-qiime-2-within-a-conda-environment)
 
 Hints:
 - Click on "Linux" to get the URL for the yaml file. Download the yaml file in the same directory as your Dockerfile.
 - You do not have to start from a bare OS in your Dockerfile. Search for `miniconda3` on Docker Hub.
-- (Recommended) There is a much faster dependency solver than conda - micromamba. See [here](https://micromamba-docker.readthedocs.io/en/latest/quick_start.html) for instructions.
+- (Recommended) There is a much faster dependency solver than conda - micromamba. See the [Micromamba Docker quick start guide (opens in new tab)](https://micromamba-docker.readthedocs.io/en/latest/quick_start.html) for instructions.
 - Use the suggested `COPY` and `ENTRYPOINT` statements.
-- After you're done, compare with the [official Dockerfile](https://github.com/qiime2/vm-playbooks/blob/0fda9dce42802596756986e2f80c38437872c66e/docker/Dockerfile) and image size. What is the biggest reason for the difference?
+- After you're done, compare with the [official Dockerfile (opens in new tab)](https://github.com/qiime2/vm-playbooks/blob/0fda9dce42802596756986e2f80c38437872c66e/docker/Dockerfile) and image size. What is the biggest reason for the difference?
 
 ### General Remarks
 
 - Play with different base images and package managers.
 - If you encounter a Docker statement that you have not used before, first check the official documentation for best practices.
 - A comprehensive list of dependencies may be lacking. Some developers may not specify any at all. You will have to rely on a combination of experience, error message, and web search. (Most likely all of the above.)
 - Especially for Python packages, versions may be too permissive or too restrictive such that, in either case, future installation of the application will fail. (I have encountered both.) Tweak the versions until it works.
-- The next step is "multi-stage build" which is covered in the [Minimal Containers](../minimal) workshop. There you will learn how to distinguish between buildtime versus runtime dependencies and separate them out.
+- The next step is "multi-stage build" which is covered in the [Minimal Containers](/notes/containers-for-hpc/minimal) workshop. There you will learn how to distinguish between buildtime versus runtime dependencies and separate them out.
 
 ## Clean Up
 
@@ -375,8 +375,8 @@ If you build containers often, you can run out of disk space quickly. To clean u
 
 ## References
 
-- [UVA Rivanna-Docker GitHub](https://github.com/uvarc/rivanna-docker)
+- [UVA Rivanna-Docker GitHub (opens in new tab)](https://github.com/uvarc/rivanna-docker)
     - Dockerfiles by UVA Research Computing
-    - [Tips](https://github.com/uvarc/rivanna-docker/wiki/Tips)
-- [_Best practices for writing Dockerfiles_](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/)
-- [Natanael Copa, _Small, Simple, and Secure: Alpine Linux under the Microscope_, DockerCon EU (2017)](https://youtu.be/sIG2P9k6EjA)
+    - [Tips (opens in new tab)](https://github.com/uvarc/rivanna-docker/wiki/Tips)
+- [_Best practices for writing Dockerfiles_ (opens in new tab)](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/)
+- [Natanael Copa, _Small, Simple, and Secure: Alpine Linux under the Microscope_, DockerCon EU (2017) (opens in new tab)](https://youtu.be/sIG2P9k6EjA)

content/notes/containers-for-hpc/intro.md

@@ -7,11 +7,10 @@ weight: 2
 
 menu:
     containers-for-hpc:
-        parent: Software Containers for HPC
-
+       parent: Software Containers for HPC
 ---
 
-"A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another." (<https://www.docker.com/resources/what-container>)
+"A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another." ([Docker definition of a container (opens in new tab)](https://www.docker.com/resources/what-container))
 
 ## Why use software containers?