Skip to content
This repository was archived by the owner on Dec 9, 2022. It is now read-only.

Commit 1a56d98

Browse files
committed
update docs
1 parent 58bd0ce commit 1a56d98

File tree

84 files changed

+4937
-1
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

84 files changed

+4937
-1
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
I"X{"source"=>"/data", "destination"=>"/data/_site", "collections_dir"=>"", "cache_dir"=>".jekyll-cache", "plugins_dir"=>"_plugins", "layouts_dir"=>"_layouts", "data_dir"=>"_data", "includes_dir"=>"_includes", "collections"=>{"posts"=>{"output"=>true, "permalink"=>"/:categories/:year/:month/:day/:title:output_ext"}}, "safe"=>false, "include"=>[".htaccess"], "exclude"=>[".sass-cache", ".jekyll-cache", "gemfiles", "Gemfile", "Gemfile.lock", "node_modules", "vendor/bundle/", "vendor/cache/", "vendor/gems/", "vendor/ruby/"], "keep_files"=>[".git", ".svn"], "encoding"=>"utf-8", "markdown_ext"=>"markdown,mkdown,mkdn,mkd,md", "strict_front_matter"=>true, "show_drafts"=>nil, "limit_posts"=>0, "future"=>false, "unpublished"=>false, "whitelist"=>[], "plugins"=>["jekyll-seo-tag", "jekyll-feed"], "markdown"=>"kramdown", "highlighter"=>"rouge", "lsi"=>false, "excerpt_separator"=>"\n\n", "incremental"=>false, "detach"=>false, "port"=>"4000", "host"=>"0.0.0.0", "baseurl"=>"", "show_dir_listing"=>false, "permalink"=>"date", "paginate_path"=>"/page:num", "timezone"=>nil, "quiet"=>false, "verbose"=>false, "defaults"=>[], "liquid"=>{"error_mode"=>"warn", "strict_filters"=>false, "strict_variables"=>false}, "kramdown"=>{"auto_ids"=>true, "toc_levels"=>"1..6", "entity_output"=>"as_char", "smart_quotes"=>"lsquo,rsquo,ldquo,rdquo", "input"=>"GFM", "hard_wrap"=>false, "guess_lang"=>true, "footnote_nr"=>1, "show_warnings"=>false, "syntax_highlighter"=>"rouge", "syntax_highlighter_opts"=>{:guess_lang=>true}, "coderay"=>{}}, "title"=>"MLOps Docs", "description"=>"A collection of docs that are relevant to MLOps with GitHub.", "url"=>"http://0.0.0.0:4000", "twitter_username"=>"github", "github_username"=>"github", "theme"=>"just-the-docs", "color_scheme"=>"dark", "aux_links"=>{"Back To MLOps Site"=>["//mlops-github.com"]}, "livereload_port"=>35729, "serving"=>true, "watch"=>true}:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
I"�<p>Experiment tracking for machine learning involves logging of metrics and artificacts associated with different training runs. This is essential for effective mlops as it allows you to track your performance metrics and promotes reproduceability in a transparent, reuseable way.</p>
2+
3+
<p>There are several experiment tracking systems available as third-party solutions that GitHub Actions can integrate with. For example, the below example illustrates how results of a training run can be fetched from <a href="https://www.wandb.com/">Weights and Biases</a> and dropped into a pull request:</p>
4+
5+
<p><img src="https://raw.githubusercontent.com/machine-learning-apps/actions-ml-cicd/master/images/mlops.png" />&lt;/img&gt;</p>
6+
7+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
I"5<h1 id="about">About</h1>
2+
3+
<p>These docs are maintained by the <a href="https://mlops-github.com/team">this team</a>.</p>
4+
5+
<h1 id="contributing">Contributing</h1>
6+
7+
<p>The source for these docs are hosted <a href="https://github.com/machine-learning-apps/website-docs">here</a>. We welcome contributions.</p>
8+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
I"+<p>Self-Hosted K8s Actions Runner</p>
2+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
I"�<p>There are many interesting things you can do with <a href="https://github.com/features/actions">Actions</a>, and <a href="https://github.com/features/codespaces/">CodeSpaces</a> to compliment jupyter notebooks on GitHub.</p>
2+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
I"�<p>Resources for MLOps on Kubernetes with GitHub. This is useful if you need to access broader infrastructure resources in your machine learning workflows. For example:</p>
2+
<ul>
3+
<li>Deploying and serving models, including a <a href="https://en.wikipedia.org/wiki/Service_mesh">service mesh</a></li>
4+
<li>Deploying and serving arbitrary applications, such as Jupyter Notebooks or documentation sites.</li>
5+
<li>Accessing other resources that are visible from your internal k8s cluster, such as:
6+
<ul>
7+
<li>Databases, i.e. Presto, Hive</li>
8+
<li>Storage, i.e. HDFS</li>
9+
<li>Distributed data processing, such as Dask or Spark</li>
10+
<li>Specialized computing such as GPUs.</li>
11+
<li>Integration with experiment tracking systems.</li>
12+
</ul>
13+
</li>
14+
<li>Access to <a href="https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/">secrets</a>.</li>
15+
<li>Native resource management and kubernetes for scalability and resiliance.</li>
16+
<li>Interoperability with machine learning or data pipelines, such as <a href="https://argoproj.github.io/">Argo</a>, <a href="https://mlflow.org/">MLFlow</a>, <a href="https://www.prefect.io/">Prefect</a>, etc.</li>
17+
<li>… and much more.</li>
18+
</ul>
19+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
I"�<h2 id="create-a-self-hosted-actions-runner-on-your-kubernetes-cluster-with-3-commands">Create A Self-Hosted Actions Runner On Your Kubernetes Cluster With 3 Commands</h2>
2+
3+
<h2 id="motivation">Motivation</h2>
4+
5+
<p><a href="https://github.com/features/actions">GitHub Actions</a> allow you to use <a href="https://help.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners">self hosted runners</a>. From the docs:</p>
6+
7+
<blockquote>
8+
<p>Self-hosted runners offer more control of hardware, operating system, and software tools than GitHub-hosted runners provide. With self-hosted runners, you can choose to create a custom hardware configuration with more processing power or memory to run larger jobs, install software available on your local network, and choose an operating system not offered by GitHub-hosted runners. Self-hosted runners can be physical, virtual, in a container, on-premises, or in a cloud.</p>
9+
</blockquote>
10+
11+
<p><a href="https://github.com/machine-learning-apps/self-hosted-k8s-runner">This repository</a> shows <strong>how to run a self hosted runner in a Kubernetes cluster</strong>, which is useful if your Actions runner needs to create resources or update deployments.</p>
12+
13+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
I"�<h2 id="fetch-runs-from-weights--biases">Fetch Runs From Weights &amp; Biases</h2>
2+
3+
<p><a href="https://www.wandb.com/">Weights and Biases</a> is a system for experiment tracking, model optimization, and dataset versioning.</p>
4+
5+
<p>The <a href="https://github.com/machine-learning-apps/wandb-action">Weights and Biases (W&amp;B) Action</a> can help you fetch runs from W&amp;B for reporting in your GitHub workflows.</p>
6+
7+
<p><strong>This Action saves a csv file called <code class="highlighter-rouge">wandb_report.csv</code> into the path specified by the <a href="https://help.github.com/en/articles/virtual-environments-for-github-actions#environment-variables">default environment variable</a> <code class="highlighter-rouge">GITHUB_WORKSPACE</code> set for you in GitHub Actions</strong>, which allows this data to be accessed by subsequent Actions. Information in this CSV can be displayed in a variety of ways, such as a markdown formatted comment in a pull request or via the <a href="https://developer.github.com/v3/checks/">GitHub Checks</a> API.</p>
8+
9+
<p>This csv file always has the following fields:</p>
10+
<ul>
11+
<li><code class="highlighter-rouge">run.url</code>: the url for the run in the W&amp;B api.</li>
12+
<li><code class="highlighter-rouge">run.name</code>: the name of the run. This is automatically set by wandb if not specified by the user.</li>
13+
<li><code class="highlighter-rouge">run.tags</code>: a list with all of the tags assigned to the run.</li>
14+
<li><code class="highlighter-rouge">run.id</code>: the id associated with the run. This corresponds to the input <code class="highlighter-rouge">RUN_ID</code></li>
15+
<li><code class="highlighter-rouge">run.entity</code>: this name of the entity that contains the project the run can be found in. This is similar to an org in GitHub.</li>
16+
<li><code class="highlighter-rouge">run.project</code>: the name of the project that contains the run. This is simlar to a repo in GitHub.</li>
17+
<li><code class="highlighter-rouge">github_sha</code>: the config variable <code class="highlighter-rouge">github_sha</code>.</li>
18+
<li><code class="highlighter-rouge">__eval.category</code>: this field will contain either the value <code class="highlighter-rouge">candiate</code> or <code class="highlighter-rouge">baseline</code>, depending on how the run was queried.</li>
19+
</ul>
20+
21+
<p>In addition to the above fields the user can specify additional fields.</p>
22+
23+
<p>Below is an example of how this Action can be used to fetch model runs:</p>
24+
25+
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">name</span><span class="pi">:</span> <span class="s">Get WandB Runs</span>
26+
<span class="na">on</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">issue_comment</span><span class="pi">]</span>
27+
28+
<span class="na">jobs</span><span class="pi">:</span>
29+
<span class="na">get-runs</span><span class="pi">:</span>
30+
<span class="na">if</span><span class="pi">:</span> <span class="s">(github.event.issue.pull_request != </span><span class="no">null</span><span class="s">) &amp;&amp; contains(github.event.comment.body, '/get-runs')</span>
31+
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
32+
33+
<span class="na">steps</span><span class="pi">:</span>
34+
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Get the latest SHA for the PR that was commented on</span>
35+
<span class="na">id</span><span class="pi">:</span> <span class="s">chatops</span>
36+
<span class="na">uses</span><span class="pi">:</span> <span class="s">machine-learning-apps/actions-chatops@master</span>
37+
<span class="na">with</span><span class="pi">:</span>
38+
<span class="na">TRIGGER_PHRASE</span><span class="pi">:</span> <span class="s2">"</span><span class="s">/get-runs"</span>
39+
<span class="na">env</span><span class="pi">:</span>
40+
<span class="na">GITHUB_TOKEN</span><span class="pi">:</span> <span class="s">${{ secrets.GITHUB_TOKEN }}</span>
41+
42+
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Get Runs Using SHA</span>
43+
<span class="na">uses</span><span class="pi">:</span> <span class="s">machine-learning-apps/wandb-action@master</span>
44+
<span class="na">with</span><span class="pi">:</span>
45+
<span class="na">PROJECT_NAME</span><span class="pi">:</span> <span class="s">${{ format('{0}/{1}', secrets.WANDB_ENTITY, secrets.WANDB_PROJECT) }}</span>
46+
<span class="na">FILTER_GITHUB_SHA</span><span class="pi">:</span> <span class="s">${{ steps.chatops.outputs.SHA }}</span>
47+
<span class="na">BASELINE_TAGS</span><span class="pi">:</span> <span class="s2">"</span><span class="s">['baseline',</span><span class="nv"> </span><span class="s">'reference']"</span>
48+
<span class="na">DISPLAY_METRICS</span><span class="pi">:</span> <span class="s2">"</span><span class="s">['accuracy',</span><span class="nv"> </span><span class="s">'loss',</span><span class="nv"> </span><span class="s">'best_val_acc',</span><span class="nv"> </span><span class="s">'best_val_loss',</span><span class="nv"> </span><span class="s">'_runtime']"</span>
49+
<span class="na">WANDB_API_KEY</span><span class="pi">:</span> <span class="s">${{ secrets.WANDB_API_KEY }}</span>
50+
<span class="na">DEBUG</span><span class="pi">:</span> <span class="s1">'</span><span class="s">true'</span>
51+
</code></pre></div></div>
52+
53+
<p>See <a href="https://github.com/machine-learning-apps/wandb-action">this project</a> for more information.</p>
54+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
I"�<h2 id="fetch-runs-from-weights--biases">Fetch Runs From Weights &amp; Biases</h2>
2+
3+
<p><a href="https://www.wandb.com/">Weights and Biases</a> is a system for experiment tracking, model optimization, and dataset versioning.</p>
4+
5+
<p>The <a href="https://github.com/machine-learning-apps/wandb-action">Weights and Biases (W&amp;B) Action</a> can help you fetch runs from W&amp;B for reporting in your GitHub workflows.</p>
6+
7+
<p><strong>This Action saves a csv file called <code class="highlighter-rouge">wandb_report.csv</code> into the path specified by the <a href="https://help.github.com/en/articles/virtual-environments-for-github-actions#environment-variables">default environment variable</a> <code class="highlighter-rouge">GITHUB_WORKSPACE</code> set for you in GitHub Actions</strong>, which allows this data to be accessed by subsequent Actions. Information in this CSV can be displayed in a variety of ways, such as a markdown formatted comment in a pull request or via the <a href="https://developer.github.com/v3/checks/">GitHub Checks</a> API.</p>
8+
9+
<p>This csv file always has the following fields:</p>
10+
<ul>
11+
<li><code class="highlighter-rouge">run.url</code>: the url for the run in the W&amp;B api.</li>
12+
<li><code class="highlighter-rouge">run.name</code>: the name of the run. This is automatically set by wandb if not specified by the user.</li>
13+
<li><code class="highlighter-rouge">run.tags</code>: a list with all of the tags assigned to the run.</li>
14+
<li><code class="highlighter-rouge">run.id</code>: the id associated with the run. This corresponds to the input <code class="highlighter-rouge">RUN_ID</code></li>
15+
<li><code class="highlighter-rouge">run.entity</code>: this name of the entity that contains the project the run can be found in. This is similar to an org in GitHub.</li>
16+
<li><code class="highlighter-rouge">run.project</code>: the name of the project that contains the run. This is simlar to a repo in GitHub.</li>
17+
<li><code class="highlighter-rouge">github_sha</code>: the config variable <code class="highlighter-rouge">github_sha</code>.</li>
18+
<li><code class="highlighter-rouge">__eval.category</code>: this field will contain either the value <code class="highlighter-rouge">candiate</code> or <code class="highlighter-rouge">baseline</code>, depending on how the run was queried.</li>
19+
</ul>
20+
21+
<p>In addition to the above fields the user can specify additional fields.</p>
22+
23+
<p>Below is an example of how this Action can be used to fetch model runs:</p>
24+
25+
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">name</span><span class="pi">:</span> <span class="s">Get WandB Runs</span>
26+
<span class="na">on</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">issue_comment</span><span class="pi">]</span>
27+
28+
<span class="na">jobs</span><span class="pi">:</span>
29+
<span class="na">get-runs</span><span class="pi">:</span>
30+
<span class="na">if</span><span class="pi">:</span> <span class="s">(github.event.issue.pull_request != </span><span class="no">null</span><span class="s">) &amp;&amp; contains(github.event.comment.body, '/get-runs')</span>
31+
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
32+
33+
<span class="na">steps</span><span class="pi">:</span>
34+
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Get the latest SHA for the PR that was commented on</span>
35+
<span class="na">id</span><span class="pi">:</span> <span class="s">chatops</span>
36+
<span class="na">uses</span><span class="pi">:</span> <span class="s">machine-learning-apps/actions-chatops@master</span>
37+
<span class="na">with</span><span class="pi">:</span>
38+
<span class="na">TRIGGER_PHRASE</span><span class="pi">:</span> <span class="s2">"</span><span class="s">/get-runs"</span>
39+
<span class="na">env</span><span class="pi">:</span>
40+
<span class="na">GITHUB_TOKEN</span><span class="pi">:</span> <span class="s">${{ secrets.GITHUB_TOKEN }}</span>
41+
42+
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Get Runs Using SHA</span>
43+
<span class="na">uses</span><span class="pi">:</span> <span class="s">machine-learning-apps/wandb-action@master</span>
44+
<span class="na">with</span><span class="pi">:</span>
45+
<span class="na">PROJECT_NAME</span><span class="pi">:</span> <span class="s">${{ format('{0}/{1}', secrets.WANDB_ENTITY, secrets.WANDB_PROJECT) }}</span>
46+
<span class="na">FILTER_GITHUB_SHA</span><span class="pi">:</span> <span class="s">${{ steps.chatops.outputs.SHA }}</span>
47+
<span class="na">BASELINE_TAGS</span><span class="pi">:</span> <span class="s2">"</span><span class="s">['baseline',</span><span class="nv"> </span><span class="s">'reference']"</span>
48+
<span class="na">DISPLAY_METRICS</span><span class="pi">:</span> <span class="s2">"</span><span class="s">['accuracy',</span><span class="nv"> </span><span class="s">'loss',</span><span class="nv"> </span><span class="s">'best_val_acc',</span><span class="nv"> </span><span class="s">'best_val_loss',</span><span class="nv"> </span><span class="s">'_runtime']"</span>
49+
<span class="na">WANDB_API_KEY</span><span class="pi">:</span> <span class="s">${{ secrets.WANDB_API_KEY }}</span>
50+
<span class="na">DEBUG</span><span class="pi">:</span> <span class="s1">'</span><span class="s">true'</span>
51+
</code></pre></div></div>
52+
53+
<p>See <a href="https://github.com/machine-learning-apps/wandb-action">this project</a> for more information.</p>
54+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
I"<p>About</p>
2+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
I"!<p>Weights &amp; Biases</p>
2+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
I"�<h2 id="fastaifastpages--blogging-with-jupyter-notebooks"><a href="https://github.com/fastai/fastpages">fastai/fastpages</a>: Blogging with Jupyter Notebooks</h2>
2+
3+
<p><img src="https://raw.githubusercontent.com/fastai/fastpages/master/images/diagram.png" /></p>
4+
5+
<p>From the <a href="https://github.com/fastai/fastpages">project page</a>:</p>
6+
7+
<p><code class="highlighter-rouge">fastpages</code> uses <a href="https://github.com/features/actions">GitHub Actions</a> to simplify the process of creating <a href="https://jekyllrb.com/">Jekyll blog posts</a> on <a href="https://pages.github.com/">GitHub Pages</a> from a variety of input formats.</p>
8+
9+
<h3 id="fastpages-provides-the-following-features"><code class="highlighter-rouge">fastpages</code> provides the following features:</h3>
10+
11+
<ul>
12+
<li>Create posts containing code, outputs of code (which can be interactive), formatted text, etc directly from <a href="https://jupyter.org/">Jupyter Notebooks</a>; Notebook posts support features such as:
13+
<ul>
14+
<li>Interactive visualizations made with <a href="https://altair-viz.github.io/">Altair</a> remain interactive.</li>
15+
<li>Hide or show cell input and output.</li>
16+
<li>Collapsable code cells that are either open or closed by default.</li>
17+
<li>Define the Title, Summary and other metadata via a special markdown cells</li>
18+
<li>Ability to add links to <a href="https://colab.research.google.com/">Colab</a> and GitHub automatically.</li>
19+
</ul>
20+
</li>
21+
<li>Support for comments, supported natively through GitHub Issues.</li>
22+
<li>Built-in search.</li>
23+
<li>Support for customizing the styling of your site.</li>
24+
<li>Embed Twitter cards and YouTube videos.</li>
25+
<li>Categorization of blog posts by user-supplied tags for discoverability.</li>
26+
<li>Create and edit <a href="https://guides.github.com/features/mastering-markdown/">Markdown</a> posts.</li>
27+
<li>Create posts, including formatting and images, directly from Microsoft Word documents.</li>
28+
<li>Write posts on your local machine and <a href="#running-the-blog-on-your-local-machine">preview them with live reload</a>.</li>
29+
</ul>
30+
31+
<p>See <a href="https://github.com/fastai/fastpages">the project</a> for more details.</p>
32+
:ET
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
I"<p>Argo</p>
2+
:ET

0 commit comments

Comments
 (0)