Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds new artifacts colab. #526

Merged
merged 27 commits into from
Jul 25, 2024
Merged
Changes from 2 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
bd188ed
Adds new artifacts colab.
katjacksonWB May 15, 2024
e2f825c
Finishes all the runs in the notebook.
katjacksonWB May 15, 2024
4d9573f
Adds new section.
katjacksonWB May 20, 2024
bbb30b2
Adds a new section.
katjacksonWB May 20, 2024
01edf7c
Update colabs/artifact_basics/Artifact_Basics.ipynb
katjacksonWB May 23, 2024
663e980
Added more specific text.
katjacksonWB May 23, 2024
46cc6e9
Update colabs/artifact_basics/Artifact_Basics.ipynb
katjacksonWB May 23, 2024
eefe863
New header.
katjacksonWB May 23, 2024
65639ab
Lineage changes.
katjacksonWB May 23, 2024
3e2feb5
Update colabs/artifact_basics/Artifact_Basics.ipynb
katjacksonWB May 28, 2024
3a8c8da
Update colabs/artifact_basics/Artifact_Basics.ipynb
katjacksonWB May 28, 2024
567b8d4
More contextual links.
katjacksonWB May 28, 2024
51aff38
Technical changes.
katjacksonWB May 28, 2024
350c50d
Grammar and technical corrections.
katjacksonWB May 28, 2024
14dbc00
Additional section.
katjacksonWB May 28, 2024
cf3dcaf
Code corrections.
katjacksonWB May 30, 2024
4c05cfe
Updates to metadata section.
katjacksonWB May 30, 2024
14d2db4
Code edits.
katjacksonWB May 30, 2024
69d71d1
Addtional edits.
katjacksonWB May 31, 2024
57ad97c
New Artifact version section.
katjacksonWB May 31, 2024
66dc524
addressing outstanding issues with the new basic Artifacts colab
Jul 8, 2024
6b8bf2b
making the artifact-basics.ipynb colab environment agnostic
Jul 10, 2024
7cee2f6
removing output from the artifact-basics.ipynb colab
Jul 10, 2024
84750d8
Added Pandas import, removed extra img tags (#548)
ngrayluna Jul 24, 2024
01abaac
Fixed typo
ngrayluna Jul 24, 2024
60ffbc7
Renamed notebook
ngrayluna Jul 24, 2024
0398013
clean up
tcapelle Jul 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
280 changes: 280 additions & 0 deletions colabs/artifact_basics/Artifact_Basics.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,280 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "gsp5VlKQVMud"
},
"source": [
"<a href=\"https://colab.research.google.com/github/wandb/examples/blob/master/colabs/pytorch/Simple_PyTorch_Integration.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"<!--- @wandbcode{pytorch-video} -->"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TbzoV1XvVMug"
},
"source": [
"<img src=\"http://wandb.me/logo-im-png\" width=\"400\" alt=\"Weights & Biases\" />\n",
"\n",
"<!--- @wandbcode{pytorch-video} -->\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MGA0PLfsVMuh"
},
"source": [
"\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"Use [Weights & Biases](https://wandb.com) for machine learning experiment tracking, dataset versioning, and project collaboration.\n",
"\n",
"<div><img /></div>\n",
"\n",
"<img src=\"https://wandb.me/mini-diagram\" width=\"650\" alt=\"Weights & Biases\" />\n",
"\n",
"<div><img /></div>\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "11CLrFgFExMe"
},
"source": [
"\n",
"Use W&B Artifacts to track and version data as the inputs and outputs of your W&B Runs. For example, a model training run might take in a dataset as input and trained model as output. In addition to logging hyperparameters and metadata to a run, you can use an artifact to log the dataset used to train the model as input and the resulting model checkpoints as outputs."
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dX5cFz_XCd5h"
},
"source": [
"# Set Up"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "q2EeMdcpC7Dl"
},
"source": [
"In order to log data to our web service, you need to log in and import `wandb`. If this is your first time using W&B, you'll need to sign up for a free account at the link that appears."
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "nwEF5d7NChdB"
},
"outputs": [],
"source": [
"!pip install wandb\n",
"import wandb\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split pip install from import wandb please

"wandb.login()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vp9Zhzp5Hvt1"
},
"source": [
"# Create An Artifact"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3xEsZzOEnPFY"
},
"source": [
"The general workflow for creating an Artifact is:\n",
"\n",
"\n",
"1. Intialize a run.\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just feel like we are not following with code this plan.

"2. Create an Artifact.\n",
"3. Add a dataset, a model, or another Artifact as an input.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"4. Log the artifact in the W&B platform.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"This can by accomplished in two lines of code:"
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Vndk289hgVVK"
},
"outputs": [],
"source": [
"run = wandb.init(project=\"artifact-basics\")\n",
"run.log_artifact(artifact_or_path=\"/content/sample_data/mnist_test.csv\", name=\"my_first_artifact\", type=\"dataset\")\n",
"run.finish()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tgzxAogXnwCj"
},
"source": [
"First, initalize the run with `wandb.init()`. In this demo, the code adds the run to the `artifact-basic` project, but you can change the name to anyting you'd like.\n",
"\n",
"Next, log the artifact with `wandb.artifact()`. In this demo, the artifact is a `dataset` using data from `mnist_test.csv`. You can customize your artifact with a `name` and other metadata- see the Artifacts Reference guide for more information.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"\n",
"If you change or add any parameters names, be sure to replicate those changes in the following code samples as well."
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bpGidixAICM1"
},
"source": [
"# Use an Artifact"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jZC69sSMdX4j"
},
"source": [
"When you need to select a particular Artifact, indicate the artifact you want to use with the `use_artifact` method. This returns a run object. In the proceeding code snippet specifies an artifact called `my_first_artifact` with the alias `latest`:\n"
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7Ix_ciolJICe"
},
"outputs": [],
"source": [
"run = wandb.init(project=\"artifact-basics\")\n",
"artifact = run.use_artifact(artifact_or_name=\"my_first_artifact:latest\")\n",
"run.finish()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o-rcMsZYh0Ig"
},
"source": [
"# Download an Artifact"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "u60Orwg5sYuI"
},
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "B2ssTI7LrHS8"
},
"source": [
"To retrieve the path of an Artifact for external use, use the `artifact.download()` method. This will retrieve the directory of an Artifact you select."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "18-7Kd9Oh8cl"
},
"outputs": [],
"source": [
"run = wandb.init(project=\"artifact-basics\")\n",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think would be better here to init the run at the start of the code block.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • maybe split in 2 cells.
  • csv_data instead of csvData.

"artifact = run.use_artifact(artifact_or_name=\"my_first_artifact:latest\")\n",
"# downloads the artifact to a directory on the W&B platform.\n",
"datadir = artifact.download()\n",
"run.finish()\n",
"# prints the current artifact directory\n",
"print(u'\\u2500' * 10)\n",
"print(\"Data directory located at:\" + datadir)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sxox0H9grrcx"
},
"source": [
"For more information on ways to customize your Artifact download, see the [Download and Usage guide](https://docs.wandb.ai/guides/artifacts/download-and-use-an-artifact)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AyANijPlIN5m"
},
"source": [
"# Update an Artifact"
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e4psfYzGeAVS"
},
"source": [
"You can pass new values to update the `description`, `metadata`, and `alias` of an artifact. Call the `save()` method to update the artifact on the W&B servers. You can update an artifact during or outside a W&B Run.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"\n",
"This example changes the `description` of the `my_first_artifact` artifact inside a run:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "16fCvZUZJD29"
},
"outputs": [],
"source": [
"run = wandb.init(project=\"artifact-basics\")\n",
"artifact = run.use_artifact(artifact_or_name=\"my_first_artifact:latest\")\n",
"artifact.description = \"This is an edited description.\"\n",
"artifact.save()\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"run.finish()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "b6F6b6XHVMus"
},
"source": [
"# Next steps\n",
"1. [Artifacts reference documentation](https://docs.wandb.ai/ref/python/artifact): Deep dive into artifact parameters and advanced methods.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"2. [Traversing Artifacts](https://docs.wandb.ai/guides/artifacts/explore-and-traverse-an-artifact-graph): Tips on how to work with Artifacts on W&B's web platform.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"3. [Artifact Automation](https://docs.wandb.ai/guides/artifacts/project-scoped-automations): Automate downstream actions for versioning artifacts.\n",
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
"4. [Artifacts FAQ](https://docs.wandb.ai/guides/artifacts/artifacts-faqs): Frequently asked questions on Artifact security, workflows, and retention policy."
katjacksonWB marked this conversation as resolved.
Show resolved Hide resolved
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Loading