diff --git a/docs/hub/overview.md b/docs/hub/overview.md new file mode 100644 index 00000000..5e6c7b34 --- /dev/null +++ b/docs/hub/overview.md @@ -0,0 +1,99 @@ +--- +title: "Overview" +description: "The RunPod Hub is a centralized repository for discovering, sharing, and deploying preconfigured AI repos optimized for RunPod's Serverless infrastructure." +sidebar_position: 1 +--- + +# RunPod Hub overview + +:::info + +The RunPod Hub is currently in **beta**. We're actively developing features and gathering user feedback to improve the experience. Please [join our discord](https://discord.gg/runpod) if you'd like to provide feedback. + +::: + +The RunPod Hub is a centralized repository that enables users to **discover, share, and deploy preconfigured AI repos** optimized for RunPod's [Serverless infrastructure](/serverless/overview/). It serves as a catalog of vetted, open-source repositories that can be deployed with minimal setup, creating a collaborative ecosystem for AI developers and users. + +Whether you're a developer looking to **share your work** or a user seeking **preconfigured solutions**, the Hub makes discovering and deploying AI projects seamless and efficient. + +Screenshot of the Hub page in the RunPod console + +## Why use the Hub? + +The Hub simplifies the entire lifecycle of repo sharing and deployment, from initial submission through testing, discovery, and usage. + +### For RunPod users + +- **Find production-ready AI solutions**: Discover vetted, open-source repositories optimized for RunPod with minimal setup required. +- **Deploy in one click**: Go from discovery to running services in minutes, not days. +- **Customize to your needs**: RunPod Hub repos expose configurable parameters for fine-tuning without diving into code. +- **Save development time**: Leverage community innovations instead of building from scratch. + +### For Hub creators + +- **Showcase your work**: Share your projects with the broader AI community. +- **Maintain control**: Your GitHub repo remains the source of truth, while the Hub automatically detects new releases. +- **Streamline your workflow**: Automated building and testing ensures your releases work as expected. + +## How it works + +The Hub operates through several key components working together: + +1. **Repository integration**: The Hub connects with GitHub repositories, using GitHub releases (not commits) as the basis for versioning and updates. +2. **Configuration system**: Repositories use standardized configuration files (`hub.json` and `tests.json`) in a `.runpod` directory to define metadata, hardware requirements, and test procedures. See the [publishing guide](/hub/publishing-guide) to learn more. +3. **Automated build pipeline**: When a repository is submitted or updated, the Hub automatically scans, builds, and tests it to ensure it works correctly on RunPod’s infrastructure. +4. **Continuous release monitoring**: The system regularly checks for new releases in registered repositories and rebuilds them when updates are detected. +5. **Deployment interface**: Users can browse repos, customize parameters, and deploy them to RunPod infrastructure with minimal configuration. + +## Getting started + +Whether you're a veteran developer who wants to share your work or a newcomer exploring AI models for the first time, the RunPod Hub makes getting started quick and straightforward. + +### Deploy a repo from the Hub + +You can deploy a repo from the Hub in seconds: + +1. Navigate to the [Hub page](https://www.runpod.io/console/hub) in the RunPod console. +2. Browse the collection and select a repo that matches your needs. +3. Review the repo details, including hardware requirements and available configuration options to ensure compatibility with your use case. +4. Click the **Deploy** button in the top-right of the repo page. You can also use the dropdown menu to deploy an older version. +5. Click **Create Endpoint** + +Within minutes you'll have access to a new Serverless endpoint, ready for integration with your applications or experimentation. + +### Publish your own repo + +Sharing your work through the Hub starts with preparing your GitHub repository with a working [Serverless endpoint](/serverless/overview) implementation, comprised of a [handler function](/serverless/handlers/overview) and `Dockerfile`. To learn how to create your first endpoint, [follow this guide](/serverless/get-started). + +Once your endpoint is ready to share: + +1. Add the required configuration files in a `.runpod` directory, following the instructions in the [Hub publishing guide](/hub/publishing-guide). +2. Create a GitHub release to establish a versioned snapshot. +3. Submit your repository to the Hub through the RunPod console, where it will undergo automated building and testing. +4. The RunPod team will review your repo. After approval, your repo will appear in the Hub. + +To learn more, see the [Hub publishing guide](/hub/publishing-guide). + +## Use cases + +The RunPod Hub supports a wide range of AI applications and workflows. Here are some common use cases that demonstrate the versatility and power of Hub repositories: + +### For AI researchers and enthusiasts + +Researchers can quickly deploy state-of-the-art models for experimentation without managing complex infrastructure. The Hub provides access to optimized implementations of popular models like Stable Diffusion, LLMs, and computer vision systems, allowing for rapid prototyping and iteration. This accessibility democratizes AI research by reducing the technical barriers to working with cutting-edge models. + +### For individual developers + +Individual developers benefit from the ability to experiment with different AI models and approaches without extensive setup time. The Hub provides an opportunity to learn from well-structured projects. Repos are designed to optimize resource usage, helping developers minimize costs while maximizing performance and potential earnings. + +### For enterprises and teams + +Enterprises and teams can accelerate their development cycle by using preconfigured repos instead of creating everything from scratch. The Hub reduces infrastructure complexity by providing standardized deployment configurations, allowing technical teams to focus on their core business logic rather than spending time configuring infrastructure and dependencies. + +## Join the community + +The RunPod Hub is more than just a list of repos—it's a community of AI builders sharing knowledge and innovation. + +By participating, you'll connect with other developers facing similar challenges and discover cutting-edge implementations that solve problems you might be struggling with. + +Whether you're deploying your first model or sharing your twentieth repo, the Hub provides both the infrastructure and community connections to help you succeed. diff --git a/docs/hub/publishing-guide.md b/docs/hub/publishing-guide.md new file mode 100644 index 00000000..433f1e74 --- /dev/null +++ b/docs/hub/publishing-guide.md @@ -0,0 +1,354 @@ +--- +title: "Publishing guide" +description: "Learn how to configure your repository for the RunPod Hub with hub.json and tests.json files, including metadata, deployment settings, and test specifications." +sidebar_position: 3 +--- + +# Hub publishing guide + +:::info + +The RunPod Hub is currently in **beta**. We're actively developing features and gathering user feedback to improve the experience. Please [join our discord](https://discord.gg/runpod) if you'd like to provide feedback. + +::: + +Learn how to publish your repositories to the [RunPod Hub](https://www.runpod.io/console/hub), including how to configure your repository with the required `hub.json` and `tests.json` files. + +Screenshot of the RunPod Hub repo setup UI + +## How to publish your repo + +Follow these steps to add your repository to the Hub: + +1. Navigate to the [Hub page](https://www.runpod.io/console/hub) in the RunPod console. +2. Under **Add your repo** click **Get Started**. +3. Enter your GitHub repo URL. +4. Follow the UI steps to add your repo to the Hub. + +The Hub UI will walk you through how to: + +1. Create your `hub.json` and `tests.json` files. +2. Ensure your repository contains a `handler.py`, `Dockerfile`, and `README.md` file (in either the `.runpod` or root directory). +3. Create a new GitHub release (the Hub indexes releases, not commits). +4. (Optional) Add a RunPod Hub badge into your GitHub `README.md` file, so that users can instantly deploy your repo from GitHub. + +After all the necessary files are in place and a release has been created, your repo will be marked "Pending" during building/testing. After testing is complete, the RunPod team will manually review the repo for publication. + +## Update a repo + +To update your repo on the Hub, just **create a new GitHub release**, and the Hub listing will be automatically indexed and built (usually within an hour). + +## Required files + +Aside from a working [Serverless implementation](/serverless/overview), every Hub repo requires two additional configuration files: + +1. `hub.json` - Defines metadata and deployment settings for your repo. +2. `tests.json` - Specifies how to test your repo. + +These files should be placed in the `.runpod` directory at the root of your repository. This directory takes precedence over the root directory, allowing you to override common files like `Dockerfile` and `README.md` specifically for the Hub. + +## hub.json reference + +The `hub.json` file defines how your listing appears and functions in the Hub. + +You can build your `hub.json` from scratch, or use [this template](#hubjson-template) as a starting point. + +### General metadata + +| Field | Description | Required | Values | +| --- | --- | --- | --- | +| `title` | Name of your tool | Yes | String | +| `description` | Brief explanation of functionality | Yes | String | +| `type` | Deployment type | Yes | `"serverless"` | +| `category` | Tool category | Yes | `"audio"`, `"embedding"`, `"language"`, `"video"`, or `"image"` | +| `iconUrl` | URL to tool icon | No | Valid URL | +| `config` | RunPod configuration | Yes | Object ([see below](#runpod-configuration)) | + +### RunPod configuration + +| Field | Description | Required | Values | +| --- | --- | --- | --- | +| `runsOn` | Machine type | Yes | `"GPU"` or `"CPU"` | +| `containerDiskInGb` | Container disk space allocation | Yes | Integer (GB) | +| `cpuFlavor` | CPU configuration | Only if `runsOn` is `"CPU"` | Valid CPU flavor string. For a complete list of available CPU flavors, see [CPU Types](https://docs.runpod.io/references/cpu-types) | +| `gpuCount` | Number of GPUs | Only if `runsOn` is `"GPU"` | Integer | +| `gpuIds` | GPU pool specification | Only if `runsOn` is `"GPU"` | Comma-separated pool IDs (e.g., `"ADA_24"`) or GPU IDs (e.g., `"RTX A4000"`) with optional GPU ID negations (e.g., `"-NVIDIA RTX 4090"`). For a complete list of available GPUs, see [GPU Types](https://docs.runpod.io/references/gpu-types). | +| `allowedCudaVersions` | Supported CUDA versions | No | Array of version strings | +| `env` | Environment variable definitions | No | Object ([see below](#environment-variables)) | +| `presets` | Default environment variable values | No | Object ([see below](#presets)) | + +### Environment variables + +Environment variables can be defined in several ways: + +1. **Static variables**: Direct value assignment. For example: + + ```json + { + "key": "API_KEY", + "value": "default-api-key-value" + } + ``` + +2. **String inputs**: User-entered text fields. For example: + + ```json + { + "key": "MODEL_PATH", + "input": { + "name": "Model path", + "type": "string", + "description": "Path to the model weights on disk", + "default": "/models/stable-diffusion-v1-5", + "advanced": false + } + } + ``` + +3. **Hugging Face inputs:** Fields for model selection from Hugging Face Hub. For example: + + ```json + { + "key": "HF_MODEL", + "input": { + "type": "huggingface", + "name": "Hugging Face Model", + "description": "Model organization/name as listed on Huggingface Hub", + "default": "runwayml/stable-diffusion-v1-5", + } + } + ``` + +4. **Option inputs**: User selected option fields. For example: + + ```json + { + "key": "PRECISION", + "input": { + "name": "Model precision", + "type": "string", + "description": "The numerical precision for model inference", + "options": [ + {"label": "Full Precision (FP32)", "value": "fp32"}, + {"label": "Half Precision (FP16)", "value": "fp16"}, + {"label": "8-bit Quantization", "value": "int8"} + ], + "default": "fp16" + } + } + ``` + +5. **Number Inputs**: User-entered numeric fields. For example: + + ```json + { + "key": "MAX_TOKENS", + "input": { + "name": "Maximum tokens", + "type": "number", + "description": "Maximum number of tokens to generate", + "min": 32, + "max": 4096, + "default": 1024 + } + } + ``` + +6. **Boolean Inputs**: User-toggled boolean fields. For example: + + ```json + { + "key": "USE_FLASH_ATTENTION", + "input": { + "type": "boolean", + "name": "Flash attention", + "description": "Enable Flash Attention for faster inference on supported GPUs", + "default": true, + "trueValue": "true", + "falseValue": "false" + } + } + ``` + +Advanced options will be hidden by default. Hide an option by setting: `"advanced": true` . + +### Presets + +Presets allow you to define groups of default environment variable values. When a user deploys your repo, they'll be offered a dropdown menu with any preset options you've defined. + +Here are some example presets: + +```json +"presets": [ + { + "name": "Quality Optimized", + "defaults": { + "MODEL_NAME": "runpod-stable-diffusion-xl", + "INFERENCE_MODE": "quality", + "BATCH_SIZE": 1, + "ENABLE_CACHING": false, + "USE_FLASH_ATTENTION": true + } + }, + { + "name": "Performance Optimized", + "defaults": { + "MODEL_NAME": "runpod-stable-diffusion-v1-5", + "INFERENCE_MODE": "fast", + "BATCH_SIZE": 8, + "ENABLE_CACHING": true, + "USE_FLASH_ATTENTION": true + } + } +] +``` + +## hub.json template + +Here’s an example `hub.json` file that you can use as a starting point: + +```json title="hub.json" +{ + "title": "Your Tool's Name", + "description": "A brief explanation of what your tool does", + "type": "serverless", + "category": "language", + "iconUrl": "https://your-icon-url.com/icon.png", + + "config": { + "runsOn": "GPU", + "containerDiskInGb": 20, + + "gpuCount": 1, + "gpuIds": "RTX A4000,-NVIDIA RTX 4090", + "allowedCudaVersions": [ + "12.8", "12.7", "12.6", "12.5", "12.4", + "12.3", "12.2", "12.1", "12.0" + ], + + "presets": [ + { + "name": "Preset Name", + "defaults": { + "STRING_ENV_VAR": "value1", + "INT_ENV_VAR": 10, + "BOOL_ENV_VAR": true + } + } + ], + + "env": [ + { + "key": "STATIC_ENV_VAR", + "value": "static_value" + }, + { + "key": "STRING_ENV_VAR", + "input": { + "name": "User-friendly Name", + "type": "string", + "description": "Description of this input", + "default": "default value", + "advanced": false + } + }, + { + "key": "OPTION_ENV_VAR", + "input": { + "name": "Select Option", + "type": "string", + "description": "Choose from available options", + "options": [ + {"label": "Option 1", "value": "value1"}, + {"label": "Option 2", "value": "value2"} + ], + "default": "value1" + } + }, + { + "key": "INT_ENV_VAR", + "input": { + "name": "Numeric Value", + "type": "number", + "description": "Enter a number", + "min": 1, + "max": 100, + "default": 50 + } + }, + { + "key": "BOOL_ENV_VAR", + "input": { + "type": "boolean", + "name": "Enable Feature", + "description": "Toggle this feature on/off", + "default": false, + "trueValue": "enabled", + "falseValue": "disabled" + } + } + ] + } +} + +``` + +## tests.json reference + +The `tests.json` file defines test cases to validate your tool's functionality. Tests are executed during the build step after [a release has been created](#publish-your-repo-to-the-runpod-hub). A test is considered valid by the Hub if the endpoint returns a [200 response](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/200). + +You can build your `tests.json` from scratch, or use [this template](#testsjson-template) as a starting point. + +### Test cases + +Each test case should include: + +| Field | Description | Required | Values | +| --- | --- | --- | --- | +| `name` | Test identifier | Yes | String | +| `input` | Raw job input payload | Yes | Object | +| `timeout` | Max execution time | No | Integer (milliseconds) | + +### Test environment configuration + +| Field | Description | Required | Values | +| --- | --- | --- | --- | +| `gpuTypeId` | GPU type for testing | Only for GPU tests | Valid GPU ID | +| `gpuCount` | Number of GPUs | Only for GPU tests | Integer | +| `cpuFlavor` | CPU configuration for testing | Only for CPU tests | Valid CPU flavor string | +| `env` | Test environment variables | No | Array of key-value pairs | +| `allowedCudaVersions` | Supported CUDA versions | No | Array of version strings | + +## tests.json template + +Here’s an example `tests.json` file that you can use as a starting point: + +```json title="tests.json" +{ + "tests": [ + { + "name": "test_case_name", + "input": { + "param1": "value1", + "param2": "value2" + }, + "timeout": 10000 + } + ], + "config": { + "gpuTypeId": "NVIDIA GeForce RTX 4090", + "gpuCount": 1, + "env": [ + { + "key": "TEST_ENV_VAR", + "value": "test_value" + } + ], + "allowedCudaVersions": [ + "12.7", "12.6", "12.5", "12.4", + "12.3", "12.2", "12.1", "12.0", "11.7" + ] + } +} + +``` \ No newline at end of file diff --git a/docs/overview.md b/docs/overview.md index 7b772a23..e4d14f19 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -29,9 +29,9 @@ Use Serverless to: Get started with Serverless: +- [Deploy a preconfigured endpoint from the RunPod Hub.](/hub/overview) - [Build a custom Serverless worker.](/serverless/get-started) - [Run any LLM as an endpoint using vLLM workers.](/serverless/vllm/get-started) -- [Tutorial: Deploy a Serverless worker with Stable Diffusion.](/tutorials/serverless/run-your-first) ## Pods diff --git a/docs/serverless/overview.md b/docs/serverless/overview.md index a9d41f86..7a1c25a6 100644 --- a/docs/serverless/overview.md +++ b/docs/serverless/overview.md @@ -50,23 +50,21 @@ runpod.serverless.start({"handler": handler}) # Required ## Deployment options -RunPod Serverless offers three ways to deploy your workloads, each designed for different use cases: +RunPod Serverless offers several ways to deploy your workloads, each designed for different use cases: -### Quick deploy an endpoint +### RunPod Hub -**Best for**: Deploying preconfigured AI models with minimal effort (no coding required). +**Best for**: Instantly deploying preconfigured AI models. -You can deploy a Serverless endpoint in minutes from the RunPod console: +You can deploy a Serverless endpoint from a repo in the [RunPod Hub](/hub/overview) in seconds: -1. Go to the [Serverless page](https://www.runpod.io/console/serverless) in the RunPod console. -2. Under **Quick Deploy**, browse the collection of preconfigured workers and select one that matches your needs. -3. Click **Configure**. Depending on your choice, you may need to enter a [Hugging Face model](https://huggingface.co/models). -4. Choose a **Worker Configuration**. Quick deploys are preconfigured with -5. Click the **Create Endpoint**. +1. Navigate to the [Hub page](https://www.runpod.io/console/hub) in the RunPod console. +2. Browse the collection and select a repo that matches your needs. +3. Review the repo details, including hardware requirements and available configuration options to ensure compatibility with your use case. +4. Click the **Deploy** button in the top-right of the repo page. You can also use the dropdown menu to deploy an older version. +5. Click **Create Endpoint** -You'll be redirected to your new endpoint. Now you're ready to [send a request](/serverless/endpoints/send-requests). - -[Quick deploy an endpoint →](https://www.runpod.io/console/serverless) +[Deploy a repo from the RunPod Hub →](https://www.runpod.io/console/hub) ### Deploy a vLLM worker diff --git a/sidebars.js b/sidebars.js index 8ca7d8c2..5b936a95 100644 --- a/sidebars.js +++ b/sidebars.js @@ -32,6 +32,17 @@ module.exports = { }, ], }, + { + type: "category", + label: "Hub [beta]", + collapsible: false, + items: [ + { + type: "autogenerated", + dirName: "hub", + }, + ], + }, { type: "category", label: "Pods", @@ -43,6 +54,7 @@ module.exports = { }, ], }, + { type: "category", label: "Instant Clusters", diff --git a/static/img/docs/hub-homepage.png b/static/img/docs/hub-homepage.png new file mode 100644 index 00000000..2ef34d72 Binary files /dev/null and b/static/img/docs/hub-homepage.png differ diff --git a/static/img/docs/hub-publish-page.png b/static/img/docs/hub-publish-page.png new file mode 100644 index 00000000..a71f4ed9 Binary files /dev/null and b/static/img/docs/hub-publish-page.png differ