diff --git a/.cursor/rules/rp-styleguide.mdc b/.cursor/rules/rp-styleguide.mdc index 23d1491d..4c6ae733 100644 --- a/.cursor/rules/rp-styleguide.mdc +++ b/.cursor/rules/rp-styleguide.mdc @@ -6,5 +6,5 @@ alwaysApply: true Always use sentence case for headings and titles. These are proper nouns: Pods, Serverless, Hub, Instant Clusters, Secure Cloud, Community Cloud, Tetra. These are generic terms: endpoint, worker, cluster, template, handler, fine-tune, network volume. -Prefer using paragraphs to bullet points unless directly asked. -When using bullet points, end each line with a period. +Prefer using paragraphs to bullet points unless specifically requested. +When using bullet points, use hyphens instead of asterisks, and end each line with a period. diff --git a/docs.json b/docs.json index a966df17..a225287f 100644 --- a/docs.json +++ b/docs.json @@ -62,7 +62,8 @@ "group": "Storage", "pages": [ "serverless/storage/overview", - "serverless/storage/network-volumes" + "serverless/storage/network-volumes", + "serverless/storage/s3-api" ] }, { diff --git a/hub/overview.mdx b/hub/overview.mdx index f2ef368d..aa14279f 100644 --- a/hub/overview.mdx +++ b/hub/overview.mdx @@ -6,7 +6,7 @@ description: "Discover, deploy, and share preconfigured AI repos." -**The RunPod Hub is currently in beta**. We're actively developing features and gathering user feedback to improve the experience. Please [join our discord](https://discord.gg/runpod) if you'd like to provide feedback. +**The RunPod Hub is currently in beta**. We're actively developing features and gathering user feedback to improve the experience. If you'd like to provide feedback, please [join our Discord](https://discord.gg/runpod). diff --git a/serverless/pricing.mdx b/serverless/pricing.mdx index 6bdc1dbb..b71ca652 100644 --- a/serverless/pricing.mdx +++ b/serverless/pricing.mdx @@ -55,6 +55,8 @@ Your total Serverless costs include both compute time (GPU usage) and temporary If you have many workers continuously running with high storage costs, you can utilize [network volumes](/serverless/storage/network-volumes) to reduce expenses. Network volumes allow you to share data efficiently across multiple workers, reduce per-worker storage requirements by centralizing common files, and maintain persistent storage separate from worker lifecycles. +Network volumes are billed hourly at a rate of \$0.07 per GB per month for the first 1TB, and \$0.05 per GB per month for additional storage beyond that. + ### Compute cost breakdown Serverless workers incur charges during these periods: diff --git a/serverless/storage/network-volumes.mdx b/serverless/storage/network-volumes.mdx index c7c77ae3..b5634732 100644 --- a/serverless/storage/network-volumes.mdx +++ b/serverless/storage/network-volumes.mdx @@ -6,10 +6,12 @@ description: "Persistent, portable storage for Serverless workers." Network volumes offer persistent storage that exists independently of the lifecycle of a [Serverless worker](/serverless/workers/overview). This means your data will be retained even if a worker is stopped or your endpoint is deleted. -Network volumes can be attached to multiple Serverless endpoints, making them ideal for sharing data, or maintaining datasets between Serverless workers. +Network volumes can be attached to multiple Serverless endpoints, making them ideal for sharing data, or maintaining datasets between workers. When attached to a Serverless endpoint, a network volume is mounted at `/runpod-volume` within the worker environment. +Network volumes are billed hourly at a rate of \$0.07 per GB per month for the first 1TB, and \$0.05 per GB per month for additional storage beyond that. + If your account lacks sufficient funds to cover storage costs, your network volume may be terminated. Once terminated, the disk space is immediately freed for other users, and RunPod cannot recover lost data. Ensure your account remains funded to prevent data loss. @@ -18,11 +20,11 @@ If your account lacks sufficient funds to cover storage costs, your network volu ## When to use a network volume -Consider using a network volume when your Serverless endpoints needs: +Consider using a network volume when your endpoints needs: * **Persistent data that outlives individual workers:** Keep your data safe and accessible even after a worker is stopped or an endpoint is scaled to zero. * **Shareable storage:** Use the same data across different workers of the same endpoint or even different endpoints. While a network volume can be *attached* to multiple Serverless endpoint configurations, ensure your application logic handles concurrent access appropriately if workers from different endpoints write to the volume to prevent data corruption. -* **Efficient data management:** Store frequently used models or large datasets to avoid re-downloading them for each new Serverless worker, saving time, bandwidth, and reducing cold start times. +* **Efficient data management:** Store frequently used models or large datasets to avoid re-downloading them for each new worker, saving time, bandwidth, and reducing cold start times. * **Stateful applications:** Maintain state across multiple invocations of a Serverless function, enabling more complex, long-running tasks. ## Create a network volume @@ -32,7 +34,7 @@ To create a new network volume: 1. Navigate to the [Storage page](https://www.runpod.io/console/user/storage) in the RunPod console. 2. Select **New Network Volume**. 3. **Configure your volume:** - * Select a datacenter for your volume. Datacenter location does not affect pricing, but the datacenter location will determine which Serverless endpoints your network volume can be used with. Your Serverless endpoint must be in the same datacenter as the network volume. + * Select a datacenter for your volume. Datacenter location does not affect pricing, but the datacenter location will determine which endpoints your network volume can be paired with. Your Serverless endpoint must be in the same datacenter as the network volume. * Provide a descriptive name for your volume (e.g., "serverless-shared-models" or "project-gamma-datasets"). * Specify the desired size for the volume in gigabytes (GB). @@ -50,7 +52,7 @@ You can edit and delete your network volumes using the [Storage page](https://ww To enable workers on an endpoint to use a network volume: -1. Navigate to the [Serverless Endpoints page](https://www.runpod.io/console/serverless/user/endpoints) in the RunPod console. +1. Navigate to the [Serverless page](https://www.runpod.io/console/serverless/user/endpoints) in the RunPod console. 2. Either create a **New Endpoint** or select an existing endpoint and choose **Edit Endpoint** from the options menu (three dots). 3. In the endpoint configuration, expand the **Advanced** section. 4. From the **Network Volume** dropdown, select the network volume you want to attach to the endpoint. @@ -64,6 +66,10 @@ Multiple workers cannot write to a single network volume simultaneously. +## Manage files + +To manage files in your network volume (for example, to upload large models used by your workers), use the [RunPod S3-compatible API](/serverless/storage/s3-api). + ## Architecture details Network volumes are backed by high-performance storage servers co-located with RunPod GPU servers. These are connected via high-speed networks and use NVMe SSDs for optimal performance, but data transfer speeds can vary widely based on location and network conditions (200-400MB/s, up to 10GB/s). @@ -74,4 +80,4 @@ Using network volumes with Serverless provides significant flexibility and can l * **Reduced cold starts:** By storing large models or datasets on a network volume, workers can access them quickly without needing to download them on each cold start. * **Cost efficiency:** Network volume storage space costs less than frequently re-downloading large files or relying solely on container storage for data that needs to persist. -* **Simplified data management:** Centralize your datasets and models for easier updates and management across multiple Serverless workers and endpoints. \ No newline at end of file +* **Simplified data management:** Centralize your datasets and models for easier updates and management across multiple workers and endpoints. diff --git a/serverless/storage/s3-api.mdx b/serverless/storage/s3-api.mdx new file mode 100644 index 00000000..a9e268b3 --- /dev/null +++ b/serverless/storage/s3-api.mdx @@ -0,0 +1,296 @@ +--- +title: "S3-compatible API" +sidebarTitle: "S3-compatible API" +description: "Use RunPod's S3-compatible API to access and manage your network volumes." +--- + + + +The S3-compatible API is currently in beta. If you'd like to provide feedback, please [join our Discord](https://discord.gg/runpod) + + + +RunPod provides an S3-protocol compatible API that allows direct access to your [network volumes](/serverless/storage/network-volumes). This feature enables you to manage files on your network volumes without needing to launch a Pod, reducing cost and operational friction. + +Using the S3-compatible API to access your network volumes does not affect pricing. Network volumes are billed hourly at a rate of \$0.07 per GB per month for the first 1TB, and \$0.05 per GB per month for additional storage beyond that. + +## Setup and authentication + + + + To use the S3-compatible API, you must generate a new key called an "S3 API key", separate from your RunPod API key. + + 1. In the RunPod console, navigate to the [Settings page](https://www.runpod.io/console/user/settings). + 2. Expand the **S3 API Keys** section and select **Create an S3 API key**. + 3. Give your key a name and select **Create**. + 4. Make a note of your access key ID and secret access key to use in the next step. + + + + For security, RunPod will show your secret access key only once, so you may wish to save it elsewhere (e.g., in your password manager, or in a GitHub secret). Treat your secret access key like a password and don't share it with anyone. + + + + + + To use the S3-compatible API with your RunPod network volumes, you must configure your AWS CLI with the RunPod S3 API key you created. + + 1. If you haven't already, [install the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) on your local machine. + 2. Run the command `aws configure` in your terminal. + 3. Provide the following when prompted: + * **AWS Access Key ID**: Enter your RunPod S3 API key's access key ID. + * **AWS Secret Access Key**: Enter your RunPod S3 API key's secret access key. + * **Default Region name**: You can leave this blank. + * **Default output format**: You can leave this blank or set it to `json`. + + This will configure the AWS CLI to use your RunPod S3 API key by storing these details in your AWS credentials file (typically at `~/.aws/credentials`). + + + + +## Using the S3-compatible API + +You can use the S3-compatible API to interact with your RunPod network volumes using standard S3 tools: + +* [AWS CLI commands](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html). +* [The Boto3 Python library](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html). + +Core AWS CLI operations such as `ls`, `cp`, `mv`, `rm`, and `sync` function as expected. + + + +## Datacenter availability + +The S3-compatible API is currently available for network volumes hosted in a limited number of datacenters. + +Each datacenter has an endpoint URL that you'll use when calling the S3-compatible API, using the format `https://s3api-[DATACENTER].runpod.dev/`. + +Create a network volume in one of the following datacenters to use the S3-compatible API: + +| Datacenter | Endpoint URL | +|-------------|--------------| +| `EUR-IS-1` | `https://s3api-eur-is-1.runpod.dev/` | +| `EU-RO-1` | `https://s3api-eu-ro-1.runpod.dev/` | + +## AWS CLI examples + +When using AWS CLI commands, you must pass in the endpoint URL for your network volume ([listed above](#datacenter-availability)) using the `--endpoint-url` flag. + + + +Unlike traditional S3 key-value stores, object names in the RunPod S3-compatible API correspond to actual file paths on your network volume. Object names containing special characters (e.g., `#`) may need to be URL encoded to ensure proper processing. + + + +### List objects + +Use `ls` to list objects in a network volume directory: + +```bash +aws s3 ls --endpoint-url https://s3api-[DATACENTER].runpod.dev/ \ + s3://[NETWORK_VOLUME_ID]/[REMOTE_DIR] +``` + +### Transfer files + +Use `cp` to copy a file to a network volume: + +```bash +aws s3 cp --endpoint-url https://s3api-[DATACENTER].runpod.dev/ \ + local-file.txt \ + s3://[NETWORK_VOLUME_ID] +``` + +Use `cp` to copy a file from a network volume to a local directory: + +```bash +aws s3 cp --endpoint-url https://s3api-[DATACENTER].runpod.dev/ \ + s3://[NETWORK_VOLUME_ID]/remote-file.txt ./[LOCAL_DIR] +``` + +Use `rm` to remove a file from a network volume: + +```bash +aws s3 rm --endpoint-url https://s3api-[DATACENTER].runpod.dev/ \ + s3://[NETWORK_VOLUME_ID]/remote-file.txt +``` + +### Sync directories + +This command syncs a local directory (source) to a network volume directory (destination): + +```bash +aws s3 sync --endpoint-url https://s3api-[DATACENTER].runpod.dev/ \ + ./[LOCAL_DIR] \ + s3://[NETWORK_VOLUME_ID]/[REMOTE_DIR] +``` + +## Boto3 Python example + +You can also use the Boto3 library to interact with the S3-compatible API, using it to transfer files to and from a RunPod network volume. + +The script below demonstrates how to upload a file to a RunPod network volume using the Boto3 library. It takes command-line arguments for the network volume ID (as an S3 bucket), the datacenter-specific S3 endpoint URL, the local file path, the desired object (file path on the network volume), and the AWS Region (which corresponds to the RunPod datacenter ID). + +Your RunPod S3 API key credentials must be set as environment variables: + +* `RUNPOD_USER_EMAIL`: Should be set to your RunPod account email. +* `RUNPOD_S3_API_KEY`: Should be set to your RunPod S3 API key's secret access key. + +```python +#!/usr/bin/env python3 + +import os +import argparse +import boto3 # AWS SDK for Python, used to interact with RunPod S3-compatible APIs + +def create_s3_client(region: str, endpoint_url: str): + + # Creates and returns an S3 client configured for RunPod network volume S3-compatible API. + # + # Args: + # region (str): The RunPod datacenter ID, used as the AWS region + # (e.g., 'ca-qc-1'). + # endpoint_url (str): The S3 endpoint URL for the specific RunPod datacenter + # (e.g., 'https://ca-qc-1-s3api.runpod.dev/'). + + # Returns: + # boto3.client: An S3 client object, configured for the RunPod S3 API. + + # Retrieve RunPod S3 API key credentials from environment variables. + aws_access_key_id = os.environ.get("RUNPOD_USER_EMAIL") + aws_secret_access_key = os.environ.get("RUNPOD_S3_API_KEY") + + # Ensure necessary S3 API key credentials are set in the environment + if not aws_access_key_id or not aws_secret_access_key: + raise EnvironmentError( + "Please set RUNPOD_USER_EMAIL (with S3 API Key Access Key) and " + "RUNPOD_S3_API_KEY (with S3 API Key Secret Access Key) environment variables. " + "These are obtained from 'S3 API Keys' in the RunPod console settings." + ) + + # Initialize and return the S3 client for RunPod's S3-compatible API + return boto3.client( + "s3", + aws_access_key_id=aws_access_key_id, + aws_secret_access_key=aws_secret_access_key, + region_name=region, # Corresponds to the RunPod datacenter ID + endpoint_url=endpoint_url, # Datacenter-specific S3 API endpoint + ) + +def put_object(s3_client, bucket_name: str, object_name: str, file_path: str): + + # Uploads a local file to the specified RunPod network volume. + # + # Args: + # s3_client: The S3 client object (e.g., returned by create_s3_client). + # bucket_name (str): The ID of the target RunPod network volume. + # object_name (str): The desired file path for the object on the network volume. + # file_path (str): The local path to the file that needs to be uploaded. + + try: + # Attempt to upload the file to the RunPod network volume. + s3_client.upload_file(file_path, bucket_name, object_name) + print(f"Successfully uploaded '{file_path}' to Network Volume '{bucket_name}' as '{object_name}'") + except Exception as e: + # Catch any exception during upload, print an error, and re-raise + print(f"Error uploading file '{file_path}' to Network Volume '{bucket_name}' as '{object_name}': {e}") + raise + +def main(): + + # Parses command-line arguments and orchestrates the file upload process + # to a RunPod network volume. + + # Set up command-line argument parsing + parser = argparse.ArgumentParser( + description="Upload a file to a RunPod Network Volume using its S3-compatible API. " + "Requires RUNPOD_USER_EMAIL and RUNPOD_S3_API_KEY env vars to be set " + "with your RunPod S3 API key credentials." + ) + parser.add_argument( + "-b", "--bucket", + required=True, + help="The ID of your RunPod Network Volume (acts as the S3 bucket name)." + ) + parser.add_argument( + "-e", "--endpoint", + required=True, + help="The S3 endpoint URL for your RunPod datacenter (e.g., 'https://s3api-[DATACENTER].runpod.dev/')." + ) + parser.add_argument( + "-f", "--file", + required=True, + help="The local path to the file to be uploaded." + ) + parser.add_argument( + "-o", "--object", + required=True, + help="The S3 object key (i.e., the desired file path on the Network Volume)." + ) + parser.add_argument( + "-r", "--region", + required=True, + help="The RunPod datacenter ID, used as the AWS region (e.g., 'ca-qc-1'). Find this in the RunPod console's Storage section or endpoint URL." + ) + + args = parser.parse_args() + + # Create the S3 client using the parsed arguments, configured for RunPod. + client = create_s3_client(args.region, args.endpoint) + + # Upload the object to the specified network volume. + put_object(client, args.bucket, args.object, args.file) + +if __name__ == "__main__": + main() +``` + +Example usage: + +```bash +./s3_example_put.py --endpoint https://s3api-eur-is-1.runpod.dev/ \ + --region 'EUR-IS-1' \ + --bucket 'network_volume_id' \ + --object 'path/to/model_file.bin' \ + --file 'model_file.bin' +``` + +## Supported S3 actions + +The S3-compatible API supports the following operations. For detailed information on each, refer to the [AWS S3 API documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html). + +| Operation | Description | +| ------------------------- | -------------------------------------------- | +| `CopyObject` | Copy objects between locations. | +| `DeleteObject` | Remove objects. | +| `GetObject` | Download objects. | +| `HeadBucket` | Verify bucket exists and you have permissions. | +| `HeadObject` | Retrieve object metadata. | +| `ListBuckets` | List available buckets. | +| `ListObjects` | List objects in a bucket. | +| `PutObject` | Upload objects. | +| `CreateMultipartUpload` | Start a multipart upload for large files. | +| `UploadPart` | Upload a part of a multipart upload. | +| `CompleteMultipartUpload`| Finish a multipart upload. | +| `AbortMultipartUpload` | Cancel a multipart upload. | +| `ListMultipartUploads` | View in-progress multipart uploads. | + +Large file handling is supported through multipart uploads, allowing you to transfer files larger than 5GB. + +## Limitations + +- **Multipart uploads**: + - Parts from multipart uploads are stored on disk until either `CompleteMultipartUpload` or `AbortMultipartUpload` is called. + - The S3-compatible API enforces the 5GB maximum single file part upload size, but not the 5TB maximum file size. + - The 5MB minimum part size for multipart uploads is not enforced. +- **Storage capacity**: Network volumes have a fixed storage capacity, unlike the virtually unlimited storage of standard S3 buckets. The `CopyObject` and `UploadPart` actions do not check for available free space beforehand and may fail if the volume runs out of space. This behavior is similar to applying a size quota in S3. +- **Object names**: Unlike traditional S3 key-value stores, object names in the RunPod S3-compatible API correspond to actual file paths on your network volume. Object names containing special characters (e.g., `#`) may need to be URL encoded to ensure proper processing. +- **Time synchronization**: Requests that are out of time sync by 1 hour will be rejected. This is more lenient than the 15-minute window specified by the AWS SigV4 authentication specification. + +## Reference documentation + +For comprehensive documentation on AWS S3 commands and libraries, refer to: + +- [AWS CLI S3 reference](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html). +- [AWS S3 API reference](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html). +- [Boto3 S3 reference](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html).