Bananas (#22)

Author: rachfop

docs/serverless/workers/handlers/handler-additional-controls.md

@@ -1,6 +1,6 @@
 ---
 title: "Additional controls"
-sidebar_position: 5
+sidebar_position: 6
 ---
 
 ## Update progress

docs/tutorials/migrations/banana/_category_.json

@@ -0,0 +1,11 @@
+{
+  "position": 1,
+  "label": "Banana",
+  "link": {
+    "type": "generated-index",
+    "title": "Bananas"
+  },
+  "customProps": {
+    "description": "Migrating from another service to RunPod, see our Overview page to get started."
+  }
+}

docs/tutorials/migrations/banana/images.md

@@ -0,0 +1,169 @@
+---
+title: Migrate Your Banana Images to RunPod
+sidebar_label: Migrate images
+draft: true
+---
+
+Migrating your AI models and applications from one cloud service to another can often present a challenge, especially when the two platforms operate differently. This tutorial aims to streamline the process of moving from Banana Dev to RunPod, focusing on transferring Docker-based applications and AI models. Whether you're shifting due to preferences in service offerings, pricing, or performance, this guide will help you through the transition smoothly.
+
+## Introduction
+
+Banana Dev provides an environment for deploying machine learning models easily, while RunPod offers robust and scalable serverless solutions. Transitioning between these platforms involves adapting your application to the new environment's requirements and deploying it effectively.
+
+Below, we'll walk through how to adapt a Python application from Banana Dev to RunPod, including necessary changes to your Dockerfile and deployment configurations.
+
+## Step 1: Understand Your Application
+
+First, take a comprehensive look at your current Banana Dev application. Our example application uses the `potassium` framework for serving a machine learning model:
+
+```python
+from io import BytesIO
+from potassium import Potassium, Request, Response
+from diffusers import DiffusionPipeline, DDPMScheduler
+import torch
+import base64
+
+# create a new Potassium app
+app = Potassium("my_app")
+
+# @app.init runs at startup, and loads models into the app's context
+@app.init
+def init():
+    repo_id="Meina/MeinaUnreal_V3"
+
+    ddpm = DDPMScheduler.from_pretrained(repo_id, subfolder="scheduler")
+    
+    model = DiffusionPipeline.from_pretrained(
+        repo_id, 
+        use_safetensors=True,
+        torch_dtype=torch.float16,
+        scheduler=ddpm
+    ).to("cuda")
+
+    context = {
+        "model": model,
+    }
+
+    return context
+
+# @app.handler runs for every call
+@app.handler()
+def handler(context: dict, request: Request) -> Response:
+    model = context.get("model")
+
+    prompt = request.json.get("prompt")
+    negative_prompt = "(worst quality, low quality:1.4), monochrome, zombie, (interlocked fingers), cleavage, nudity, naked, nude"
+
+    image = model(
+        prompt=prompt,
+        negative_prompt=negative_prompt,
+        guidance_scale=7,
+        num_inference_steps=request.json.get("steps", 30),
+        generator=torch.Generator(device="cuda").manual_seed(request.json.get("seed")) if request.json.get("seed") else None,
+        width=512,
+        height=512,
+    ).images[0]
+
+    buffered = BytesIO()
+    image.save(buffered, format="JPEG", quality=80)
+    img_str = base64.b64encode(buffered.getvalue())
+
+    return Response(
+        json = {"output": str(img_str, "utf-8")}, 
+        status=200
+    )
+
+if __name__ == "__main__":
+    app.serve()
+```
+
+This application initializes a BERT model for fill-mask tasks and serves it over HTTP.
+
+---
+
+## Step 2: Adapt Your Code for RunPod
+
+In RunPod, applications can be adapted to run in a serverless manner, which involves modifying your application logic to fit into the RunPod's handler function format. Below is an example modification that adapts our initial Banana Dev application to work with RunPod, using the `diffusers` library for AI model inference:
+
+```python
+import runpod
+from diffusers import AutoPipelineForText2Image
+import base64
+import io
+import time
+
+# If your handler runs inference on a model, load the model here.
+# You will want models to be loaded into memory before starting serverless.
+
+try:
+    pipe = AutoPipelineForText2Image.from_pretrained("meina/meinaunreal_v3")
+    pipe.to("cuda")
+except RuntimeError:
+    quit()
+
+def handler(job):
+    """ Handler function that will be used to process jobs. """
+    job_input = job['input']
+    prompt = job_input['prompt']
+
+    time_start = time.time()
+    image = pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0.0).images[0]
+    print(f"Time taken: {time.time() - time_start}")
+
+    buffer = io.BytesIO()
+    image.save(buffer, format="PNG")
+    image_bytes = buffer.getvalue()
+
+    return base64.b64encode(image_bytes).decode('utf-8')
+
+
+runpod.serverless.start({"handler": handler})
+```
+
+---
+
+This modification involves initializing your model outside of the handler function to ensure it's loaded into memory before processing jobs, a crucial step for efficient serverless execution.
+
+## Step 3: Update Your Dockerfile
+
+The Dockerfile must also be adapted for RunPod's environment. Here's a comparison between a typical Banana Dev Dockerfile and the adapted version for RunPod:
+
+### Banana Dev Dockerfile
+
+```dockerfile
+FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime
+...
+CMD python3 -u app.py
+```
+
+---
+
+### RunPod Dockerfile
+
+```dockerfile
+FROM runpod/base:0.4.0-cuda11.8.0
+
+CMD python3.11 -u /handler.py
+```
+
+---
+
+The key differences include the base image and the execution command, reflecting RunPod's requirements and Python version specifics.
+
+## Step 4: Deploy to RunPod
+
+Once your code and Dockerfile are ready, the next steps involve building your Docker image and deploying it on RunPod. This process typically involves:
+
+- Building your Docker image with the adapted Dockerfile.
+- Pushing the image to a container registry (e.g., DockerHub).
+- Creating a serverless function on RunPod and configuring it to use your Docker image.
+
+## Testing and Verification
+
+After deployment, thoroughly test your application to ensure it operates as expected within the RunPod environment. This may involve sending requests to your serverless endpoint and verifying the output.
+
+## Conclusion
+
+Migrating from Banana Dev to RunPod involves several key steps: adapting your application code, updating the Dockerfile, and deploying the adapted application on RunPod. By following this guide, you can make the transition smoother and take advantage of RunPod's serverless capabilities for your AI applications.
+
+Remember to review RunPod's documentation for specific details on serverless deployment and configuration options to optimize your application's performance and cost-efficiency.

docs/tutorials/migrations/banana/overview.md

@@ -0,0 +1,203 @@
+---
+title: Overview
+sidebar_position: 1
+description: Migrating from Banana to RunPod? Checkout our migration page to get started.
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+To get started with RunPod:
+
+- [Create a RunPod account](/get-started/manage-accounts)
+- [Add funds](/get-started/billing-information)
+- [Use the RunPod SDK](#setting-up-your-project) to build and connect with your Serverless Endpoints
+
+<details>
+<summary>
+
+**Quick migration with Docker**
+
+</summary>
+
+Transitioning from Banana to RunPod doesn't have to be a lengthy process.
+For users seeking a swift migration path while maintaining Banana's dependencies for the interim, the Docker approach provides an efficient solution.
+This method allows you to leverage Docker to encapsulate your environment, simplifying the migration process and enabling a smoother transition to RunPod.
+
+**Why consider the Dockerfile approach?**
+
+Utilizing a Dockerfile for migration offers a bridge between Banana and RunPod, allowing for immediate deployment of existing projects without the need to immediately discard Banana's dependencies. This approach is particularly beneficial for those looking to test or move their applications to RunPod with minimal initial adjustments.
+
+**Dockerfile**
+
+The provided Dockerfile outlines a straightforward process for setting up your application on RunPod.
+
+Add this Dockerfile to your project.
+
+```dockerfile
+FROM runpod/banana:peel as bread
+FROM repo/image:tag
+
+RUN pip install runpod
+
+COPY --from=bread /handler.py .
+COPY --from=bread /start.sh .
+
+RUN chmod +x start.sh
+CMD ["./start.sh"]
+```
+
+**Building and deploying**
+
+After creating your Dockerfile, build your Docker image and deploy it to RunPod.
+This process involves using Docker commands to build the image and then deploying it to RunPod.
+
+**Advantages and considerations**
+
+This Dockerfile approach expedites the migration process, allowing you to leverage RunPod's powerful features with minimal initial changes to your project.
+It's an excellent way to quickly transition and test your applications on RunPod.
+
+However, while this method facilitates a quick start on RunPod, it's advisable to plan for a future migration away from Banana's dependencies, as there is overhead to building Banana's dependencies and deploying them to RunPod.
+
+Gradually adapting your project to utilize RunPod's native features and services will optimize your application's performance and scalability.
+
+**Moving forward**
+
+Once you've migrated your application using the Docker approach, consider exploring RunPod's full capabilities.
+Transitioning away from Banana's dependencies and fully integrating with RunPod's services will allow you to take full advantage of what RunPod has to offer.
+
+This quick migration guide is just the beginning.
+Continue with the rest of our tutorial to learn how to leverage RunPod's features to their fullest and ensure your project is fully adapted to its new environment.
+
+</details>
+
+The rest of this guide will help you set up a RunPod project.
+
+## Setting up your project
+
+Just like with Banana, RunPod provides a Python SDK to run your projects.
+
+To get started, install setup a virtual environment then install the SDK library.
+
+<Tabs>
+  <TabItem value="macos" label="macOS" default>
+
+Create a Python virtual environment with venv:
+
+    ```command
+    python3 -m venv env
+    source env/bin/activate
+    ```
+
+</TabItem>
+  <TabItem value="windows" label="Windows">
+
+Create a Python virtual environment with venv:
+
+    ```command
+    python -m venv env
+    env\Scripts\activate
+    ```
+
+</TabItem>
+
+</Tabs>
+
+To install the SDK, run the following command from the terminal.
+
+```command
+python -m pip install runpod
+```
+
+## Project examples
+
+RunPod provides a [repository of templates for your project](https://github.com/runpod-workers).
+
+You can use the template to get started with your project.
+
+```command
+gh repo clone runpod-workers/worker-template
+```
+
+Now that you've got a basic RunPod Worker template created:
+
+- Continue reading to see how you'd migrate from Banana to RunPod
+- See [Generate SDXL Turbo](/tutorials/serverless/generate-sdxl-turbo) for a general approach on deploying your first Serverless Endpoint with RunPod.
+
+## Project structure
+
+When beginning to migrate your Banana monorepo to RunPod, you will need to understand the structure of your project.
+
+<Tabs>
+
+<TabItem value="banana" label="Banana" default>
+
+Banana is a monorepo that contains multiple services. The basic structure for Banana projects is aligned with the RunPod Serverless projects for consistency:
+
+```text
+.
+├── Dockerfile               # Docker configuration
+├── README.md                # Project documentation
+├── banana_config.json       # Configuration settings
+├── requirements.txt         # Dependencies
+└── src
+    ├── app.py               # Main application code
+    └── download.py          # Download script
+```
+
+</TabItem>
+  <TabItem value="runpod" label="RunPod">
+
+RunPod Serverless is a monorepo that contains multiple services.
+
+```text
+.
+├── Dockerfile               # Docker configuration
+├── LICENSE                  # License information
+├── README.md                # Project documentation
+├── builder
+│   ├── requirements.txt     # Dependencies
+│   └── setup.sh             # Setup script
+└── src
+    └── handler.py           # Main handler code
+```
+
+</TabItem>
+</Tabs>
+
+Both project setups at a minimum contain:
+
+- `Dockerfile`: Defines the container for running the application.
+- Application code: The executable code within the container.
+
+Optional files included in both setups:
+
+- `requirements.txt`: Lists dependencies needed for the application.
+
+### Banana Configuration settings
+
+Banana configuration settings are stored in a `banana_config.json` file.
+
+Banana uses a `banana_config.json` file which contains things like Idle Timeout, Inference Timeout, and Max Replicas.
+
+**Idle Timeout**
+
+RunPod allows you to set an [Idle Timeout](/serverless/references/endpoint-configurations#idle-timeout) when creating the Endpoint.
+The default value is 5 seconds.
+
+**Inference Timeout**
+
+RunPod has a similar concept to Inference Timeout.
+For runs that are take less than 30 seconds to execute, you should use the `run_sync` handler.
+For runs that take longer than 30 seconds to execute, you should use the `sync` handler.
+
+**Max Replicas**
+
+When creating a Worker in RunPod, you can set the max Workers that will scale up depending on the amount of Worker sent to your Endpoint.
+For more informaiton, see [Scale Type](/serverless/references/endpoint-configurations#scale-type)
+
+:::note
+
+When creating a Worker, select the **Flashboot** option to optimize your startup time.
+
+:::