Skip to content

Commit c77a6c0

Browse files
Shorten and streamline tutorial readme
1 parent 9df3896 commit c77a6c0

File tree

1 file changed

+26
-102
lines changed

1 file changed

+26
-102
lines changed

beta/serverless-fleets/README.md

Lines changed: 26 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -1,128 +1,50 @@
1-
# Serverless Fleets (beta)
1+
# Simplify and optimize large-scale parallel computation with Serverless Fleets
22

3-
Serverless Fleets is an new feature of IBM Cloud Code Engine.
3+
As artificial intelligence continues to grow and demand for cloud-based solutions increases, the ability to run large-scale, compute-intensive workloads both quickly and efficiently has become critical.
4+
5+
In this hands-on lab, you will deploy your first Serverless Fleet on IBM Code Engine—IBM’s strategic container platform designed to handle large-scale, compute-intensive workloads.
6+
7+
Using both the intuitive graphical user interface and the command line, you will be guided step by step through the process. With just three clicks, you will have a Serverless Fleet up and running on IBM Cloud.
48

59
**Table of Contents:**
10+
11+
- [Key differentiators of Fleets](#key-differentiators-of-fleets)
612
- [What is a fleets](#what-is-a-fleet)
7-
- [Why using a fleets](#why-using-a-fleet)
8-
- [The fleet concept](#the-fleet-concept)
9-
- [Fleet specification](#fleet-specification)
1013
- [Architecture](#architecture)
1114
- [One Time Setup](#one-time-setup)
1215
- [Launch a Fleet](#launch-a-fleet)
1316
- [Launch a Fleet with GPUs](#launch-a-fleet-with-gpus)
1417
- [Launch a fleet with parallel tasks](#launch-a-fleet-with-parallel-tasks)
1518
- [Launch a fleet to count words of novels](#launch-a-fleet-to-count-words-of-novels)
16-
- [Tutorials](#tutorials)
19+
- [Docling tutorial](./tutorials/docling/README.md)
20+
- [Batch Inferencing tutorial](./tutorials/inferencing/README.md)
21+
- [Monte Carlo Simulation tutorial](./tutorials/docling/README.md)
1722
- [HowTo](#howto)
1823
- [Troubleshooting](#troubleshooting)
1924

20-
## What is a fleet
21-
22-
A fleet, also serverless fleet, is a Code Engine compute component that runs one or more instances of user code in order to complete its tasks. Instances run on workers which are automatically provisioned and de-provisioned based on the number and resource requirements of the instances. Fleets can provision any machine type which connect to Virtual Private Clouds (VPCs) and securely interoperate with user data and services there.
2325

24-
## Why using a fleet
26+
## Key differentiators of Fleets
2527

26-
Fleets provide the following main key differentiators:
27-
1. Any machine type including GPUs
28-
2. Connected to the customers VPC network
29-
3. Large scale parallel computing without limits on vCPU, Memory and duration
30-
4. Dynamic task queuing to millions of tasks
28+
Fleets offer the following advantages:
29+
1. Support for large-scale parallel computing tasks, with no limits on vCPU, memory, or task duration.
30+
2. Automatic, dynamic scaling—from a single task to millions of tasks.
31+
3. Consumption-based pricing: pay only for the resources you use, with no idle or fixed costs.
32+
4. Fully managed service—no infrastructure administration required.
33+
5. Broad machine type support, including GPU-enabled instances.
34+
6. Seamless integration with your VPC network.
3135

32-
## The fleet concept
33-
34-
Fleets have three principal elements: tasks, instances and workers.
36+
## What is a fleet
3537

3638
![](./images/prototype_concept.png)
3739

38-
### Tasks
39-
40-
The tasks of a fleet represent the work that the fleet is intended to perform and are specified as part of the fleet specification at creation time.
41-
42-
To perform that work, the fleet starts instances of user code on behalf of the tasks and maintains a representation of the current task statuses as instances are started and ending. Tasks change their initial “pending” status to “running” as soon as an instance is started on their behalf. The task status changes to “succeeded” if its instance process ends with a successful return code. If the instance ends unsuccessfully the associated task changes to status “failed” unless its maximum number of retries is not yet exhausted. In that case the task status is set back to “pending” so that a new instance can be started on behalf of the task. A special task status is “cancelled” which applies if the fleet is cancelled by user action.
43-
44-
Once all tasks of a fleet have reached a final status, the fleet status also changes to a final status. The final fleet status is “succeeded” if all tasks have finished successfully, “failed” if at least one task failed and “cancelled” if the user has cancelled the fleet. Once a fleet has reached a final status, all instances have ended and all worker nodes are (being) de-provisioned - unless specific configuration settings change this behavior for debugging purposes.
45-
46-
Through the tasks specification users can control the number of tasks, the order in which instances are started and which specific command and arguments are used to start an instance for a task.
47-
48-
### Instances
49-
50-
Instances of user code are started for the fleet’s tasks on top of worker nodes. Each instance is started on behalf of exactly one task, its associated task. Different instances always have different associated tasks.
51-
52-
Fleets can work on many tasks in parallel by starting multiple instances concurrently. The maximum number of concurrent instances (max_scale) is part of the fleet’s specification. All instances are created with the same amount of vCPU and memory as per the fleet’s specification.
53-
54-
Instances run user code as per the fleet’s code specification in combination with task parameters that allow for task-specific start commands and arguments.
55-
56-
Instances terminate when the user code exits the instance process. The return code provided at that point signals whether the associated task was successfully completed (exit 0) or failed. The status of the associated task is updated accordingly and retries might be attempted as described in the preceding section on tasks.
57-
58-
Instances might also be stopped if the fleet is cancelled with the “hard stop” option by user action or by exceeding the maximum execution time.
59-
60-
### Workers
61-
62-
Worker nodes are virtual machines automatically provisioned and de-provisioned based on the number and amount of resources required to run the fleet’s instances.
40+
A fleet (also referred to as a serverless fleet) is a Code Engine compute resource that runs one or more instances of user code in parallel to process a large set of compute-intensive tasks.
6341

64-
Worker nodes are the basis for charging fleet resource consumption in terms of vCPU consumption, memory consumption and potential GPU uplifts.
42+
Fleets can connect to Virtual Private Clouds (VPCs) to securely access user data and services. They provide dynamic task queuing, single-tenant isolation, and support for GPU workloads.
6543

66-
Users can influence the selection of worker node machine profiles by defining minimum requirements for eligible machine profiles or even specifying a certain one.
44+
A fleet consists of a collection of worker nodes that automatically scale up or down based on resource requirements. Each instance runs on a worker node to complete a single task. When a task finishes, the worker node immediately starts the next task in the queue. This process continues until all tasks are completed, after which the worker nodes are automatically deprovisioned.
6745

68-
## Fleet specification
46+
Like applications, jobs, and functions, fleets run within a Code Engine project. A project is a grouping of Code Engine resources within a specific IBM Cloud region. Projects are used to organize resources and manage access to entities such as configmaps, secrets, and persistent data stores.
6947

70-
Fleets run as soon as they are created so that “running a fleet” is the same operation as “creating a fleet”. Therefore the CLI provides `fleet run` and `fleet create` as synonyms. When creating a fleet the following aspects are specified - either explicitly or by default:
71-
- name
72-
- code
73-
- tasks
74-
- instance resources and scaling
75-
- worker nodes
76-
- connectivity
77-
- environment variables (opt.)
78-
- data store mounts (opt.)
79-
80-
The default values are suitable in many cases so that running a fleet can be very easy and quick as shown in the examples section.
81-
82-
### Name specification
83-
84-
The fleet name identifies the fleet entity within the Code Engine project. It has to be a unique within fleets of the same Code Engine project, i.e. it might be the same as an app’s or job’s name in the same Code Engine project.
85-
86-
### Tasks specification
87-
88-
Fleets require at least one task and are designed to handle large number of tasks. There are two options to specify tasks:
89-
- number of tasks: N
90-
- tasks from file: <file>
91-
92-
Each tasks gets an index assigned from 0..N. The tasks index is provided as an environment variable `CE_TASK_INDEX` into the instance.
93-
94-
In order to specify tasks in a file create a text file with line-wise definition of parameters in JSON syntax (according to JSONL standard). The task parameters "command" and "args" can be used to override of the command and arguments when starting an instance of user code on behalf of the task. If one or both of these parameters are specified their values are used instead of the respective definitions in the container image or in the fleet's code specification. For example, see [wordcount_commands.jsonl](./wordcount_commands.jsonl)
95-
96-
### Code specification
97-
98-
The fleet’s code determines what is run in one or more instances in order to work on tasks. The specification has two parts: the base specification defines a container image reference and optional command and arguments overrides. (This is the same for Code Engine apps and jobs.). In addition, fleets can override command and arguments in a task-specific way as described in the “Task specification” section.
99-
100-
### Instance resources and scaling specifications
101-
102-
vCPU and memory required by each instance can be specified and determines how many instances can fit/run on a fleet worker. In addition, the maximum number of concurrent instances (max_scale) can be specified.
103-
104-
For example, if an instance requires 2 vCPU and 8 GB memory and a total of 100 instances should run concurrently, the fleet will provision a total of 200 vCPU and 800 GB memory.
105-
106-
### Worker specifications
107-
108-
Users can influence what machine profiles are used as worker nodes to different degrees.
109-
110-
In the example above, if the user selects a bx2-8x32 worker profile, each worker can run 4 instances. Therefore a total of 25 workers will be provisioned.
111-
112-
### Environment variables
113-
114-
The instance will get the following environment provided by the system:
115-
```
116-
CE_FLEET_VERSION=v1
117-
CE_REQUEST_ID=33af980d-8175-4925-85d0-0f0cf8812cb5
118-
CE_PROJECT_ID=e1501040-e56e-48b6-b9f0-1695908199bf
119-
CE_FLEET_CONCURRENCY=1
120-
CE_TASK_ID=0
121-
CE_USER_MOUNT_POINT=/mnt/ce/data
122-
CE_FLEET_KEEP_WORKER=false
123-
CE_FLEET_ID=33af980d-8175-4925-85d0-0f0cf8812cb5
124-
CE_FLEET_IS_GPU=false
125-
```
12648

12749
## Architecture
12850

@@ -593,6 +515,8 @@ Download the results from the output COS bucket to `./data/output`
593515
## Tutorials
594516
595517
- [Tutorial: Docling](./tutorials/docling/README.md)
518+
- [Tutorial: Inferencing](./tutorials/inferencing/README.md)
519+
- [Tutorial: Simulation](./tutorials/simulation/README.md)
596520
597521
598522
## HowTo

0 commit comments

Comments
 (0)