Skip to content

Commit e52c66c

Browse files
mudlermudler
authored
chore(docs/install.sh): image changes (#5354)
chore(docs): image changes Signed-off-by: Ettore Di Giacinto <[email protected]>
1 parent cb28aef commit e52c66c

File tree

4 files changed

+133
-61
lines changed

4 files changed

+133
-61
lines changed

‎README.md

+67-5
Original file line numberDiff line numberDiff line change
@@ -113,22 +113,84 @@ For more installation options, see [Installer Options](https://localai.io/docs/a
113113
Or run with docker:
114114

115115
### CPU only image:
116+
116117
```bash
117-
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-cpu
118+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
118119
```
119-
### Nvidia GPU:
120+
121+
### NVIDIA GPU Images:
122+
120123
```bash
124+
# CUDA 12.0 with core features
121125
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
126+
127+
# CUDA 12.0 with extra Python dependencies
128+
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12-extras
129+
130+
# CUDA 11.7 with core features
131+
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-11
132+
133+
# CUDA 11.7 with extra Python dependencies
134+
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-11-extras
135+
136+
# NVIDIA Jetson (L4T) ARM64
137+
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64
122138
```
123-
### CPU and GPU image (bigger size):
139+
140+
### AMD GPU Images (ROCm):
141+
124142
```bash
125-
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
143+
# ROCm with core features
144+
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas
145+
146+
# ROCm with extra Python dependencies
147+
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas-extras
148+
```
149+
150+
### Intel GPU Images (oneAPI):
151+
152+
```bash
153+
# Intel GPU with FP16 support
154+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel-f16
155+
156+
# Intel GPU with FP16 support and extra dependencies
157+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel-f16-extras
158+
159+
# Intel GPU with FP32 support
160+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel-f32
161+
162+
# Intel GPU with FP32 support and extra dependencies
163+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel-f32-extras
126164
```
127-
### AIO images (it will pre-download a set of models ready for use, see https://localai.io/basics/container/)
165+
166+
### Vulkan GPU Images:
167+
128168
```bash
169+
# Vulkan with core features
170+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan
171+
```
172+
173+
### AIO Images (pre-downloaded models):
174+
175+
```bash
176+
# CPU version
129177
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu
178+
179+
# NVIDIA CUDA 12 version
180+
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12
181+
182+
# NVIDIA CUDA 11 version
183+
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-11
184+
185+
# Intel GPU version
186+
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-gpu-intel-f16
187+
188+
# AMD GPU version
189+
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-aio-gpu-hipblas
130190
```
131191

192+
For more information about the AIO images and pre-downloaded models, see [Container Documentation](https://localai.io/basics/container/).
193+
132194
To load models:
133195

134196
```bash

‎docs/content/docs/advanced/installer.md

+10-1
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,9 @@ List of the Environment Variables:
2323
|----------------------|--------------------------------------------------------------|
2424
| **DOCKER_INSTALL** | Set to "true" to enable the installation of Docker images. |
2525
| **USE_AIO** | Set to "true" to use the all-in-one LocalAI Docker image. |
26+
| **USE_EXTRAS** | Set to "true" to use images with extra Python dependencies. |
27+
| **USE_VULKAN** | Set to "true" to use Vulkan GPU support. |
2628
| **API_KEY** | Specify an API key for accessing LocalAI, if required. |
27-
| **CORE_IMAGES** | Set to "true" to download core LocalAI images. |
2829
| **PORT** | Specifies the port on which LocalAI will run (default is 8080). |
2930
| **THREADS** | Number of processor threads the application should use. Defaults to the number of logical cores minus one. |
3031
| **VERSION** | Specifies the version of LocalAI to install. Defaults to the latest available version. |
@@ -34,6 +35,14 @@ List of the Environment Variables:
3435
| **FEDERATED** | Set to "true" to share the instance with the federation (p2p token is required see [documentation]({{%relref "docs/features/distributed_inferencing" %}})) |
3536
| **FEDERATED_SERVER** | Set to "true" to run the instance as a federation server which forwards requests to the federation (p2p token is required see [documentation]({{%relref "docs/features/distributed_inferencing" %}})) |
3637

38+
## Image Selection
39+
40+
The installer will automatically detect your GPU and select the appropriate image. By default, it uses the standard images without extra Python dependencies. You can customize the image selection using the following environment variables:
41+
42+
- `USE_EXTRAS=true`: Use images with extra Python dependencies (larger images, ~17GB)
43+
- `USE_AIO=true`: Use all-in-one images that include all dependencies
44+
- `USE_VULKAN=true`: Use Vulkan GPU support instead of vendor-specific GPU support
45+
3746
## Uninstallation
3847

3948
To uninstall, run:

‎docs/content/docs/getting-started/container-images.md

+17-25
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ The AIO Images are inheriting the same environment variables as the base images
150150

151151
Standard container images do not have pre-installed models.
152152

153-
Images are available with and without python dependencies. Note that images with python dependencies are bigger (in order of 17GB).
153+
Images are available with and without python dependencies (images with the `extras` suffix). Note that images with python dependencies are bigger (in order of 17GB).
154154

155155
Images with `core` in the tag are smaller and do not contain any python dependencies.
156156

@@ -160,10 +160,8 @@ Images with `core` in the tag are smaller and do not contain any python dependen
160160
| Description | Quay | Docker Hub |
161161
| --- | --- |-----------------------------------------------|
162162
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master` | `localai/localai:master` |
163-
| Latest tag | `quay.io/go-skynet/local-ai:latest-cpu` | `localai/localai:latest-cpu` |
163+
| Latest tag | `quay.io/go-skynet/local-ai:latest` | `localai/localai:latest` |
164164
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}` | `localai/localai:{{< version >}}` |
165-
| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-ffmpeg` | `localai/localai:{{< version >}}-ffmpeg` |
166-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-ffmpeg-core` | `localai/localai:{{< version >}}-ffmpeg-core` |
167165

168166
{{% /tab %}}
169167

@@ -172,10 +170,9 @@ Images with `core` in the tag are smaller and do not contain any python dependen
172170
| Description | Quay | Docker Hub |
173171
| --- | --- |-------------------------------------------------------------|
174172
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-cublas-cuda11` | `localai/localai:master-cublas-cuda11` |
175-
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-11` | `localai/localai:latest-gpu-nvidia-cuda-11` |
173+
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-11` | `localai/localai:latest-gpu-nvidia-cuda-11` |
174+
| Latest tag with extras | `quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-11-extras` | `localai/localai:latest-gpu-nvidia-cuda-11-extras` |
176175
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11` | `localai/localai:{{< version >}}-cublas-cuda11` |
177-
| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11-ffmpeg` | `localai/localai:{{< version >}}-cublas-cuda11-ffmpeg` |
178-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11-ffmpeg-core` | `localai/localai:{{< version >}}-cublas-cuda11-ffmpeg-core` |
179176

180177
{{% /tab %}}
181178

@@ -185,9 +182,8 @@ Images with `core` in the tag are smaller and do not contain any python dependen
185182
| --- | --- |-------------------------------------------------------------|
186183
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-cublas-cuda12` | `localai/localai:master-cublas-cuda12` |
187184
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-12` | `localai/localai:latest-gpu-nvidia-cuda-12` |
185+
| Latest tag with extras | `quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-12-extras` | `localai/localai:latest-gpu-nvidia-cuda-12-extras` |
188186
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12` | `localai/localai:{{< version >}}-cublas-cuda12` |
189-
| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12-ffmpeg` | `localai/localai:{{< version >}}-cublas-cuda12-ffmpeg` |
190-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12-ffmpeg-core` | `localai/localai:{{< version >}}-cublas-cuda12-ffmpeg-core` |
191187

192188
{{% /tab %}}
193189

@@ -197,9 +193,8 @@ Images with `core` in the tag are smaller and do not contain any python dependen
197193
| --- | --- |-------------------------------------------------------------|
198194
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-sycl-f16` | `localai/localai:master-sycl-f16` |
199195
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-intel-f16` | `localai/localai:latest-gpu-intel-f16` |
200-
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f16-core` | `localai/localai:{{< version >}}-sycl-f16-core` |
201-
| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f16-ffmpeg` | `localai/localai:{{< version >}}-sycl-f16-ffmpeg` |
202-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f16-ffmpeg-core` | `localai/localai:{{< version >}}-sycl-f16-ffmpeg-core` |
196+
| Latest tag with extras | `quay.io/go-skynet/local-ai:latest-gpu-intel-f16-extras` | `localai/localai:latest-gpu-intel-f16-extras` |
197+
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f16` | `localai/localai:{{< version >}}-sycl-f16` |
203198

204199
{{% /tab %}}
205200

@@ -209,9 +204,8 @@ Images with `core` in the tag are smaller and do not contain any python dependen
209204
| --- | --- |-------------------------------------------------------------|
210205
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-sycl-f32` | `localai/localai:master-sycl-f32` |
211206
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-intel-f32` | `localai/localai:latest-gpu-intel-f32` |
212-
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f32-core` | `localai/localai:{{< version >}}-sycl-f32-core` |
213-
| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f32-ffmpeg` | `localai/localai:{{< version >}}-sycl-f32-ffmpeg` |
214-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f32-ffmpeg-core` | `localai/localai:{{< version >}}-sycl-f32-ffmpeg-core` |
207+
| Latest tag with extras | `quay.io/go-skynet/local-ai:latest-gpu-intel-f32-extras` | `localai/localai:latest-gpu-intel-f32-extras` |
208+
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-sycl-f32` | `localai/localai:{{< version >}}-sycl-f32` |
215209

216210
{{% /tab %}}
217211

@@ -220,20 +214,18 @@ Images with `core` in the tag are smaller and do not contain any python dependen
220214
| Description | Quay | Docker Hub |
221215
| --- | --- |-------------------------------------------------------------|
222216
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-hipblas` | `localai/localai:master-hipblas` |
223-
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-hipblas` | `localai/localai:latest-gpu-hipblas` |
217+
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-hipblas` | `localai/localai:latest-gpu-hipblas` |
218+
| Latest tag with extras | `quay.io/go-skynet/local-ai:latest-gpu-hipblas-extras` | `localai/localai:latest-gpu-hipblas-extras` |
224219
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-hipblas` | `localai/localai:{{< version >}}-hipblas` |
225-
| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-hipblas-ffmpeg` | `localai/localai:{{< version >}}-hipblas-ffmpeg` |
226-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-hipblas-ffmpeg-core` | `localai/localai:{{< version >}}-hipblas-ffmpeg-core` |
227220

228221
{{% /tab %}}
229222

230-
231223
{{% tab tabName="Vulkan Images" %}}
232224
| Description | Quay | Docker Hub |
233225
| --- | --- |-------------------------------------------------------------|
234-
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai: master-vulkan-ffmpeg-core ` | `localai/localai: master-vulkan-ffmpeg-core ` |
235-
| Latest tag | `quay.io/go-skynet/local-ai: latest-vulkan-ffmpeg-core ` | `localai/localai: latest-vulkan-ffmpeg-core` |
236-
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-vulkan-fmpeg-core` | `localai/localai:{{< version >}}-vulkan-fmpeg-core` |
226+
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-vulkan` | `localai/localai:master-vulkan` |
227+
| Latest tag | `quay.io/go-skynet/local-ai:latest-gpu-vulkan` | `localai/localai:latest-gpu-vulkan` |
228+
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-vulkan` | `localai/localai:{{< version >}}-vulkan` |
237229
{{% /tab %}}
238230

239231
{{% tab tabName="Nvidia Linux for tegra" %}}
@@ -242,9 +234,9 @@ These images are compatible with Nvidia ARM64 devices, such as the Jetson Nano,
242234

243235
| Description | Quay | Docker Hub |
244236
| --- | --- |-------------------------------------------------------------|
245-
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core` | `localai/localai:master-nvidia-l4t-arm64-core` |
246-
| Latest tag | `quay.io/go-skynet/local-ai:latest-nvidia-l4t-arm64-core` | `localai/localai:latest-nvidia-l4t-arm64-core` |
247-
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-nvidia-l4t-arm64-core` | `localai/localai:{{< version >}}-nvidia-l4t-arm64-core` |
237+
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64` | `localai/localai:master-nvidia-l4t-arm64` |
238+
| Latest tag | `quay.io/go-skynet/local-ai:latest-nvidia-l4t-arm64` | `localai/localai:latest-nvidia-l4t-arm64` |
239+
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-nvidia-l4t-arm64` | `localai/localai:{{< version >}}-nvidia-l4t-arm64` |
248240

249241
{{% /tab %}}
250242

0 commit comments

Comments
 (0)