[Bug]: Endless looping in the Queue for the AI KB upon initial installation with a Tesla P4 GPU

[glabifrons]

Issue Category

Installation/Setup

Bug Description

Specs:

• i7-8700
• 16GB DDR4 RAM (soon to be 32GB)
• Tesla P4 8GB GPU
• 1TB SATA SSD
• Fresh Ubuntu Server 24.04 minimal installation with nouveau driver blocked and nvidia-driver-580-server nvidia-utils-580-server installed for GPU/CUDA use.

Background and setup:

I just installed Project NOMAD on a system with the above specs, went through and selected the "additional content" I wanted for the knowledge base, and let it download and install everything.
I saw that it had errors at the bottom about the LLM models not downloading due to missing ollama, so I waited for the ollama docker container to be installed, then went through and re-selected the models.
The models downloaded successfully, then it started processing the knowledge base.

Problem:

This is where it fell into a loop. For a few hours, it looped on 16 of them, repeatedly flagging them as "stalled" and retrying. Eventually it got down to 14 that are still in queue.
I'm not saying it's taking a long time to process them, it's looping over the same ones repeatedly. I watched as it cycled the entire list at least 4 times. Oddly, sometimes it retries ones that appeared to be successful (I copied, control-F, pasted to verify I wasn't misreading similar ones).

My questions:

1. Where can I find logs to see what it's doing/what's failing?
2. How can I adjust the timeout, as it is retrying these awfully quickly (40 seconds to 2 minutes). I'm guessing it is expecting that if it sees a Tesla, it must be faster than an RTX 5090 or something. It's only a P4. Some of these are huge (like Gutenberg) and will take some time on even a really powerful GPU.
3. Why would it reprocess successful ones?
4. Why do the numbers not match up? (28 stalled vs 14 waiting)

Steps to Reproduce

1. Install Tesla P4 (or equivalent?)
2. Install Ubuntu Server 24.04 using the entire disk.
   Be sure to configure the bulk of the space in /opt as that's the target installation directory for this software and all its data.
3. Launch the installer script.
4. Select all "additional content", Comprehensive level for each.
5. Select all 3 AI model offerings.
6. Leave Auto-Index at default "Yes, always".
7. Complete Setup.
8. Click AI Assistant from Home Page
9. Click "Knowledge Base" at the bottom.
10. Observe Processing Queue repeatedly looping over the same failures, rather than waiting for them to complete.

Expected Behavior

At step 10 above, it should wait a more reasonable period of time for the LLM to ingest the data, or it should simply fail and not loop over them endlessly.
Alternately, a log of what actually failed or a more detailed explanation should be provided.

Actual Behavior

At step 10 above, it keeps looping over the same files trying to index them.
Example failure message:

electronics.stackexchange.com_en_all_2026-02.zim
Indexed
⚠ Only 5,154 of est. 398,870 chunks embedded — ingestion may have stalled.

N.O.M.A.D. Version

v1.32.0

Operating System

Ubuntu 24.04

Docker Version

Docker version 29.5.2, build 79eb04c

Do you have a dedicated GPU?

Yes

GPU Model (if applicable)

NVIDIA Tesla P4

System Specifications

• i7-8700
• 16GB DDR4 RAM (soon to be 32GB)
• Tesla P4 8GB GPU
• 1TB SATA SSD
• Fresh Ubuntu Server 24.04 minimal installation with nouveau driver blocked and nvidia-driver-580-server nvidia-utils-580-server installed for GPU/CUDA use.

Service Status (if relevant)

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
776d27e6674e ollama/ollama:0.18.1 "/bin/ollama serve" 6 hours ago Up 6 hours 0.0.0.0:11434->11434/tcp, [::]:11434->11434/tcp nomad_ollama
ebf3fa6c1569 ghcr.io/gchq/cyberchef:10.22.1 "/docker-entrypoint.…" 6 hours ago Up 6 hours 0.0.0.0:8100->80/tcp, [::]:8100->80/tcp nomad_cyberchef
896eab1cf204 ghcr.io/kiwix/kiwix-serve:3.8.1 "/usr/bin/dumb-init …" 6 hours ago Up 6 hours 80/tcp, 0.0.0.0:8090->8080/tcp, [::]:8090->8080/tcp nomad_kiwix_server
97496a567743 qdrant/qdrant:v1.16 "./entrypoint.sh" 6 hours ago Up 6 hours 0.0.0.0:6333-6334->6333-6334/tcp, [::]:6333-6334->6333-6334/tcp nomad_qdrant
ad466efa9fe4 ghcr.io/crosstalk-solutions/project-nomad:latest "/usr/local/bin/entr…" 6 hours ago Up 6 hours (healthy) 0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp nomad_admin
8a258113a07f redis:7-alpine "docker-entrypoint.s…" 6 hours ago Up 6 hours (healthy) 6379/tcp nomad_redis
19934d138285 mysql:8.0 "docker-entrypoint.s…" 6 hours ago Up 6 hours (healthy) 3306/tcp, 33060/tcp nomad_mysql
b0028f5ffaa9 amir20/dozzle:v10.0 "/dozzle" 6 hours ago Up 6 hours 0.0.0.0:9999->8080/tcp, [::]:9999->8080/tcp nomad_dozzle
aca2b5da33a7 ghcr.io/crosstalk-solutions/project-nomad-disk-collector:latest "/usr/local/bin/coll…" 6 hours ago Up 6 hours nomad_disk_collector
efe8802807cb ghcr.io/crosstalk-solutions/project-nomad-sidecar-updater:latest "/usr/local/bin/upda…" 6 hours ago Up 6 hours nomad_updater

Relevant Logs

The output of docker logs nomad_ollama is 184891 lines long.
Stripping it down to just stderr:

$ sudo docker logs nomad_ollama > /dev/null
time=2026-05-25T20:25:07.950Z level=INFO source=routes.go:1727 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:true OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-05-25T20:25:07.951Z level=INFO source=routes.go:1729 msg="Ollama cloud disabled: true"
time=2026-05-25T20:25:07.952Z level=INFO source=images.go:477 msg="total blobs: 0"
time=2026-05-25T20:25:07.952Z level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-05-25T20:25:07.953Z level=INFO source=routes.go:1782 msg="Listening on [::]:11434 (version 0.18.1)"
time=2026-05-25T20:25:07.953Z level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-05-25T20:25:07.958Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46471"
time=2026-05-25T20:25:08.230Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 40425"
time=2026-05-25T20:25:08.651Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled.  To enable, set OLLAMA_VULKAN=1"
time=2026-05-25T20:25:08.652Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 42137"
time=2026-05-25T20:25:08.652Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46463"
time=2026-05-25T20:25:09.047Z level=INFO source=types.go:42 msg="inference compute" id=GPU-db02c40d-7046-9f7f-0f95-46e884819130 filter_id="" library=CUDA compute=6.1 name=CUDA0 description="Tesla P4" libdirs=ollama,cuda_v12 driver=13.0 pci_id=0000:01:00.0 type=discrete total="7.5 GiB" available="7.4 GiB"
time=2026-05-25T20:25:09.047Z level=INFO source=routes.go:1832 msg="vram-based default context" total_vram="7.5 GiB" default_num_ctx=4096
time=2026-05-25T20:25:10.219Z level=INFO source=download.go:179 msg="downloading 970aa74c0a90 in 3 100 MB part(s)"
time=2026-05-25T20:25:25.334Z level=INFO source=download.go:179 msg="downloading c71d239df917 in 1 11 KB part(s)"
time=2026-05-25T20:25:26.480Z level=INFO source=download.go:179 msg="downloading ce4a164fc046 in 1 17 B part(s)"
time=2026-05-25T20:25:27.617Z level=INFO source=download.go:179 msg="downloading 31df23ea7daa in 1 420 B part(s)"
time=2026-05-25T20:25:29.276Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 40521"
time=2026-05-25T20:25:29.550Z level=WARN source=cpu_linux.go:130 msg="failed to parse CPU allowed micro secs" error="strconv.ParseInt: parsing \"max\": invalid syntax"
time=2026-05-25T20:25:29.562Z level=WARN source=server.go:168 msg="requested context size too large for model" num_ctx=8192 n_ctx_train=2048
time=2026-05-25T20:25:29.562Z level=WARN source=server.go:208 msg="flash attention enabled but not supported by model"
time=2026-05-25T20:25:29.562Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-970aa74c0a90ef7482477cf803618e776e173c007bf957f635f1015bfcfef0e6 --port 39945"
time=2026-05-25T20:25:29.563Z level=INFO source=sched.go:489 msg="system memory" total="15.4 GiB" free="14.9 GiB" free_swap="4.0 GiB"
time=2026-05-25T20:25:29.563Z level=INFO source=sched.go:496 msg="gpu memory" id=GPU-db02c40d-7046-9f7f-0f95-46e884819130 library=CUDA available="7.0 GiB" free="7.4 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-05-25T20:25:29.563Z level=INFO source=server.go:757 msg="loading model" "model layers"=13 requested=-1
time=2026-05-25T20:25:29.574Z level=INFO source=runner.go:1411 msg="starting ollama engine"
time=2026-05-25T20:25:29.574Z level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:39945"
time=2026-05-25T20:25:29.584Z level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Disabled KvSize:2048 KvCacheType: NumThreads:6 GPULayers:13[ID:GPU-db02c40d-7046-9f7f-0f95-46e884819130 Layers:13(0..12)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-05-25T20:25:29.588Z level=INFO source=ggml.go:136 msg="" architecture=nomic-bert file_type=F16 name=nomic-embed-text-v1.5 description="" num_tensors=112 num_key_values=25
load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-haswell.so
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: Tesla P4, compute capability 6.1, VMM: yes, ID: GPU-db02c40d-7046-9f7f-0f95-46e884819130
load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so
time=2026-05-25T20:25:29.918Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
time=2026-05-25T20:25:29.920Z level=WARN source=runner.go:1219 msg="model does not support caching, setting batch size to context length" batch_size=2048
time=2026-05-25T20:25:30.023Z level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Disabled KvSize:2048 KvCacheType: NumThreads:6 GPULayers:13[ID:GPU-db02c40d-7046-9f7f-0f95-46e884819130 Layers:13(0..12)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-05-25T20:25:30.030Z level=WARN source=runner.go:1219 msg="model does not support caching, setting batch size to context length" batch_size=2048
time=2026-05-25T20:25:30.034Z level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Disabled KvSize:2048 KvCacheType: NumThreads:6 GPULayers:13[ID:GPU-db02c40d-7046-9f7f-0f95-46e884819130 Layers:13(0..12)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-05-25T20:25:30.034Z level=INFO source=ggml.go:482 msg="offloading 12 repeating layers to GPU"
time=2026-05-25T20:25:30.034Z level=INFO source=ggml.go:489 msg="offloading output layer to GPU"
time=2026-05-25T20:25:30.034Z level=INFO source=ggml.go:494 msg="offloaded 13/13 layers to GPU"
time=2026-05-25T20:25:30.034Z level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="260.9 MiB"
time=2026-05-25T20:25:30.034Z level=INFO source=device.go:245 msg="model weights" device=CPU size="44.7 MiB"
time=2026-05-25T20:25:30.034Z level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="250.0 MiB"
time=2026-05-25T20:25:30.034Z level=INFO source=device.go:267 msg="compute graph" device=CPU size="6.0 MiB"
time=2026-05-25T20:25:30.034Z level=INFO source=device.go:272 msg="total memory" size="561.6 MiB"
time=2026-05-25T20:25:30.034Z level=INFO source=sched.go:565 msg="loaded runners" count=1
time=2026-05-25T20:25:30.034Z level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-05-25T20:25:30.035Z level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model"
time=2026-05-25T20:25:30.287Z level=INFO source=server.go:1388 msg="llama runner started in 0.72 seconds"
time=2026-05-25T20:25:52.163Z level=INFO source=download.go:179 msg="downloading 667b0c1932bc in 16 307 MB part(s)"
time=2026-05-25T20:25:52.256Z level=INFO source=download.go:179 msg="downloading e6a7edc1a4d7 in 16 326 MB part(s)"
time=2026-05-25T20:28:57.620Z level=INFO source=download.go:297 msg="667b0c1932bc part 2 attempt 0 failed: unexpected EOF, retrying in 1s"
time=2026-05-25T20:28:58.956Z level=INFO source=download.go:297 msg="667b0c1932bc part 7 attempt 0 failed: unexpected EOF, retrying in 1s"
time=2026-05-25T20:29:05.007Z level=INFO source=download.go:297 msg="e6a7edc1a4d7 part 12 attempt 0 failed: unexpected EOF, retrying in 1s"
time=2026-05-25T20:29:08.705Z level=INFO source=download.go:297 msg="667b0c1932bc part 14 attempt 0 failed: unexpected EOF, retrying in 1s"
time=2026-05-25T20:29:41.995Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:43.632Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:43.633Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:43.634Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:43.659Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:45.293Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:45.294Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:29:45.294Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T20:30:08.410Z level=INFO source=download.go:179 msg="downloading c5ad996bda6e in 1 556 B part(s)"
time=2026-05-25T20:30:09.540Z level=INFO source=download.go:179 msg="downloading 6e4c38e1172f in 1 1.1 KB part(s)"
time=2026-05-25T20:30:10.687Z level=INFO source=download.go:179 msg="downloading ed8474dc73db in 1 179 B part(s)"
time=2026-05-25T20:30:11.825Z level=INFO source=download.go:179 msg="downloading f64cd5418e4b in 1 487 B part(s)"
time=2026-05-25T20:30:50.346Z level=INFO source=download.go:179 msg="downloading 948af2743fc7 in 1 1.5 KB part(s)"
time=2026-05-25T20:30:51.468Z level=INFO source=download.go:179 msg="downloading 0ba8f0e314b4 in 1 12 KB part(s)"
time=2026-05-25T20:30:52.612Z level=INFO source=download.go:179 msg="downloading 56bb8bd477a5 in 1 96 B part(s)"
time=2026-05-25T20:30:53.734Z level=INFO source=download.go:179 msg="downloading 455f34728c9b in 1 487 B part(s)"
time=2026-05-25T21:23:07.289Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T21:23:08.022Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T21:23:08.904Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T21:23:09.065Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:13.272Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:13.556Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:14.155Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:14.156Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:17.477Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:18.257Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-25T22:08:18.560Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T00:34:47.910Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T00:34:48.177Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T00:34:49.435Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T00:34:49.721Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T01:06:08.251Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T01:06:08.871Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T01:06:09.714Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T01:06:09.759Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"
time=2026-05-26T01:49:15.128Z level=INFO source=server.go:1751 msg="llm embedding error: the input length exceeds the context length"



Example of stdout:

[GIN] 2026/05/26 - 02:05:12 | 200 |  783.813431ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |  698.322026ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |  633.842973ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   35.583509ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   57.009852ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   51.317261ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   49.177943ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   48.442966ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   83.603365ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   97.445177ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   73.962664ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   99.274263ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   60.284624ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   39.331737ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   93.052854ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:13 | 200 |   72.275466ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:14 | 200 |  372.966868ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:14 | 200 |  374.097772ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:14 | 200 |   38.798687ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:15 | 200 |  926.498486ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:15 | 200 |  919.471364ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:16 | 200 |  1.006280934s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:16 | 200 |  1.579748888s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:17 | 200 |  1.106880854s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:17 | 200 |  1.099438578s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:18 | 200 |  795.490132ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:18 | 200 |  320.876255ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:18 | 200 |  298.214845ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:19 | 200 |  818.440433ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:20 | 200 |  1.471111149s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:20 | 200 |  1.519647799s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:20 | 200 |   730.74252ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:21 | 200 |  669.893322ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:22 | 200 |  1.359322095s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:22 | 200 |  1.067150928s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:22 | 200 |  294.288455ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:22 | 200 |   32.902787ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:22 | 200 |  212.685277ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:22 | 200 |  186.114599ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:23 | 200 |  343.843721ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:23 | 200 |  347.991567ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:23 | 200 |  471.058778ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:23 | 200 |  477.709655ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:23 | 200 |  363.311842ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:23 | 200 |  368.504902ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:24 | 200 |  294.310664ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:24 | 200 |  299.176252ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:24 | 200 |  144.112675ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:24 | 200 |  152.143423ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:25 | 200 |  647.018144ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:25 | 200 |  979.752218ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:25 | 200 |  453.778582ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  600.467696ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  605.824594ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  113.734585ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  372.252476ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  372.951206ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |   99.571764ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  104.159417ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  218.139163ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |  227.066015ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |   38.527705ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:26 | 200 |   34.043867ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |  100.510605ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |  118.925644ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |   379.04004ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |  362.728973ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |   55.245897ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |   63.013765ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:27 | 200 |   67.893136ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:28 | 200 |  948.809076ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:28 | 200 |  954.052706ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:29 | 200 |  1.067235818s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:29 | 200 |  1.017074816s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:30 | 200 |   1.01363679s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:30 | 200 |  1.074877691s |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:31 | 200 |    953.4328ms |      172.18.0.4 | POST     "/api/embed"
[GIN] 2026/05/26 - 02:05:31 | 200 |  1.011316696s |      172.18.0.4 | POST     "/api/embed"

Browser Console Errors (if UI issue)

Screenshots

Additional Context

No response

Pre-submission Checklist

• I have searched for existing issues that might be related to this bug
• I am running the latest version of Project N.O.M.A.D. (or have noted my version above)
• I have redacted any personal or sensitive information from logs and screenshots
• This issue is NOT related to running N.O.M.A.D. on an unsupported/non-Debian-based OS

[FORCEGC]

I have a tesla p4 and its installed in a r720 server. I have it all running in dockge but i run into the exact same issue you are. But mine wont even load that lower stored area. i get "File warnings unavailable — couldn't read storage state. Retrying…" I also get at the top "Knowledge Base unavailable: The Qdrant vector database is offline." I am unable to start the qdrant part. Is yours at least working?