Merge pull request #196 from SharpAI/fix/ssd-streaming-followups-2

fix(ssd): stream the directory we load, resume partial first-run downloads, keep exiting events on their own line

Author: solderzzc

README.md

@@ -69,7 +69,7 @@ The first SwiftLM numbers from a **32 GB** Mac. Every other table in this README
 | `Qwen3.6-35B-A3B-UD-MLX-4bit` | 21.6 GB | `--stream-experts` | 13.2 tok/s | 40.8K tokens | 5.8 GB |
 | `Qwen3.8-27B-4bit` (dense) | 11.3 GB | GPU | 9.3 tok/s | 40.8K tokens | 18.4 GB |
 | `gemma-4-26b-a4b-it-8bit` | ~26 GB | GPU | swaps (+3.1 GB on the first prompt) | — | — |
-| `gemma-4-26b-a4b-it-8bit` | ~26 GB | `--stream-experts` | 8.8 tok/s | 9.5K tokens (32K swapped) | 7.6 GB |
+| `gemma-4-26b-a4b-it-8bit` | ~26 GB | `--stream-experts` | 9.1 tok/s | 9.5K tokens (32K swapped) | 7.3 GB |
 
 - **MoE models are the sweet spot at 32 GB.** Only the active experts are read for each token, so they decode 5–6× faster than a dense 27B. A 4-bit MoE with up to about 22 GB of weights runs entirely on the GPU.
 - **Qwen3.6-35B-A3B on a base M6 reaches 76%** of the M1 Ultra 64 GB decode speed below (47.0 vs 61.7 tok/s).
@@ -90,14 +90,16 @@ Every needle check passed. `--mtp` with the bf16 assistant (`gemma-4-26B-A4B-it-
 
 ### Qwen3.6-35B-A3B 4-bit — GPU vs SSD streaming
 
+![Qwen3.6-35B-A3B with --stream-experts on release b782: 13.6 tok/s decode, 5.1 GB peak, no swap, on a base Mac mini M6 32 GB](docs/profiling/m6/media/m6_qwen36_35b_a3b_ssd_stream.gif)
+
 | Prompt tokens | GPU prefill / decode (tok/s) | GPU peak | `--stream-experts` prefill / decode (tok/s) | SSD peak |
 |---|---|---|---|---|
 | ~550 | 714 / 47.0 | 19.8 GB | 256 / 13.2 | 5.6 GB |
 | ~2.3K | 968 / 45.7 | 20.1 GB | 403 / 13.0 | 5.6 GB |
 | ~9.8K | 858 / 43.4 | 20.4 GB | 401 / 12.7 | 5.6 GB |
 | 40.8K | 615 / 36.1 | 21.5 GB | 336 / 12.0 | 5.8 GB |
 
-> ⚠️ **`--stream-experts` crashes on Qwen3.5/3.6 in releases b769 and b773** (`broadcast_shapes … (N,8,8,2048)` on the first request). The mlx-swift-lm upstream sync in #167 broke the SSD path. Earlier versions of this table were measured before that sync and were never re-checked afterwards. Fixed in SharpAI/mlx-swift-lm#69 and #71; the table above was re-measured with those fixes.
+> ⚠️ **`--stream-experts` crashes on quantized MoE models in releases b769 and b773** (`broadcast_shapes … (N,8,8,D)` on the first request). Reproduced on M6 with Qwen3.6-35B-A3B and Gemma 4 26B-A4B. The mlx-swift-lm upstream sync in #167 broke the SSD path. Earlier versions of this table were measured before that sync and were never re-checked afterwards. Fixed in SharpAI/mlx-swift-lm#69 and #71; the table above was re-measured with those fixes.
 
 ### Qwen3.8-27B-4bit (dense)
 
@@ -121,7 +123,7 @@ Every needle check passed. `--mtp` with the bf16 assistant (`gemma-4-26B-A4B-it-
 1. **The MLX buffer cache was unbounded on full-GPU loads.** It could grow to the whole 26.8 GB working set. It is now sized from the RAM left after weights and KV.
 2. **The KV-cache estimate counted every layer as full attention.** Gemma 4 (25 of 30 layers use a 1,024-token sliding window) was overestimated 10×, and Qwen3.5/3.8 (48 of 64 layers are linear attention) 4×. On 32 GB that pushed Gemma into CPU/GPU layer partitioning, which crashed with a Metal GPU timeout.
 3. **An auto-detected VLM that failed to load exited the server.** `Qwen3.6-35B-A3B-UD-MLX-4bit` ships a `preprocessor_config.json` without `image_mean`. SwiftLM now falls back to text-only unless you pass `--vision`.
-4. **Vision-capable models skipped chunked prefill.** On the older mlx-swift-lm pin, a text-only prompt on the VLM path ran through the model in a single pass. It's fixed by the mlx-swift-lm bump in #167. Every number in this section was measured on `main` after that bump; the Qwen3.6 table was re-measured with SharpAI/mlx-swift-lm#69 and #71.
+4. **Vision-capable models skipped chunked prefill.** On the older mlx-swift-lm pin, a text-only prompt on the VLM path ran through the model in a single pass. It's fixed by the mlx-swift-lm bump in #167. Every number in this section was measured on `main` after that bump. The `--stream-experts` rows (Qwen3.6 and Gemma 4 8-bit) were re-measured with SharpAI/mlx-swift-lm#69 and #71, because b769/b773 crash in that mode.
 
 > ℹ️ **`--turbo-kv` long-range recall is fixed** ([#175](https://github.com/SharpAI/SwiftLM/issues/175), SharpAI/mlx-swift-lm#65). Before the fix, once a prompt passed the 2,048-token compression threshold, attention only saw the recent hot window and positions restarted, so Qwen3.8-27B-4bit got exact lookups wrong. Attention now covers the compressed history too, which makes `--turbo-kv` slower than before (97 s vs 72 s on an 11.8K-token prompt on the M6). `--turbo-kv` still has no effect when `--ctx-size` is set (the attention layers use `RotatingKVCache`).
 >

Sources/SwiftLM/Server.swift

@@ -281,6 +281,21 @@ final class ProgressTracker {
     init(modelId: String) {
         self.modelId = modelId
     }
+
+    /// True while a `\r` progress bar is on screen without its closing newline.
+    nonisolated(unsafe) static var barOpen = false
+
+    /// Stops redrawing and ends the bar's line, so later output (and the `exiting`
+    /// event, which the daemon parses per line) starts on a fresh line.
+    func finish() {
+        isDone = true
+        trackingTask?.cancel()
+        if Self.barOpen {
+            print("")
+            fflush(stdout)
+            Self.barOpen = false
+        }
+    }
 
     func getDownloadedBytes() -> Int64 {
         let home = FileManager.default.homeDirectoryForCurrentUser
@@ -358,11 +373,14 @@ final class ProgressTracker {
 
                     let msg = String(format: "\r[SwiftLM] Download: [%@] %@ %@ (%@ MB / %@ MB) %@", bars, pctStr, spinner, completedMB, totalMB, speedText)
 
+                    if self.isDone { break }
                     print(msg.padding(toLength: 100, withPad: " ", startingAt: 0), terminator: "")
                     fflush(stdout)
-                    
+                    Self.barOpen = true
+
                     if fraction >= 1.0 {
                         print("")
+                        Self.barOpen = false
                         self.isDone = true
                         break
                     }
@@ -398,6 +416,10 @@ func emitEvent(_ payload: [String: Any]) {
             Data("[SwiftLM] failed to encode event for stdout: \(payload)\n".utf8))
         return
     }
+    if ProgressTracker.barOpen {
+        print("")
+        ProgressTracker.barOpen = false
+    }
     print(json)
     fflush(stdout)
 }
@@ -644,6 +666,13 @@ struct MLXServer: AsyncParsableCommand {
         var modelDirectory =
             ModelStorage.validatedContentDirectory(for: modelId)
             ?? resolveModelDirectory(modelId: modelId)
+        // resolveModelDirectory doesn't check the weights are there; streaming must not be
+        // activated for an empty or partial snapshot the loader won't read.
+        if self.streamExperts, let dir = modelDirectory,
+            !ModelStorage.validateLocalModelDirectory(dir)
+        {
+            modelDirectory = nil
+        }
         if self.streamExperts, !self.info, modelDirectory == nil,
             !FileManager.default.fileExists(atPath: modelId)
         {
@@ -654,22 +683,30 @@ struct MLXServer: AsyncParsableCommand {
                     .appendingPathComponent("MLX", isDirectory: true)
                     .appendingPathComponent("HuggingFace", isDirectory: true))
             let localRepo = hub.localRepoLocation(Hub.Repo(id: modelId))
-            if FileManager.default.fileExists(
-                atPath: localRepo.appendingPathComponent("config.json").path)
+            // Every shard must be present: an interrupted download (or the config.json
+            // the architecture probe fetches) would otherwise plan with a partial size.
+            if FileManager.default.fileExists(atPath: localRepo.path),
+                ModelStorage.validateLocalModelDirectory(localRepo)
             {
                 modelDirectory = localRepo
             } else {
                 // First run. A failed download is a model problem, not a binary one.
                 phase = .architectureProbe
                 print("[SwiftLM] --stream-experts: downloading \(modelId) before loading...")
                 let prefetchTracker = ProgressTracker(modelId: modelId)
+                defer { prefetchTracker.finish() }
                 modelDirectory = try await hub.snapshot(
                     from: modelId, matching: ["*.safetensors", "*.json", "*.jinja"]
                 ) { progress in
                     prefetchTracker.printProgress(progress)
                 }
             }
         }
+        // Streaming is activated for `modelDirectory`, and only a load of that exact
+        // directory streams. Load from it, or a different lookup could pick another copy.
+        if self.streamExperts, let dir = modelDirectory {
+            modelConfig = ModelConfiguration(directory: dir)
+        }
         var mainModelProfile: ModelProfile? = nil
         if self.streamExperts, let dir = modelDirectory {
             mainModelProfile = ModelProfiler.profile(modelDirectory: dir, modelId: modelId)
@@ -937,6 +974,7 @@ struct MLXServer: AsyncParsableCommand {
             return self.model
         }()
         let tracker = ProgressTracker(modelId: resolvedModelId)
+        defer { tracker.finish() }
 
         let isAudio = self.audio
         phase = .mainModelLoad
@@ -1003,6 +1041,7 @@ struct MLXServer: AsyncParsableCommand {
             print("[SwiftLM] Note: the prompt cache is not used for VLM/Omni loads; each text request re-prefills its full prompt.")
         }
 
+        tracker.finish()
         print("[SwiftLM] Loaded model configuration. Inferred tool call format: \(String(describing: await container.configuration.toolCallFormat))")
 
         // ── Check if target model supports DFlash ──

docs/profiling/m6/gemma4_26b_a4b_8bit.jsonl

@@ -8,3 +8,10 @@
 {"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 8192, "run": 0, "warmup": false, "peak_gpu_gb": 7.58, "swap_delta_gb": 0.0, "min_free_pct": 66, "status": "OK", "prompt_tokens": 9543, "prefill_tps": 147.6, "ttft_s": 64.86, "decode_tps": 7.08, "gen_tokens": 128, "needle_ok": true, "degenerate": false}
 {"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 8192, "run": 1, "warmup": false, "peak_gpu_gb": 7.19, "swap_delta_gb": 0.0, "min_free_pct": 60, "status": "OK", "prompt_tokens": 9518, "prefill_tps": 138.5, "ttft_s": 68.94, "decode_tps": 6.92, "gen_tokens": 128, "needle_ok": true, "degenerate": false}
 {"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 8192, "run": 2, "warmup": false, "peak_gpu_gb": 7.26, "swap_delta_gb": 0.0, "min_free_pct": 53, "status": "OK", "prompt_tokens": 9577, "prefill_tps": 136.9, "ttft_s": 70.22, "decode_tps": 6.64, "gen_tokens": 128, "needle_ok": true, "degenerate": false}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 512, "run": -1, "warmup": true, "peak_gpu_gb": 5.87, "swap_delta_gb": 0.0, "min_free_pct": 71, "status": "OK", "prompt_tokens": 520, "prefill_tps": 70.6, "ttft_s": 7.39, "decode_tps": 9.19, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "MARLIN-60", "answer_head": "The secret code word is MARLIN-60.\n\n***\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Reach lighthouse for twenty-two years, a duration long enough for the rhythmic pulse of the Fresnel lens to become his own heartb"}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 512, "run": 0, "warmup": false, "peak_gpu_gb": 6.5, "swap_delta_gb": 0.0, "min_free_pct": 71, "status": "OK", "prompt_tokens": 533, "prefill_tps": 90.6, "ttft_s": 5.93, "decode_tps": 9.13, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "MARLIN-23", "answer_head": "The secret code word is MARLIN-23.\n\n***\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Reach lighthouse for twenty-two years, a span of time measured not by calendars, but by the rhythmic pulse of the great Fresnel l"}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 512, "run": 1, "warmup": false, "peak_gpu_gb": 6.31, "swap_delta_gb": 0.0, "min_free_pct": 71, "status": "OK", "prompt_tokens": 542, "prefill_tps": 91.8, "ttft_s": 5.95, "decode_tps": 9.09, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "PELICAN-81", "answer_head": "The secret code word is PELICAN-81.\n\n***\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Reach Lighthouse for twenty-two years, a span of time that had turned his skin into something resembling cured leather and his e"}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 512, "run": 2, "warmup": false, "peak_gpu_gb": 5.6, "swap_delta_gb": 0.0, "min_free_pct": 71, "status": "OK", "prompt_tokens": 527, "prefill_tps": 88.7, "ttft_s": 5.99, "decode_tps": 9.04, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "HERON-20", "answer_head": "The secret code word is HERON-20.\n\n***\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Reach Lighthouse for twenty-two years, a span of time that had turned his skin to leather and his eyes to the color of a storm-tos"}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 8192, "run": 0, "warmup": false, "peak_gpu_gb": 6.46, "swap_delta_gb": 0.0, "min_free_pct": 66, "status": "OK", "prompt_tokens": 9546, "prefill_tps": 148.4, "ttft_s": 64.52, "decode_tps": 8.22, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "PELICAN-68", "answer_head": "The secret code word is PELICAN-68.\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Reach Lighthouse for twenty-two years, a duration that felt less like a career and more like a slow merging of man and stone. His wor"}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 8192, "run": 1, "warmup": false, "peak_gpu_gb": 5.92, "swap_delta_gb": 0.0, "min_free_pct": 66, "status": "OK", "prompt_tokens": 9511, "prefill_tps": 145.4, "ttft_s": 65.61, "decode_tps": 8.18, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "HERON-85", "answer_head": "The secret code word is HERON-85.\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Reach lighthouse for twenty-two years, a duration that felt less like a career and more like a slow merging of man and stone. His world"}
+{"model": "mlx-community/gemma-4-26b-a4b-it-8bit", "config": "SSD", "context": 8192, "run": 2, "warmup": false, "peak_gpu_gb": 7.3, "swap_delta_gb": 0.0, "min_free_pct": 59, "status": "OK", "prompt_tokens": 9549, "prefill_tps": 145.7, "ttft_s": 65.73, "decode_tps": 8.16, "gen_tokens": 128, "needle_ok": true, "degenerate": false, "code": "MARLIN-30", "answer_head": "The secret code word is MARLIN-30.\n\nElias Thorne had lived within the salt-crusted walls of the Blackwood Rock Lighthouse for twenty-two years, a duration long enough for the rhythmic pulse of the Fresnel lens to become his own heartbeat. T"}

docs/profiling/m6/gemma4_26b_a4b_8bit.md

@@ -12,3 +12,13 @@ Apple M6 · 32 GB · runs=3 (long=1) · warmup=1 · gen=128 · temperature 0 ·
 | SSD | 2048 (2291) | 193.7 | 11.93 | 7.7 | 7.32 | 0.0 | 69 | ok |
 | SSD | 8192 (9543) | 138.5 | 68.94 | 6.92 | 7.58 | 0.0 | 53 | ok |
 | SSD | 32768 | — | — | — | 5.8 | 2.09 | 32 | **MEM_ABORT** swap grew 2.1 GB |
+
+#### `--stream-experts` re-measured with SharpAI/mlx-swift-lm#69 and #71 (b769/b773 crash in this mode)
+
+Apple M6 · 32 GB · runs=3 (long=1) · warmup=1 · gen=128 · temperature 0 · medians
+
+| Config | Context (prompt tok) | Prefill tok/s | TTFT s | Decode tok/s | Peak GPU GB | Swap Δ GB | Min free % | Checks |
+|---|---|---|---|---|---|---|---|---|
+| SSD | 512 (533) | 90.6 | 5.95 | 9.09 | 6.5 | 0.0 | 71 | ok |
+| SSD | 8192 (9546) | 145.7 | 65.61 | 8.18 | 7.3 | 0.0 | 59 | ok |
+| SSD | 32768 | — | — | — | 5.79 | 2.04 | 31 | **MEM_ABORT** swap grew 2.0 GB |

docs/profiling/m6/media/m6_qwen36_35b_a3b_ssd_stream.cast

@@ -0,0 +1,145 @@
+{"version":3,"term":{"cols":112,"rows":40},"timestamp":1790449820,"idle_time_limit":2.0,"command":"/Users/simba/.cache/swiftlm-bench/demo/record_ssd.sh","title":"SwiftLM · Qwen3.6-35B-A3B --stream-experts · Mac mini M6 32GB","env":{"SHELL":"/bin/zsh"}}
+[0.329, "o", "\u001b[36mSwiftLM — github.com/SharpAI/SwiftLM\u001b[39m\r\n\u001b[3244mOpenAI-compatible LLM server for Apple Silicon, in Swift on MLX · release b782\u001b[39m\r\n\r\n"]
+[0.000, "o", "\u001b[3244m$\u001b[39m sysctl -n machdep.cpu.brand_string hw.memsize | paste -sd' ' -\r\n"]
+[0.641, "o", "Apple M6 · 32 GB · macOS 27.0\r\n\r\n"]
+[0.000, "o", "\u001b[3244m$\u001b[39m ./SwiftLM --model unsloth/Qwen3.6-35B-A3B-UD-MLX-4bit --stream-experts --port 5431 &\r\n"]
+[2.203, "o", "💾 SSD Expert Streaming enabled (lazy load + layer-sync)\r\n✅ Ready. Listening on http://127.0.0.1:5431\r\n"]
+[0.000, "o", "\r\n"]
+[0.000, "o", "\u001b[3244m$\u001b[39m python3 scripts/demo/stream_client.py warmup   # first request after load (warm-up)\r\n"]
+[1.545, "o", "\r\u001b[K\u001b[32m1"]
+[0.070, "o", ","]
+[0.069, "o", " "]
+[0.073, "o", "2"]
+[0.075, "o", ","]
+[0.073, "o", " "]
+[0.072, "o", "3"]
+[0.070, "o", ","]
+[0.071, "o", " "]
+[0.072, "o", "4"]
+[0.071, "o", ","]
+[0.072, "o", " "]
+[0.072, "o", "5"]
+[0.071, "o", ","]
+[0.072, "o", " "]
+[0.072, "o", "6"]
+[0.069, "o", ","]
+[0.072, "o", " "]
+[0.073, "o", "7"]
+[0.075, "o", ","]
+[0.078, "o", " "]
+[0.071, "o", "8"]
+[0.072, "o", "\u001b[0m\r\n\r\n\u001b[36m\u001b[1m  22 tokens · decode 13.3 tok/s · TTFT 0.9s ·\u001b[0m\r\n\u001b[36m\u001b[1m  peak SwiftLM 5.1 GB · swap +0.0 GB\u001b[0m\r\n"]
+[0.004, "o", "\u001b[3244m$\u001b[39m python3 scripts/demo/stream_client.py short    # measured request\r\n"]
+[1.448, "o", "\r\u001b[K\u001b[32m//"]
+[0.068, "o", " Returns"]
+[0.070, "o", " the"]
+[0.070, "o", " n"]
+[0.070, "o", "-th"]
+[0.074, "o", " Fibonacci"]
+[0.079, "o", " number"]
+[0.074, "o", " using"]
+[0.068, "o", " an"]
+[0.069, "o", " iterative"]
+[0.071, "o", " approach"]
+[0.072, "o", "."]
+[0.068, "o", "\r\n"]
+[0.071, "o", "func"]
+[0.070, "o", " fib"]
+[0.072, "o", "(_"]
+[0.070, "o", " n"]
+[0.072, "o", ":"]
+[0.072, "o", " Int"]
+[0.070, "o", ")"]
+[0.076, "o", " ->"]
+[0.081, "o", " Int"]
+[0.073, "o", " {"]
+[0.072, "o", "\r\n"]
+[0.070, "o", "   "]
+[0.070, "o", " guard"]
+[0.071, "o", " n"]
+[0.070, "o", " >"]
+[0.071, "o", " "]
+[0.071, "o", "0"]
+[0.069, "o", " else"]
+[0.071, "o", " {"]
+[0.072, "o", " return"]
+[0.072, "o", " "]
+[0.072, "o", "0"]
+[0.080, "o", " }"]
+[0.083, "o", "\r\n"]
+[0.071, "o", "   "]
+[0.071, "o", " guard"]
+[0.071, "o", " n"]
+[0.069, "o", " >"]
+[0.073, "o", " "]
+[0.072, "o", "1"]
+[0.071, "o", " else"]
+[0.070, "o", " {"]
+[0.072, "o", " return"]
+[0.072, "o", " "]
+[0.073, "o", "1"]
+[0.070, "o", " }"]
+[0.075, "o", "\r\n"]
+[0.089, "o", "    \r\n"]
+[0.082, "o", "   "]
+[0.070, "o", " var"]
+[0.070, "o", " prev"]
+[0.068, "o", "2"]
+[0.070, "o", " ="]
+[0.070, "o", " "]
+[0.071, "o", "0"]
+[0.070, "o", "\r\n"]
+[0.070, "o", "   "]
+[0.071, "o", " var"]
+[0.070, "o", " prev"]
+[0.070, "o", "1"]
+[0.073, "o", " ="]
+[0.086, "o", " "]
+[0.093, "o", "1"]
+[0.071, "o", "\r\n    \r\n"]
+[0.070, "o", "   "]
+[0.070, "o", " for"]
+[0.069, "o", " _"]
+[0.069, "o", " in"]
+[0.072, "o", " "]
+[0.071, "o", "2"]
+[0.070, "o", "..."]
+[0.070, "o", "n"]
+[0.071, "o", " {"]
+[0.071, "o", "\r\n"]
+[0.069, "o", "       "]
+[0.074, "o", " let"]
+[0.091, "o", " current"]
+[0.076, "o", " ="]
+[0.076, "o", " prev"]
+[0.074, "o", "1"]
+[0.072, "o", " +"]
+[0.072, "o", " prev"]
+[0.072, "o", "2"]
+[0.070, "o", "\r\n"]
+[0.072, "o", "       "]
+[0.069, "o", " prev"]
+[0.070, "o", "2"]
+[0.071, "o", " ="]
+[0.072, "o", " prev"]
+[0.074, "o", "1"]
+[0.087, "o", "\r\n"]
+[0.096, "o", "       "]
+[0.072, "o", " prev"]
+[0.071, "o", "1"]
+[0.071, "o", " ="]
+[0.072, "o", " current"]
+[0.071, "o", "\r\n"]
+[0.073, "o", "   "]
+[0.071, "o", " }"]
+[0.072, "o", "\r\n"]
+[0.072, "o", "    \r\n"]
+[0.070, "o", "   "]
+[0.071, "o", " return"]
+[0.072, "o", " prev"]
+[0.087, "o", "1"]
+[0.098, "o", "\r\n"]
+[0.070, "o", "}"]
+[0.072, "o", "\u001b[0m\r\n\r\n\u001b[36m\u001b[1m  110 tokens · decode 13.6 tok/s · TTFT 0.8s ·\u001b[0m\r\n\u001b[36m\u001b[1m  peak SwiftLM 5.1 GB · swap +0.0 GB\u001b[0m\r\n"]
+[0.004, "o", "\r\n"]
+[0.032, "x", "0"]