Skip to content

Commit 4e071f9

Browse files
solderzzcclaude
andcommitted
docs(m6): Qwen3.6-35B-A3B --stream-experts recording on release b782
Real-time recording on the official b782 binary: 13.6 tok/s decode, 5.1 GB peak, no swap. Shown in the README's Qwen3.6 GPU vs SSD section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent a0875c8 commit 4e071f9

3 files changed

Lines changed: 147 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,8 @@ Every needle check passed. `--mtp` with the bf16 assistant (`gemma-4-26B-A4B-it-
9090

9191
### Qwen3.6-35B-A3B 4-bit — GPU vs SSD streaming
9292

93+
![Qwen3.6-35B-A3B with --stream-experts on release b782: 13.6 tok/s decode, 5.1 GB peak, no swap, on a base Mac mini M6 32 GB](docs/profiling/m6/media/m6_qwen36_35b_a3b_ssd_stream.gif)
94+
9395
| Prompt tokens | GPU prefill / decode (tok/s) | GPU peak | `--stream-experts` prefill / decode (tok/s) | SSD peak |
9496
|---|---|---|---|---|
9597
| ~550 | 714 / 47.0 | 19.8 GB | 256 / 13.2 | 5.6 GB |
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
{"version":3,"term":{"cols":112,"rows":40},"timestamp":1790449820,"idle_time_limit":2.0,"command":"/Users/simba/.cache/swiftlm-bench/demo/record_ssd.sh","title":"SwiftLM · Qwen3.6-35B-A3B --stream-experts · Mac mini M6 32GB","env":{"SHELL":"/bin/zsh"}}
2+
[0.329, "o", "\u001b[36mSwiftLM — github.com/SharpAI/SwiftLM\u001b[39m\r\n\u001b[3244mOpenAI-compatible LLM server for Apple Silicon, in Swift on MLX · release b782\u001b[39m\r\n\r\n"]
3+
[0.000, "o", "\u001b[3244m$\u001b[39m sysctl -n machdep.cpu.brand_string hw.memsize | paste -sd' ' -\r\n"]
4+
[0.641, "o", "Apple M6 · 32 GB · macOS 27.0\r\n\r\n"]
5+
[0.000, "o", "\u001b[3244m$\u001b[39m ./SwiftLM --model unsloth/Qwen3.6-35B-A3B-UD-MLX-4bit --stream-experts --port 5431 &\r\n"]
6+
[2.203, "o", "💾 SSD Expert Streaming enabled (lazy load + layer-sync)\r\n✅ Ready. Listening on http://127.0.0.1:5431\r\n"]
7+
[0.000, "o", "\r\n"]
8+
[0.000, "o", "\u001b[3244m$\u001b[39m python3 scripts/demo/stream_client.py warmup # first request after load (warm-up)\r\n"]
9+
[1.545, "o", "\r\u001b[K\u001b[32m1"]
10+
[0.070, "o", ","]
11+
[0.069, "o", " "]
12+
[0.073, "o", "2"]
13+
[0.075, "o", ","]
14+
[0.073, "o", " "]
15+
[0.072, "o", "3"]
16+
[0.070, "o", ","]
17+
[0.071, "o", " "]
18+
[0.072, "o", "4"]
19+
[0.071, "o", ","]
20+
[0.072, "o", " "]
21+
[0.072, "o", "5"]
22+
[0.071, "o", ","]
23+
[0.072, "o", " "]
24+
[0.072, "o", "6"]
25+
[0.069, "o", ","]
26+
[0.072, "o", " "]
27+
[0.073, "o", "7"]
28+
[0.075, "o", ","]
29+
[0.078, "o", " "]
30+
[0.071, "o", "8"]
31+
[0.072, "o", "\u001b[0m\r\n\r\n\u001b[36m\u001b[1m 22 tokens · decode 13.3 tok/s · TTFT 0.9s ·\u001b[0m\r\n\u001b[36m\u001b[1m peak SwiftLM 5.1 GB · swap +0.0 GB\u001b[0m\r\n"]
32+
[0.004, "o", "\u001b[3244m$\u001b[39m python3 scripts/demo/stream_client.py short # measured request\r\n"]
33+
[1.448, "o", "\r\u001b[K\u001b[32m//"]
34+
[0.068, "o", " Returns"]
35+
[0.070, "o", " the"]
36+
[0.070, "o", " n"]
37+
[0.070, "o", "-th"]
38+
[0.074, "o", " Fibonacci"]
39+
[0.079, "o", " number"]
40+
[0.074, "o", " using"]
41+
[0.068, "o", " an"]
42+
[0.069, "o", " iterative"]
43+
[0.071, "o", " approach"]
44+
[0.072, "o", "."]
45+
[0.068, "o", "\r\n"]
46+
[0.071, "o", "func"]
47+
[0.070, "o", " fib"]
48+
[0.072, "o", "(_"]
49+
[0.070, "o", " n"]
50+
[0.072, "o", ":"]
51+
[0.072, "o", " Int"]
52+
[0.070, "o", ")"]
53+
[0.076, "o", " ->"]
54+
[0.081, "o", " Int"]
55+
[0.073, "o", " {"]
56+
[0.072, "o", "\r\n"]
57+
[0.070, "o", " "]
58+
[0.070, "o", " guard"]
59+
[0.071, "o", " n"]
60+
[0.070, "o", " >"]
61+
[0.071, "o", " "]
62+
[0.071, "o", "0"]
63+
[0.069, "o", " else"]
64+
[0.071, "o", " {"]
65+
[0.072, "o", " return"]
66+
[0.072, "o", " "]
67+
[0.072, "o", "0"]
68+
[0.080, "o", " }"]
69+
[0.083, "o", "\r\n"]
70+
[0.071, "o", " "]
71+
[0.071, "o", " guard"]
72+
[0.071, "o", " n"]
73+
[0.069, "o", " >"]
74+
[0.073, "o", " "]
75+
[0.072, "o", "1"]
76+
[0.071, "o", " else"]
77+
[0.070, "o", " {"]
78+
[0.072, "o", " return"]
79+
[0.072, "o", " "]
80+
[0.073, "o", "1"]
81+
[0.070, "o", " }"]
82+
[0.075, "o", "\r\n"]
83+
[0.089, "o", " \r\n"]
84+
[0.082, "o", " "]
85+
[0.070, "o", " var"]
86+
[0.070, "o", " prev"]
87+
[0.068, "o", "2"]
88+
[0.070, "o", " ="]
89+
[0.070, "o", " "]
90+
[0.071, "o", "0"]
91+
[0.070, "o", "\r\n"]
92+
[0.070, "o", " "]
93+
[0.071, "o", " var"]
94+
[0.070, "o", " prev"]
95+
[0.070, "o", "1"]
96+
[0.073, "o", " ="]
97+
[0.086, "o", " "]
98+
[0.093, "o", "1"]
99+
[0.071, "o", "\r\n \r\n"]
100+
[0.070, "o", " "]
101+
[0.070, "o", " for"]
102+
[0.069, "o", " _"]
103+
[0.069, "o", " in"]
104+
[0.072, "o", " "]
105+
[0.071, "o", "2"]
106+
[0.070, "o", "..."]
107+
[0.070, "o", "n"]
108+
[0.071, "o", " {"]
109+
[0.071, "o", "\r\n"]
110+
[0.069, "o", " "]
111+
[0.074, "o", " let"]
112+
[0.091, "o", " current"]
113+
[0.076, "o", " ="]
114+
[0.076, "o", " prev"]
115+
[0.074, "o", "1"]
116+
[0.072, "o", " +"]
117+
[0.072, "o", " prev"]
118+
[0.072, "o", "2"]
119+
[0.070, "o", "\r\n"]
120+
[0.072, "o", " "]
121+
[0.069, "o", " prev"]
122+
[0.070, "o", "2"]
123+
[0.071, "o", " ="]
124+
[0.072, "o", " prev"]
125+
[0.074, "o", "1"]
126+
[0.087, "o", "\r\n"]
127+
[0.096, "o", " "]
128+
[0.072, "o", " prev"]
129+
[0.071, "o", "1"]
130+
[0.071, "o", " ="]
131+
[0.072, "o", " current"]
132+
[0.071, "o", "\r\n"]
133+
[0.073, "o", " "]
134+
[0.071, "o", " }"]
135+
[0.072, "o", "\r\n"]
136+
[0.072, "o", " \r\n"]
137+
[0.070, "o", " "]
138+
[0.071, "o", " return"]
139+
[0.072, "o", " prev"]
140+
[0.087, "o", "1"]
141+
[0.098, "o", "\r\n"]
142+
[0.070, "o", "}"]
143+
[0.072, "o", "\u001b[0m\r\n\r\n\u001b[36m\u001b[1m 110 tokens · decode 13.6 tok/s · TTFT 0.8s ·\u001b[0m\r\n\u001b[36m\u001b[1m peak SwiftLM 5.1 GB · swap +0.0 GB\u001b[0m\r\n"]
144+
[0.004, "o", "\r\n"]
145+
[0.032, "x", "0"]
127 KB
Loading

0 commit comments

Comments
 (0)