diff --git a/docs/assets/blog/squish-building.png b/docs/assets/blog/squish-building.png
new file mode 100644
index 00000000..af5acacd
Binary files /dev/null and b/docs/assets/blog/squish-building.png differ
diff --git a/docs/assets/blog/squish-fistpump.png b/docs/assets/blog/squish-fistpump.png
new file mode 100644
index 00000000..e6c3bac6
Binary files /dev/null and b/docs/assets/blog/squish-fistpump.png differ
diff --git a/docs/assets/blog/squish-flying.png b/docs/assets/blog/squish-flying.png
new file mode 100644
index 00000000..e31f8659
Binary files /dev/null and b/docs/assets/blog/squish-flying.png differ
diff --git a/docs/assets/blog/squish-hero.png b/docs/assets/blog/squish-hero.png
new file mode 100644
index 00000000..def64d7e
Binary files /dev/null and b/docs/assets/blog/squish-hero.png differ
diff --git a/docs/blog/posts/local-llm-fast-enough.md b/docs/blog/posts/local-llm-fast-enough.md
index eb0f3a1f..dc0f6db0 100644
--- a/docs/blog/posts/local-llm-fast-enough.md
+++ b/docs/blog/posts/local-llm-fast-enough.md
@@ -1,9 +1,11 @@
---
-date: 2026-06-26
+date:
+ created: 2026-06-26
+ updated: 2026-07-06
slug: local-llm-fast-enough
title: "I Couldn't Find a Local LLM Tool Fast Enough, So I Built My Own"
subtitle: "Fast local LLM inference for Apple Silicon (M-series)."
-description: "A local LLM inference server for Apple Silicon, up to 9.8× faster than Ollama on long prompts, with the honest benchmarks."
+description: "A local LLM inference server for Apple Silicon, 1.15 to 14.7× faster than Ollama depending on how much your prompts repeat, with the honest benchmarks."
authors:
- wesley
og_author: Wesley Scholl
@@ -21,28 +23,403 @@ image: assets/blog/chart-speed.png
# I Couldn't Find a Local LLM Tool Fast Enough, So I Built My Own
-*This requires Apple Silicon (M-series). If you're on Intel, Linux, or Windows, the numbers in this article will not apply to your hardware.*
-
-**TL;DR:** Squish is a local LLM inference server for Apple Silicon, up to 9.8× faster than Ollama on long prompts. [Skip the story and install it →](#how-you-actually-use-it)
+
+
+
+
+
+*This requires Apple Silicon (M-series) and macOS 13 (Ventura) or later. If you're on Intel, Linux, or Windows, the numbers in this article will not apply to your hardware.*
+
+**TL;DR:** Squish is a local LLM inference server for Apple Silicon, 1.15 to 14.7× faster than Ollama depending on how much your prompts repeat. [Skip the story and install it →](#how-you-actually-use-it)
Since February, many of my commits have been written by a local AI in under two seconds. No API keys, no rate limits, no internet connection, and no data leaving my machine. Getting it working took considerable effort, but once it did, it's been very reliable across dozens of repositories. The local AI software I'm describing didn't exist, not without heavy modifications to source code you don't control, or building your own from scratch. So I built it, and it solved my problem. It may not solve yours, but you can clone it, fork it, take it apart, and have fun with it.
-It started with Gemini. I wired the API to a script I wrote that automates git commits and pushes. It was fast and intelligent, until you hit the hard free tier rate limits. In my case, I hit them every day. Sometimes on the second commit, sometimes on the first. Gemini returns a response message saying you've been rate-limited. Use it once, use it twice, and you're cut off. This was for simple stuff, maybe 500 to 1,000 tokens for a commit, a little more for a large diff. I put up with it for a while, but the annoying limits drove me to drop it. I searched around but couldn't find anything that could solve my problem.
+It started with Gemini. I wired the API to a script I wrote that automates git commits and pushes. It was fast and intelligent, until I hit the hard free tier rate limits. In my case, I hit them every day. Sometimes on the second commit, sometimes on the first. Gemini returned a response message saying I was rate limited. I used it once or used it twice and was cut off. This was for simple stuff, maybe 500 to 1,000 tokens for a commit, a little more for a large diff. I put up with it for a while, but the annoying limits drove me to drop it. I searched around but couldn't find anything that could solve my problem.
-So I went local, off the cloud entirely, with no rate-limiting. I pointed the script at Ollama running Mistral and configured and tested it for weeks: built a custom Modelfile, iterated prompts, fine-tuned output until the commit messages described the diff accurately. The descriptions came out great. They also took too long. End to end, a response landed anywhere from seven to ten seconds on a normal commit, and north of a minute for a large diff. So I pulled every lever and turned every knob, but the adjustments failed to reduce the response time, and my problem was still unsolved. The slow and unpredictable responses were the hard wall. When the software is written by someone else, their speed limit is your speed limit. I wanted a coherent commit message in under five seconds, ideally under three. Ollama and the models it serves could not deliver. I thought about it for over a month and came to the conclusion that I had to build something myself. Something lean and elegant that doesn't just "work", it beats the baseline outright. That something is called Squish, a local AI inference server.
+So I went local, off the cloud entirely, with no rate-limiting. I pointed the script at Ollama running Mistral and configured and tested it for weeks: built a custom Modelfile, iterated prompts, fine-tuned output until the commit messages described the diff accurately. The descriptions came out great. They also took too long. End to end, a response landed anywhere from seven to ten seconds on a normal commit, and north of a minute for a large diff. So I pulled every lever and turned every knob, but the adjustments failed to reduce the response time, and my problem was still unsolved. The slow and unpredictable responses were the hard wall. When the software is written by someone else, their speed limit is your speed limit. I wanted a coherent commit message in under five seconds, ideally under three. Ollama and the models it serves could not deliver. I thought about it for over a month and came to the conclusion that I had to build something myself. Something lean and elegant that doesn't just work, it beats the baseline outright. That something is called Squish, a local AI inference server.
## What Squish Actually Is
-**To be clear:** I did not build a model. I built the server that runs models, a framework that quantizes and compresses them, and a local format that reduces how much memory they need to run. Squish is an MLX-based local inference server, and three architectural components set it apart from existing tools:
+**To be clear:** I did not build a model. I built the server that runs models, a framework that quantizes and compresses them, and a local format that reduces how much memory they need to run. Squish is an MLX-based local inference server, and five architectural components set it apart from existing tools:
+
+
+
+
+
+
+
**1. A persistent daemon that keeps the model awake.** Ollama loads its model lazily, on the first request, and that first request pays for it: twenty to thirty seconds of cold start while the weights load, before a single token comes back. Worse, the model gets unloaded once it sits idle, so an intermittent workflow pays that toll over and over. Squish loads the model once, when the daemon starts, and keeps it resident for as long as it's running. The cost is paid a single time; every request after is warm. Load once, never load again.
**2. A two-tier KV cache that remembers.** Before a model can answer a prompt, it runs prefill. It reads the entire prompt and builds the internal attention state required in memory before producing a single token. That memory state is called the KV cache. Normally the KV cache is discarded once a response is completed, and on the next request it gets rebuilt from scratch. Squish retains the KV cache, in two layers.
-- **The prompt cache** handles exact repeats, an identical prompt sent again. If that prompt is resent, there's nothing to rebuild. Time-to-first-token (TTFT) drops to roughly 4 to 11 ms, versus about 192 ms to prefill from scratch.
+- **The prompt cache** handles exact repeats, an identical prompt sent again. If that prompt is resent, there's nothing to rebuild. Time-to-first-token (TTFT) drops to roughly 4 to 11 ms, versus about 800 ms to prefill from scratch.
- **The block cache** handles prompts that partially overlap previous prompts. The cache stores KV state in fixed-size blocks on disk. A block is computed only once. Any future prompt that contains overlapping blocks reuses the stored copy. This ensures the model only computes tokens it hasn't seen before. Examples include agent loops that resend a long system prompt each turn, and multi-turn conversations.
-**3. INT3 quantization that genuinely works, on some models.** A model's knowledge lives in its parameters, also called weights, the values it learned during training. Quantization stores each parameter at lower precision, like rounding a long decimal to a couple of places. It saves memory and adds speed, but can cost accuracy. Three-bit quantization (INT3) is aggressive enough that it usually breaks a model outright. However, for some model families, INT3 is stable, more on that below.
+Exact or partial prompts are processed once, never repeated.
+
+**3. INT3 quantization that genuinely works, on some models.** A model's knowledge lives in its parameters, also called weights, the values it learned during training. Quantization stores each parameter at lower precision, like rounding a long decimal to a couple of places. This saves memory, and on Apple Silicon it also makes the model faster. The reason is simple: the slow part of producing each token is not the math, it is moving the model's weights out of memory. Smaller weights mean less to move, and less to move means more tokens per second. The tradeoff is accuracy. Three-bit quantization (INT3) is aggressive enough that it usually breaks a model outright. However, for some model families, INT3 is stable, more on that below.
+
+
+
+
+
+
+
+
+**4. The optimization engine: where the speed and memory savings come from.** To generate each new token, the GPU works through every weight in the model. Most tools expand the rounded-off weights back to full precision in memory before the GPU can use them. Squish never expands the full model to full-size in memory, saving a significant amount of memory. To achieve this, the quantized model weights are streamed from memory in small blocks by the GPU. During this process, each quantized block is expanded to full-size, then processed, and the next block overwrites the previous block sequentially. Only a few blocks are expanded to full-size inside the GPU at a time. Squish runs the same computation as full precision, without expanding the model weights to full-size in memory. With this memory savings, fewer bytes move through memory, improving speed at the same time.
+
+
+
+
+
+
+
+
+**5. A memory governor that reacts to real pressure.** On a 16 GB Mac, everything shares the same memory, including the model, long conversation contexts, macOS, and every other running application. If all of these exceed 16 GB, the memory spills into disk space, which is slow and can cause thrashing, or macOS may kill the process outright. Squish watches memory-pressure signals from macOS. As memory pressure rises, the governor reduces the KV cache's size, first gently, then more aggressively if pressure keeps climbing. Under memory pressure, new requests are also given a smaller context window. This prevents allocating space the machine does not have. If memory pressure is critical, Squish rejects new requests with a clear response instead of crashing. Any requests already generating remain to finish processing. Once memory pressure reduces to a normal level, the original KV cache's size and prompt context window limits are restored.
## The Benchmarking Methodology
@@ -61,37 +438,135 @@ With that protocol settled, here is how the two inference servers compare (apple
| Metric | Ollama 0.30.7 | Squish |
|---|---:|---:|
| Cold start: load + first token (1.5B) | 20–30 s | ≈0.5 s (54×) |
-| Full response @ 4000-token prompt | 37.5 s | 3.8 s (9.8×) |
+| Full response @ 4000-token prompt (repeated exactly)* | 37.5 s | 3.8 s (9.8×) |
| Decode throughput @ 75 tokens | 20.3 tok/s | 24.0 tok/s (INT3) |
+| Decode throughput @ 2000 tokens | 19.7 tok/s | 22.6 tok/s (INT3) |
| Inter-token tail p95 @ 75 tokens | 52.4 ms | 42.7 ms (INT3) |
+| Inter-token tail p95 @ 2000 tokens | 52.9 ms | 45.4 ms (INT3) |
| Repeat-prompt TTFT (KV cache hit) | ~160 ms | 4–11 ms |
| Peak RAM during inference | 5.14 GB | 3.50 GB |
| Disk: 7B INT4 / INT3 | 4.36 GB / n/a | 4.00 / 3.56 GB |
-| Cold short-prompt TTFT | 167 ms | 192 ms |
-*Ollama 0.18.2 was tested identically. Its decode throughput and latency matched 0.30.7 to within noise, with one exception: cold short-prompt TTFT, where 0.18.2 measured 126 ms.*
+*See "How Prompt Reuse Scales" below, this is the ceiling, not the typical case.*
-Ollama wins one category, TTFT on a cold, completely unique short prompt, measuring 167 milliseconds compared to 192 ms for Squish. I have to give props where props are due: Ollama 0.30.7's TTFT is 25 ms faster. I tried to beat it, but quickly realized that 25 ms is something the end user doesn't notice the way they notice end-to-end response time. Squish wins on end-to-end response time. That's the metric that actually matters for my commit message use case.
+*Ollama 0.18.2 was tested identically. Its decode throughput and latency matched 0.30.7 to within noise.*
-So what is the trade-off for those 25 milliseconds? Let's say you have an agent resending the system prompt every turn. For a 4,000-token prompt, Ollama takes around 37.5 seconds to return a full response. Squish's full response returns in 3.8 seconds flat. It's 9.8× faster, and it's the difference between a tool that responds in a timely manner and one where users stare at the screen wondering why it hasn't responded yet.
+Squish leads on every metric in this table: decode throughput, tail latency, peak memory, and full end-to-end response time. An earlier version of this comparison also reported an Ollama win on short-prompt TTFT, 167 ms vs 192 ms, using the same fixed prompt sent five times. That number didn't hold up under scrutiny: it was measuring Ollama's own cache hit on the repeat sends, not a genuinely cold comparison. Under real cold, unique-prompt conditions, covered in the next section, Squish leads TTFT too, at every length tested, down to the shortest prompt measured.
-
+Let's say you have an agent resending the system prompt every turn. For a 4,000-token prompt, Ollama takes around 37.5 seconds to return a full response. Squish's full response returns in 3.8 seconds flat. It's 9.8× faster, and it's the difference between a tool that responds in a timely manner and one where users stare at the screen wondering why it hasn't responded yet.
-*Full response at a 4000-token prompt. Squish returns in 3.8 s versus Ollama's 37.5 s, a 9.8× speedup on the thermally controlled run. Source: BENCHMARKS.md.*
+## How Prompt Reuse Scales
-
+The 9.8× number above assumes the prompt repeats exactly. What if the prompt is completely unique with nothing to reuse at all? I benchmarked Qwen2.5-7B-Instruct using Squish INT4 vs Ollama Q4_K_M. Tested five context lengths and verified 0% reuse for every single request.
-*Memory and disk footprint. Squish uses less peak RAM (3.50 vs 5.14 GB) and less disk for a 7B model (3.56 vs 4.36 GB). Source: BENCHMARKS.md.*
+| Context | Ollama TTFT | Squish TTFT | Ollama decode | Squish decode | Ollama E2E | Squish E2E | Speedup |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| 75 | 812 ms | 800 ms | 16.4 tok/s | 17.5 tok/s | 14.12 s | 12.30 s | 1.15× |
+| 512 | 3.90 s | 3.33 s | 16.7 tok/s | 19.1 tok/s | 16.27 s | 13.74 s | 1.18× |
+| 1024 | 10.01 s | 8.59 s | 10.8 tok/s | 14.8 tok/s | 28.93 s | 21.98 s | 1.32× |
+| 2048 | 19.79 s | 17.31 s | 10.5 tok/s | 14.2 tok/s | 38.84 s | 32.24 s | 1.20× |
+| 4096 | 41.50 s | 36.14 s | 9.4 tok/s | 11.9 tok/s | 62.80 s | 52.93 s | 1.19× |
-## The Most Interesting Thing I Found
+*Squish's 4096-token TTFT ranged from 25.4 to 43.6 seconds across its five measured runs, wider than the other rows, though each run independently confirmed 0% cache hit. This row should be treated as the least precise of the five. The 75-token result is the closest race of the set, Squish's TTFT edge there is just 12 ms, about 1.5%, still a real win but a thin one.*
+
+Even with 0% prompt reuse, Squish wins TTFT, decode, and end-to-end response time at every context size tested, down to the shortest prompt measured. The floor is 1.15 to 1.32× faster than Ollama.
+
+The 9.8× ceiling resulted from the same 200-token, thermally-controlled benchmark as the floor above, so the numbers are directly comparable. The floor tests completely unique prompts; the ceiling resends an identical prompt every time: **1.15 to 1.32× on unique prompts, up to 9.8× on exact repeats.** The benchmark below measures the prompt reuse effect in isolation and reaches 14.7× with 95% overlapping prompts.
+
+I also wanted to see how speedup scales as prompt reuse increases from 5% to 95%. The second benchmark isolates and tests the reuse effect at four context lengths and five overlap percentage levels. Each result was identically validated byte-for-byte against a cold run to confirm reuse output is the exact same.
+
+| Overlap | 512 tok | 1024 tok | 2048 tok | 4096 tok |
+|---|---:|---:|---:|---:|
+| 5% | 1.15× | 1.07× | 1.06× | 1.07× |
+| 25% | 1.32× | 1.30× | 1.34× | 1.30× |
+| 50% | 1.55× | 1.84× | 1.97× | 1.91× |
+| 75% | 2.75× | 3.23× | 3.51× | 3.64× |
+| 95% | 5.86× | 6.95× | 11.0× | 14.7× |
-I wanted local models to run faster and use less memory. I started with INT4 quantization, which is well studied and a stable standard. But I wanted to know exactly how small the models would compress, so I tried quantizing using INT3. Most models broke. Qwen2.5-7B didn't. The INT3 quantized model remained stable, and tied INT4 on the arc_easy reasoning benchmark. The scores were 0.551 to 0.541, INT3 slightly ahead but within the margin of error. The INT3 model decodes roughly 18% faster and compresses the model from 4.00 GB to 3.56 GB on disk. The rest of the Qwen model family remained stable using INT3, remaining within roughly 1% of the original model's performance. I also quantized Gemma-3 using INT3, and it collapsed, a fifteen-point drop in accuracy.
+*The results above reflect three runs, lighter than the 5-run protocol above. These five overlap levels are quartiles pulled from a finer sweep measured at every 5% interval between 5% and 95%. The full 19-point sweep isn't committed to the repo, reproduce it with `python -m benchmarks.prefix_reuse_curve --contexts 512,1024,2048,4096`.*
-The Qwen model family could handle INT3, in some cases marginally beating INT4 within the margin of error. With that finding in hand, I wanted to find the actual floor, so I moved on to INT2. Qwen broke, and for good reason. The model I tested started spitting out gibberish in response to my prompt, "`IFYINGIFYIN`", completely incoherent, basically random. When a 16-bit model is crushed down to 2 bits, its intelligence is lost. Aggressive quantization has a hard floor, and eventually every model breaks. That breaking point is different for every model and every model family. That's why Squish has a quantization safety mechanism that blocks INT3 for the families it fails on and INT2 altogether. The result is a tool that compresses every model as far as it can safely go, and no further.
+This table's results are higher than the floor-to-ceiling range above. An 8-token generation is almost entirely prefill and the ratio isolates the prompt reuse effect at full strength. The 9.8× ceiling includes a full 200-token response including decode time. Reuse can't speed up decode, reducing the ratio. Both measure the same mechanism, one in isolation, and the other inside a real response. Speedup is modest at low overlap percentages and improves significantly at 50% and above. At 95% overlap, a 512-token prompt sees 5.9× while a 4096-token prompt sees 14.7×. At 75% and 95% overlap, the longer the context the faster the response, though the pattern is less clean right at 50%.
-
+Agent loops and multi-turn conversations benefit heavily from increasing context with overlapping prompts. However, reuse only helps when there's previously processed content to reuse. The other lever is making the weights themselves smaller, and that's where things get interesting.
-*arc_easy acc_norm by precision. INT3 holds (0.551, tied with INT4 at 0.541); INT2 collapses to 0.290, at the random baseline. Source: BENCHMARKS.md.*
+*Ollama version 0.31.1 was released five days after the benchmark above was measured. A spot-check using 0.31.1 on the cold/unique floor reproduced the exact same 1.32× result at 1024 tokens. The reuse-ceiling spot-check initially failed its own drift check due to unrelated machine load; after clearing it, a clean re-run landed at 8.8×, within noise of the published 9.8×.*
+
+## The Most Interesting Thing I Found
+
+I wanted local models to run faster and use less memory. I started with INT4 quantization, which is well studied and a stable standard. But I wanted to know exactly how small the models would compress, so I tried quantizing using INT3. Most models broke. Qwen2.5-7B didn't. The INT3 quantized model remained stable, and tied INT4 on the arc_easy reasoning benchmark. The scores were 0.551 to 0.541, INT3 slightly ahead but within the margin of error. The INT3 model decodes roughly 18% faster and compresses the model from 4.00 GB to 3.56 GB on disk. The other Qwen models I tested remained stable using INT3, within roughly 1% of the original model's performance. I also quantized Gemma-3 using INT3, and it collapsed, a fifteen-point drop in accuracy.
+
+
+
+
+
+
+
+
+The Qwen models I tested could handle INT3, in some cases marginally beating INT4 within the margin of error. With that finding in hand, I wanted to find the actual floor, so I moved on to INT2. Qwen broke, and for good reason. The model responded to my prompt with gibberish: `IFYINGIFYIN`, completely incoherent, basically random. When a 16-bit model is crushed down to 2 bits, its intelligence is lost. Aggressive quantization has a hard floor, and eventually every model breaks. That breaking point is different for every model and every model family. That's why Squish has a quantization safety mechanism that blocks INT3 for the families it fails on and INT2 altogether. The result is a tool that compresses every model as far as it can safely go, and no further.
## How You Actually Use It
@@ -111,15 +586,13 @@ pip install squish-ai # Python 3.11–3.14 required - OR
pipx install squish-ai --python python3.13 # Isolated in its own environment
```
-It runs on macOS 13 or later, and the Rust quantizer that does the heavy lifting installs itself.
-
With Squish installed, start a model with `squish run`. Specifying a model is optional. With no argument, Squish pulls a default sized to your machine's RAM, so you get the largest model that comfortably fits.
```bash
squish run # or specify a model, e.g. squish run qwen2.5:7b
```
-Behind the scenes on that first run, Squish pulls a pre-squished model from the Squish Hugging Face, the conversion and accuracy-gating already done, then loads and serves it. Once the model is loaded and ready, the web UI opens automatically.
+Behind the scenes on that first run, Squish pulls a pre-squished model from the squishai Hugging Face community, the conversion and accuracy-gating already done, then loads and serves it. Once the model is loaded and ready, the web UI opens automatically.
A caveat before you go looking for models: Squish doesn't run every model, only those in MLX format (fp16 or bf16 weights). More on that in the next section.
@@ -132,7 +605,7 @@ export OPENAI_API_KEY=squish
Ollama clients work too: point `OLLAMA_HOST` at the same port and they won't know the difference.
-From there, several ways to work with the running model:
+From there, there are several ways to work with the running model:
- **The web UI**, a chat client backed by a live instrument panel: the KV cache filling and reusing in real time, a quantization comparator, a latency waterfall splitting prefill from decode, and Apple Silicon power telemetry.
- **The VS Code extension**, the same chat plus agentic features and repo access, right in your editor.
@@ -165,6 +638,92 @@ When I first started building Squish, my goal was to get my commit messages writ
- A drop-in OpenAI endpoint for testing other applications against a real model, no API bill.
- Model benchmarking and inference metrics: TTFT, end-to-end latency, RAM/GPU utilization live in the web dashboard and VS Code extension.
-Your workflow may not look like mine; everyone is different. Squish solves my problems, and it may or may not solve yours. But if you have more than 16 GB RAM, Squish can run larger models than I've been able to while using significantly less RAM and hard disk space. If you want to modify it, the source is available and every benchmark number is reproducible from the repo.
+Squish solves my problems, and it may or may not solve yours. But if you have more than 16 GB RAM, Squish can run larger models than I've been able to while using significantly less RAM and hard disk space. If you want to modify it, the source is available and every benchmark number is reproducible from the repo.
+
+I built Squish because I was tired of waiting. It keeps the model loaded, the cache warm, and answers before I've moved my hands. What I haven't seen is what it can do beyond my one machine.
+
+## Join the Squish Community
+
+
+
+
+
+
+I've run and tested Squish on one machine, an M3 MacBook Pro with 16 GB of RAM. Every number in this article reflects that same laptop. I have no idea what Squish looks like on a Mac Studio with 128 GB, or what happens when you point it at a 70B model instead of a 7B one, and I'd like to find out.
+
+If you have access to a Mac with 32, 64, or 128+ GB of unified memory, running Squish against larger models and publishing what you find would answer questions I can't answer myself:
+
+- How does the memory governor behave when there's real headroom to work with?
+- Does the reuse curve hold the same shape for larger models?
+- Where is the actual ceiling beyond the 16 GB RAM constraint?
+
+A few other ways to help, beyond high RAM benchmarking:
+
+- **Try to break the numbers in this article.** Every benchmark here is reproducible from the repo. If you can find a configuration where the claims don't hold, that's a more valuable contribution than a benchmark that confirms what's already published.
+- **Test model families I haven't.** The INT3 accuracy gate currently covers a handful of Qwen models and Gemma-3. There are a lot of other model families out there I simply haven't had time to run through it.
+- **File real bugs from actual use.** Edge cases from actual daily workflows surface things a benchmark suite never will.
+- **Contribute to the model catalog.** If there's an MLX-format model on Hugging Face you think belongs in Squish's catalog, open an issue or a PR.
+- **Improve the docs.** If something in this article or the repo's docs confused you, it'll confuse someone else too.
+
+Open an issue or a PR on GitHub, whichever of these you want to take on. I built Squish alone, but I'd rather not be the only one who knows what it can do.
+
+
+
+
+
+
+Every issue filed, PR opened, and benchmark reproduced is one more data point Squish didn't have before. Thanks for reading this far, now go break something.
+
+
+
+
+
+
+---
-Thanks for taking the time to read this article. If you have any requests for models to add to the Squish Hugging Face, or any problems installing or running Squish, please open an issue on GitHub.
+Requests for models to add to the squishai Hugging Face community, or trouble installing or running Squish? Open an issue on GitHub.
diff --git a/docs/paper.md b/docs/paper.md
index 7e99da42..24448129 100644
--- a/docs/paper.md
+++ b/docs/paper.md
@@ -153,7 +153,7 @@ This section reports end-to-end, on-hardware throughput and latency — the live
Per-token decode (median inter-token latency) is 39–49 ms for Squish versus 50–57 ms for Ollama across all lengths; tail latency (p95) is uniformly tighter. These gains are *serving-layer* recoveries, not model differences: decoupling the decode loop from the event loop (one inference-thread handoff per request rather than per token), suspending cyclic garbage collection during generation with a one-time post-warmup heap freeze, and pinning the inference thread to performance cores via QoS together raised short-context throughput from 14.6 to 20.5 tok/s and cut the p95 tail from 166 ms to 48 ms.
-**Long-context end-to-end.** The largest gap is full-response latency on long prompts. For a 200-token completion on a 4 000-token prompt, Ollama takes 37.5 s versus Squish's 3.8 s — a **9.8×** reduction — because Squish's KV cache reuses the prefill across requests where Ollama re-runs it. On a genuinely novel 4 000-token prompt (caches cold for both), prefill is compute-bound and the two engines are at parity (~26 s); Squish's advantage is architectural cache reuse, not faster cold prefill.
+**Long-context end-to-end.** The largest gap is full-response latency on long prompts. For a 200-token completion on a 4 000-token prompt, Ollama takes 37.5 s versus Squish's 3.8 s — a **9.8×** reduction — because Squish's KV cache reuses the prefill across requests where Ollama re-runs it. On a genuinely novel 4 000-token prompt (caches cold for both, 0% reuse), prefill is compute-bound and Squish still leads, by a genuine 1.15–1.32× margin across tested context lengths rather than the cache-reuse ceiling above; Squish's advantage there comes from serving-layer throughput, not cache reuse.
**Decode-acceleration ablation.** Decode on this hardware is *weight-bandwidth bound*: each token streams the full ~4 GB INT4 weight set from unified memory, giving a ~25 tok/s ceiling of which Squish reaches ~20–22. Two levers move that ceiling and two do not:
@@ -162,7 +162,7 @@ Per-token decode (median inter-token latency) is 39–49 ms for Squish versus 50
- **Quantised KV cache** (mlx_lm-native, GPU-side): **no decode speedup** at 4- or 8-bit (20.0 tok/s unchanged), because the KV cache is only ~10–15 % of per-token memory traffic and the dequantisation-during-attention cost offsets the bandwidth saved; 4-bit additionally degrades output. It is a *memory* lever (longer context in fixed RAM), not a throughput lever.
- **Draft-model speculative decoding** (1.5 B draft for the 7 B target): net-negative at temperature 0 — acceptance is too low to offset the draft forwards, and the path was retained as opt-in only.
-The throughline is that on bandwidth-bound Apple-Silicon decode, the techniques that help are those that read fewer weight bytes per token (lower-bit quantisation) or emit more tokens per weight read (greedy-lossless speculation); techniques that compress only the KV cache or add a draft model do not. Squish's only measured loss against Ollama is time-to-first-token on a cold short prompt (192 ms vs 167 ms); it leads on decode throughput, tail latency, full-response latency, and RAM (3.5 GB vs 5.1 GB peak).
+The throughline is that on bandwidth-bound Apple-Silicon decode, the techniques that help are those that read fewer weight bytes per token (lower-bit quantisation) or emit more tokens per weight read (greedy-lossless speculation); techniques that compress only the KV cache or add a draft model do not. Squish has no measured loss against Ollama: under cold, genuinely unique-prompt conditions it leads time-to-first-token at every length tested, down to 75 tokens, in addition to decode throughput, tail latency, full-response latency, and RAM (3.5 GB vs 5.1 GB peak).
---
diff --git a/overrides/main.html b/overrides/main.html
index 57dd8e9c..67fb6baa 100644
--- a/overrides/main.html
+++ b/overrides/main.html
@@ -35,7 +35,15 @@
{% if og_type == "article" %}
-
+ {#- `date` is either a flat value or a {created, updated} mapping. -#}
+ {% if page.meta.date is mapping %}
+
+ {% if page.meta.date.updated %}
+
+ {% endif %}
+ {% else %}
+
+ {% endif %}
{% if page.meta.og_author %}
{% endif %}
+
+
+
+