Add GPU and long-context diagnostics

Author: KSemenenko

README.md

@@ -43,7 +43,7 @@ Details: [`docs/Architecture.md`](docs/Architecture.md).
 | Is a whole short request faster? | **Yes in the 8-token smoke.** The local median was 216 ms versus 690 ms; hosted medians were 526–557 ms versus 635–1,671 ms. |
 | Does it use less memory? | **Yes in these CPU diagnostics.** The local smoke was 555 MiB versus 1,256 MiB; hosted Synapse used 539–553 MiB versus 575–1,203 MiB. |
 | Does it start answering fast? | **For the 5-token smoke prompt, yes.** On the hosted long prompt, Synapse's first token was slower than LLamaSharp on all three OSes even with batched prefill. |
-| Is it as fast as **MLX** on the Apple GPU? | **No.** MLX writes 225 tokens/s on the GPU. Synapse has no GPU code yet. |
+| Is it as fast as **MLX** on the Apple GPU? | **No.** MLX writes 225 tokens/s on the GPU. The new Synapse Metal backend wrote 135 tokens/s in one local 128-token run (llama.cpp Metal: 157). |
 | How does **Microsoft Foundry Local** do? | **Fast, but quality-limited in this test.** With separate ONNX weights on about 6 cores, Qwen2.5 0.5B reached 231 tokens/s and the 7B models 22–25 tokens/s; none of the 7 model/device variants fully completed the 128-token instruction. |
 | Is it faster than the other .NET engine, dotLLM? | **In the same 8-token smoke, 14× faster.** |
 
@@ -131,7 +131,7 @@ in the table above and the benchmark details.
 | | Problem | Fix | Where |
 |---|---|---|---|
 | 🔴 | Hosted long-prompt first token and 128-token decode trail the CPU references | Profile prefill and decode on each runner | `experiments/Synapse.ReferenceBenchmarks` |
-| 🔴 | No GPU | Metal kernels | not started |
+| 🟡 | Metal runs Qwen2 but decodes slower than llama.cpp and MLX; CUDA is written but has never run | FP16 KV cache, GPU-packed weights; run CUDA on an NVIDIA machine | `native/synapse-gpu` |
 | 🟡 | Long-output quality is unreviewed | Review generated text and add a quality gate before release claims | `.github/workflows/performance.yml` |
 | 🟡 | Takes token IDs, not text | Tokenizer and chat templates | not started |
 | 🟡 | Only Qwen2 runs | More model families | `src/Synapse.Runtime/Features/TextGeneration` |

benchmarks/README.md

@@ -3,7 +3,7 @@
 The short version is in the [main README](../README.md#benchmark). This page
 lists every recorded run, how it was measured, and its raw data.
 
-Runs A–K use Qwen2.5 0.5B Instruct, the only model Synapse runs today. Run L
+Runs A–K and N–O use Qwen2.5 0.5B Instruct, the only model Synapse runs today. Run L
 measures Microsoft Foundry Local alone on six models from four families.
 
 ## Runs
@@ -23,6 +23,8 @@ measures Microsoft Foundry Local alone on six models from four families.
 | **K. 8-token, newest** | SIMD C# kernels | 2 | 3 warm-up + 5 measured | [JSON](results/2026-09-28-m2-pro-qwen2.5-0.5b-q8_0-cpu-kernels-managed-2thread-smoke.json) |
 | L. Foundry Local, 6 models, answer + dialogue | Not included (Foundry only) | runtime default (about 6 cores) | 1 warm-up + 3 measured | 14 JSON files + 2 default-context probes, `results/2026-09-28-m2-pro-foundry-local-*` |
 | M. Hosted CPU, MLX, and Foundry, 3 operating systems | Rust CPU kernels for Synapse | CPU 2 / external runtime default | CPU smoke 3+5; long/MLX/Foundry 1+3 | [GitHub run `36435838291`](https://github.com/managedcode/Synapse/actions/runs/36435838291), 22 raw artifacts |
+| N. Metal GPU long context, 40k and 131k synthetic tokens | Metal backend (ADR-012), FP32 KV | GPU | 1 measured per row | [JSON](results/2026-09-28-m2-pro-qwen2.5-0.5b-q8_0-metal-long-context-speed-diagnostic.json) |
+| O. Metal GPU pass-key retrieval, 4k to 120k | Metal backend, FP32 KV (native window) and FP16 KV (YaRN ×4) | GPU | 1 measured per row | [native](results/2026-09-28-m2-pro-qwen2.5-0.5b-q8_0-metal-passkey-native-f32-diagnostic.json), [YaRN](results/2026-09-28-m2-pro-qwen2.5-0.5b-q8_0-metal-passkey-yarn4-f16-diagnostic.json) |
 
 Local machine: MacBook Pro, Apple M2 Pro (8 performance + 4 efficiency cores,
 19-core GPU), 32 GB, macOS 27.0 arm64. Every CPU sample starts a new process.

benchmarks/results/2026-09-28-m2-pro-llamacpp-metal-passkey-120k-control-diagnostic.json

@@ -0,0 +1,22 @@
+{
+  "schemaVersion": 1,
+  "kind": "passkey-control-diagnostic",
+  "recordedOn": "2026-09-28",
+  "host": "owner MacBook Pro, Apple M2 Pro (19-core GPU), 32 GB, macOS 27",
+  "subject": "llama.cpp llama-completion 0.4.1 (b29c606e2, Homebrew), Metal",
+  "command": "llama-completion -m qwen2.5-0.5b-instruct-q8_0.gguf -f prompt.txt -c 131072 -n 12 --temp 0 -s 0 -ngl 99 -fa on --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768 -no-cnv --no-display-prompt",
+  "scenario": "passkey-qwen2.5, key 68317 at depth 0.1, 500 fillers before and 4496 after",
+  "promptTextSha256": "dbf038309ebf1d62f43f9e1ac4f74681dfcecd73235c6a780516950f3b99d344",
+  "promptTokenIdsSha256": "9d94e457138d58a37ceaa26221054e1ce8d5bf268acb18b66bbd15de4429360a",
+  "promptTokens": 119991,
+  "promptTokenIdsIdenticalToSynapse": true,
+  "output": " The grass is green. The sky is blue. The sun",
+  "containsAnswer": false,
+  "promptEvalMilliseconds": 288020.20,
+  "promptTokensPerSecond": 416.60,
+  "decodeMillisecondsPerToken": 20.86,
+  "decodeTokensPerSecond": 47.94,
+  "timeLPeakMemoryFootprintBytes": 1954515680,
+  "kvCache": "llama.cpp default (F16)",
+  "note": "Same token IDs as the Synapse run that answered only 6; both engines miss the 10%-depth key at 120k, which points to the model under YaRN x4, not to either engine."
+}