Feat: Enable structured output in Mini-SGLang

[YzXiao101]

Motivation

Mini-SGLang lacked structured output and reasoning-model-aware constrained decoding.

Description

Adds OpenAI response_format.json_schema support with XGrammar, async grammar compilation, constrained decoding, scheduler/TP integration, and reasoning-model support.

Callout

Initial JSON-schema structured output support only.

• Only OpenAI response_format={"type":"json_schema", ...} is supported.
• The backend is XGrammar only, pinned as xgrammar==0.1.27.
• Not supported in this PR: regex, EBNF, choices, tool/function calling, or alternate backends.
• Grammar masking is deferred until after the thinking phase completes — disabling thinking at req level is not yet supported, so this applies for all requests to a hybrid reasoning model.
• For overlap-scheduling correctness, constrained requests may delay sampling at the cost of less overlap and lower throughput.

Test & Performance

Benchmark config:

• Dataset: NousResearch/json-mode-eval
• Requested samples: 100; filtered benchmark requests: 94
• max_tokens: 4096
• Sampling: greedy (temperature=0.0, top_k=1)
• Runtime: CUDA graph enabled, overlap scheduling enabled, MINISGL_TORCH_NUM_THREADS=1
• Hardware (nvidia-smi):

Tue Feb 24 03:35:54 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.127.05             Driver Version: 550.127.05     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:01:00.0 Off |                  Off |
|  0%   51C    P2            301W /  450W |   22698MiB /  24564MiB |    100%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

Model	Mode	Requests	Output length p50 / p90 / max	Actual output tokens	Time (s)	Throughput (tok/s)	JSON parse	Schema valid
Qwen/Qwen2-0.5B	constrained	94	64 / 139 / 4096	26,883	8.55	3,143.56	89/94 (94.7%)	89/94 (94.7%)
Qwen/Qwen2-0.5B	unconstrained	94	1006 / 4095 / 4095	135,203	11.04	12,242.84	0/94 (0.0%)	0/94 (0.0%)
Qwen/Qwen2-1.5B	constrained	94	66 / 143 / 4096	30,890	21.71	1,422.95	88/94 (93.6%)	88/94 (93.6%)
Qwen/Qwen2-1.5B	unconstrained	94	937 / 4095 / 4095	154,873	27.28	5,677.50	0/94 (0.0%)	0/94 (0.0%)
Qwen/Qwen2-7B	constrained	94	63 / 130 / 4096	14,877	69.14	215.17	92/94 (97.9%)	92/94 (97.9%)
Qwen/Qwen2-7B	unconstrained	94	1076 / 1231 / 4095	106,297	87.83	1,210.24	0/94 (0.0%)	0/94 (0.0%)

Notes:

• Wall time is not a pure overhead comparison because constrained outputs are much shorter.
• Remaining failures are concentrated in long-tail max_tokens=4096 cases.

Performance Analysis

Nsight profile: Qwen/Qwen2-0.5B, 100 samples, max_tokens=4096, CUDA graph enabled, overlap scheduling enabled.

Mode	Output tokens	Time (s)	Throughput (tok/s)	Output length p50 / p90 / max
constrained	26,785	12.69	2,110.91	66 / 139 / 4096
unconstrained	129,705	13.17	9,846.18	963 / 4095 / 4095

Nsight item	Constrained	Unconstrained
Sampler total	5.14s	0.21s
apply_grammar_mask	4.94s	N/A
fill_vocab_mask	3.76s	N/A
move_apply_vocab_mask	1.02s	N/A

Obvious bottleneck: CPU-side XGrammar mask filling.

Checklist

• Basic feature w/o overlap schedule.
• Adaptation for overlap schedule.
• Format code with pre-commit.
• Pass all unit tests.
• Provide speed benchmark results.
• Follow the minisgl code style.