Guiding Token-Sparse Diffusion Models

Felix Krause · Stefan Andreas Baumann · Johannes Schusterbauer · Olga Grebenkova · Ming Gui · Vincent Tao Hu · Björn Ommer

CompVis @ LMU Munich, Munich Center for Machine Learning (MCML)

TL;DR: Token-sparse diffusion models (masking/routing) train fast and can be strong conditionals, but Classifier-free Guidance (CFG) often breaks at inference. We introduce Sparse Guidance (SG), a finetune-free guidance rule that uses token sparsity as the guidance signal: combine a strong low-sparsity conditional prediction with a weak high-sparsity conditional prediction. On ImageNet-256, SG reaches 1.58 FID with 25% fewer FLOPs and enables up to 58% FLOP savings at matched baseline quality. SG also scales to a 2.5B text-to-image model, improving human preference and throughput.

Note

This repository is a landing page for Sparse Guidance and primarily exists to host the project website.

➡️ The official code lives in CompVis/tread.
Please refer to that repository for code, setup, and inference instructions.

🎓 Citation

@misc{krause2026guidingtokensparsediffusionmodels,
      title={Guiding Token-Sparse Diffusion Models}, 
      author={Felix Krause and Stefan Andreas Baumann and Johannes Schusterbauer and Olga Grebenkova and Ming Gui and Vincent Tao Hu and Björn Ommer},
      year={2026},
      eprint={2601.01608},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2601.01608}, 
}