Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.
-
Updated
Nov 3, 2025 - Python
#
sub-quadratic-attention
Here are 6 public repositories matching this topic...
[NeurIPS 2025 Oral] Official Code for Exploring Diffusion Transformer Designs via Grafting
image-generation post-training self-attention convolutions diffusion-models grafting linear-attention text-to-image-generation architecture-research diffusion-transformer sub-quadratic-attention model-grafting hyena-operator model-architecture-editing diffusion-transformers architecture-editing hyena-x hyena-y mamba-2
-
Updated
Jan 9, 2026 - Jupyter Notebook
The Cerebros package is an ultra-precise Neural Architecture Search (NAS) / AutoML that is intended to much more closely mimic biological neurons than conventional neural network architecture strategies.
machine-learning deep-learning neural-network tensorflow keras python3 hyperparameter-optimization accuracy gpt automl keras-tensorflow neural-architecture-search mean-square-error automl-algorithms automl-experiments automl-products automl-benchmark automl-pipeline llm sub-quadratic-attention
-
Updated
Jan 23, 2026 - Jupyter Notebook
A PyTorch implementation of MEGABYTE. This multi-scale transformer architecture has the excellent features of tokenization-free and sub-quadratic attention. The paper link: https://arxiv.org/abs/2305.07185
-
Updated
Feb 6, 2024 - Python
Conformal Geometric Algebra (CGA) with efficient sequence modeling by introducing a recurrent rotor mechanism and a novel bit-masked hardware kernel that solves the computational bottleneck of Clifford products.
geometry mathematical-physics clifford-algebras geometric-deep-learning explainable-ai sequence-modeling scientific-machine-learning paradigm-shift efficient-deep-learning sub-quadratic-attention isotropic-architecture ai-interpretability physical-ai manifold-constrained-recurrence
-
Updated
Feb 6, 2026 - Python
Pure PyTorch + 🤗 Transformers reimplementation of Megalodon (CEMA + chunked attention) - readable, hackable, no CUDA kernels required
pytorch rope ema pytorch-implementation linear-attention efficient-transformers llm sub-quadratic-attention long-context-modeling streaming-inference complex-ema
-
Updated
Jan 12, 2026 - Python
Improve this page
Add a description, image, and links to the sub-quadratic-attention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sub-quadratic-attention topic, visit your repo's landing page and select "manage topics."