Feature request: aarch64 wheels + sm_121 (GB10 / DGX Spark) kernel support

[elizabetht]

Summary

Requesting aarch64 builds and sm_121 (GB10 / DGX Spark) kernel support.

TileRT is currently unusable on ARM64 + Grace-Blackwell hardware, and the failure mode is silent rather than diagnostic. Related: #22 asks for an ARM64 Docker image, but an image alone is not sufficient — the shipped kernels are compiled for sm_100f only, so I'm filing this with the specific binary details.

Environment

Machine	NVIDIA DGX Spark
Arch	aarch64
GPU	NVIDIA GB10, compute capability 12.1 (sm_121), 1 device
Unified memory	119 GiB
Driver	580.95.05, CUDA 13.0
OS	Ubuntu 24.04.3 LTS, kernel 6.11.0-1016-nvidia
Python	3.12.3

Problem 1: pip install tilert silently installs a placeholder

On aarch64, every real release is filtered out as platform-incompatible, so the only installable candidate is the 0.0.0.dev0 name-reservation package:

$ pip index versions tilert
ERROR: No matching distribution found for tilert

$ pip install tilert==0.1.5.post2
ERROR: Could not find a version that satisfies the requirement tilert==0.1.5.post2
       (from versions: 0.0.0.dev0)

A bare pip install tilert reports success, but installs a single 0-byte __init__.py:

$ cat .../site-packages/tilert-0.0.0.dev0.dist-info/RECORD
tilert/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0

The resulting failure is misleading — it looks like a path or packaging problem rather than "no build exists for your platform":

$ python3 -m tilert.models.preprocess.weight_converter --model_type ... --model_dir ...
Error while finding module specification for 'tilert.models.preprocess.weight_converter'
(ModuleNotFoundError: No module named 'tilert.models')

>>> import tilert; tilert.__version__
AttributeError: module 'tilert' has no attribute '__version__'

Note that pyproject.toml declares Operating System :: OS Independent, which reinforces the wrong conclusion.

Problem 2: no source-build path for a different architecture

The repo contains no C/C++/CUDA sources — the runtime ships as two prebuilt, stripped shared objects inside the wheel:

tilert/libtilert_dsv32.so    13.35 MB
tilert/libtilert_glm5.so     13.46 MB

$ file libtilert_dsv32.so
ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped

$ strings libtilert_dsv32.so | grep -oE 'sm_[0-9]+[a-z]?' | sort -u
sm_100f

$ readelf -d libtilert_dsv32.so | grep NEEDED
libtorch.so  libc10.so  libc10_cuda.so  libtorch_cuda.so
libcudart.so.13  libnvrtc.so.13  libcuda.so.1

So pip install -e . from a git checkout yields the Python layer with no loadable engine, and there is nothing in-tree to recompile for aarch64/sm_121.

sm_100f is a family-conditional target covering the Blackwell datacenter family (sm_100/sm_103). GB10 at sm_121 is outside that family, so forward-compatibility within sm_100f does not reach it even though both are branded Blackwell.

All distribution channels are x86_64-only:

• GitHub Releases: all 7 releases from v0.1.0-alpha.1 (2025-11) through v0.1.5.post2 (2026-08) are manylinux*_x86_64 / linux_x86_64
• Docker Hub tileai/tilert: cu132-latest and v0.1.0 are linux/amd64 only
• Dockerfile base is pytorch/manylinux2_28-builder:cuda13.2-main

Requests

1. aarch64 wheels (manylinux_2_28_aarch64) and a multi-arch Docker image, with kernels built for sm_121 in addition to sm_100f.
2. Guidance on single-GPU configurations. weight_converter shards weights into dev_0…dev_7 and the published benchmarks are all 8× B200. DGX Spark is a single GB10 with 119 GiB of unified memory. Is num_devices=1 a supported configuration for any current model, or is ≥8-way tensor parallelism architectural?
3. Smaller models for this class of hardware. Only deepseek-v32 and glm-5 are in tilert/models/. Neither fits on a single 128 GB unified-memory device. Is support for smaller checkpoints on the roadmap?

Short-term suggestions (independent of ARM support)

Even if ARM64 is out of scope, these would save users a debugging cycle:

• Add Programming Language :: Python-style platform metadata / correct the OS Independent classifier, and state the x86_64 + sm_100-family requirement explicitly in the Installation section of the README.
• Make the 0.0.0.dev0 placeholder fail loudly — e.g. have its __init__.py raise on import with a message pointing at the GitHub Releases wheel and the supported platform, instead of being an empty file. Right now it is indistinguishable at runtime from a broken install.