SuperNPUBench

SuperNPUBench is a high-performance operator library and benchmark platform for NPU tile-programming ISA. It ships two architecture backends under benchmark/ (two-level-arch = LinxISA, one-level-arch = PTO ISA) plus an instruction-level microbenchmark suite, all driven by the same Linx toolchain.

IMPORTANT: Only benchmark/one-level-arch/ and microbenchmark/ are compilable with the current toolchain. The benchmark/two-level-arch/ (LinxISA) kernels are not compilable — they require a different ISA mode not supported by the current linx_blockisa_llvm_musl build. Do not include two-level-arch in batch compilation (compile_all.sh two-level will fail).

Repository Structure

SuperNPUBench/
├── benchmark/
│   ├── two-level-arch/      # Linx two-level block ISA
│   │   ├── kernels/         # header-only operator implementations
│   │   ├── test/            # test suites + build system
│   │   └── compile_all.sh
│   ├── one-level-arch/      # PTO one-level tile ISA
│   │   ├── kernels/
│   │   ├── test/
│   │   │   ├── common/      # shared Makefile.common, _start.s
│   │   │   └── kernel/      # per-operator test cases
│   │   └── compile_all.sh
├── microbenchmark/          # instruction-level micro-bench (cube/vector/memory/scalar)
├── docs/                    # MkDocs documentation site
│   ├── content/            # programming guide, tutorials, intrinsics, benchmarks
│   ├── mkdocs.yml
│   └── build.sh
└── compile_all.sh           # top-level: two-level | one-level | all

Build outputs (output/, **/output/) and .DS_Store are gitignored.

Architecture Backends

two-level-arch (LinxISA)

Block-structured ISA with heterogeneous cores: BCC (main), Cube (matrix), Vector, MTC/TMA (data transfer).
Programming model: block instructions (VPAR/VSEQ, CUBE, TMA, TEPL).

one-level-arch (PTO ISA)

Tile-centric ISA with explicit memory hierarchy: Vec, Mat, Left, Right, Acc.
Programming model: tile operations via Linx-TileOP-API C++ templates.
Programming guide: docs/content/programming/cpp-programming-guide.md.

Both backends share the same operator set and test layout; their kernel implementations differ in ISA style.

Operator Overview

Each backend implements operator categories:

Operator	Description
matmul	FP4/BF16/FP32/FP16/FP8 matrix multiply; quantization, mixed precision, A/B reuse, GMMA shared-tile
fa	Flash Attention; 2D unroll, SFA (block-sparse), HIF4 quantization, softmax_pto, unaligned boundary
flashMLA	Flash MLA (multi-head latent attention)
transpose	3D~6D tensor transpose; multiple dtypes
reduction	Row/column max & sum; single-tree, unaligned, cumsum, reduceprod
gelu	GELU activation; exact (erf) and tanh approximation
broadcast	2D~5D broadcast; vectorized variants
gather	Data gathering; large-scale, power-of-2 dims
concat	Concatenation; gather/scatter modes
control	`hashtable_lookup_simd` (pure tile-op, single-tier gfsim)
sort	`topk` (radix-bucket histogram)
deepseek	22 migrated DeepSeek kernels (engram/mhc/moe/quant/transpose)

Setup Environment

SuperNPUBench compiles with the Linx toolchain (linx_blockisa_llvm_musl, clang-15, target linx64v5-unknown-linux-musl). Build it once from the linx-toolchain-build repo, which clones the matching ISA sources and produces the linx_blockisa_llvm_musl install tree that COMPILER_DIR points at.

1. Clone the build repo

git clone https://github.com/LinxISA/linx-toolchain-build.git
cd linx-toolchain-build

2. Install host build tools

sudo apt-get install -y git make cmake ninja-build gcc g++ python3 autoconf m4

3. Initialize component sources

make init-src clones the five component repos under src/ on their pinned branches (run it again any time to fetch updates):

Directory	Repository	Branch
`src/llvm-project`	`LinxISA/llvm-project`	`dev-llvm15_56`
`src/musl`	`LinxISA/linx-musl`	`linx`
`src/jemalloc`	`LinxISA/jemalloc`	`linx`
`src/linux-linxisa`	`LinxISA/linux`	`main`
`src/Linx-TileOP-API`	`LinxISA/Linx-TileOP-API`	`linx`

make init-src

4. Build the toolchain

Only linx64v5-linux-musl is supported by the top-level Makefile:

make WITH_TARGET=linx64v5-linux-musl

This builds, in order: LLVM/clang/lld → kernel headers → musl → compiler-rt → libc++/libc++abi/libunwind → jemalloc → Linx-TileOP-API headers. Progress is tracked by stamp files under stamps/, so re-running make resumes from the last completed step; make clean rebuilds from scratch. The install tree is written to output/linx_blockisa_llvm_musl/:

output/linx_blockisa_llvm_musl/
├── bin/        # clang, clang++, ld.lld, llvm-ar/nm/ranlib,
│              # linx64v5-linux-musl-clang(++) symlinks
├── lib/        # clang runtime, libc++, ...
└── sysroot/    # musl + kernel headers + runtime libs

5. Point SuperNPUBench at the toolchain

export COMPILER_DIR=$(pwd)/output/linx_blockisa_llvm_musl/bin
$COMPILER_DIR/clang --version
# clang version 15.0.4 (linx64v5-musl-local ...)
# Target: linx64v5-unknown-linux-musl

Then proceed to Quick Start.

(Optional) Package

make package     # -> output/linx_blockisa_llvm_musl.tar.gz

Quick Start

1. Environment

Build the Linx toolchain once (see Setup Environment), then point COMPILER_DIR at it:

export COMPILER_DIR=/path/to/linx_blockisa_llvm_musl/bin

2. Compile an operator

# one-level-arch (PTO ISA)
cd benchmark/one-level-arch/test/kernel/matmul
make TESTCASE=matmul TYPE=MASK MODE=MASK_FP32 M=256 N=256 K=256 tM=16 tN=16 tK=64

# deepseek kernel
cd benchmark/one-level-arch/test/kernel/deepseek
make TESTCASE=fused_weight diss

3. Batch / full compilation

# one-level-arch only (recommended)
./compile_all.sh one-level

# microbenchmark
cd microbenchmark && bash compile_all.sh all

Do NOT run compile_all.sh two-level or compile_all.sh all — two-level-arch kernels cannot compile with the current toolchain.

Artifacts land in benchmark/<arch>/output/kernel/<operator>/elf/.

Microbenchmark

microbenchmark/ is an instruction-level bench organized by ISA family, generated by gen_cases.py.

family	covers	cases
cube (CUBE)	TMATMUL / TMATMUL_BIAS / TMATMUL_MX / ACCCVT	9
vector (TEPL)	elementwise / tile-scalar / reduce / expand (toolchain-exposed subset)	126
memory (TLSU)	TLOAD / TSTORE / TMOV / MGATHER / MSCATTER (+mask, layout)	25
scalar (GPR)	int ALU / load-store / float / conversion × throughput+latency	124
total		284

cd microbenchmark && make TESTCASE=tmatmul_fp16_64x64x64   # one case
cd microbenchmark && bash compile_all.sh all               # all families

See microbenchmark/README.md for details.

Running on the Models

Compiled ELF binaries run on the SuperScalarModel simulator suite. Build gfrun/gfsim from the SuperScalarModel repo, then point them at the ELF:

gfrun — functional model (correctness)
gfsim — cycle-accurate model (timing)

# from the SuperScalarModel repo root (where bin/ lives)
bin/gfrun -f /path/to/SuperNPUBench/benchmark/one-level-arch/output/kernel/<op>/elf/<name>.elf
bin/gfsim -f /path/to/SuperNPUBench/benchmark/one-level-arch/output/kernel/<op>/elf/<name>.elf

Tile-op kernels: single-tier gfsim mode

Kernels written purely with tile ops using TEPL template instructions (e.g. control/hashtable_lookup_simd) run on the VectorLite engine, which gfsim only steps in single-tier mode:

bin/gfsim -f <elf> -s core.singleTierMode=true

Without this flag the engine is inert and the run deadlocks. gfrun does not need the flag.

Build System

Makefile parameters

Parameter	Description	Example
`TESTCASE`	Test case name	`matmul`, `fa_2d_unroll`
`TYPE`	Operator type (matmul)	`HIF4_HIF4`, `A16W4`, `MASK`
`MODE`	Operator mode	`MASK_FP32`, `BF16x2_NOGATHER`
`M/N/K`	Matrix dimensions	`M=256 N=2048 K=2048`
`tM/tN/tK`	Tile sizes	`tM=128 tN=128 tK=128`
`COMPILER_DIR`	Compiler path	`/path/to/linx/bin`
`PLAT`	Platform	`linx` (default), `cpu`

Build targets

make TESTCASE=<case> all      # compile
make TESTCASE=<case> diss     # disassembly
make TESTCASE=<case> sim      # run in simulator
make TESTCASE=<case> debug    # debug mode
make clean                    # clean current operator
make clean_all                # clean all

Documentation

PTO C++ Programming Guide: docs/programming/pto c++ programming guide.md
End-to-end Workflow: docs/workflow/operator_to_chip_execution_flow.md
Per-operator README: see benchmark/one-level-arch/kernels/<operator>/README.md
Microbenchmark: microbenchmark/README.md
TileOP-API Reference: Linx-TileOP-API tileop-usage docs

Toolchain

Compiler: linx_blockisa_llvm_musl (clang-15, linx64v5-musl)
Flags: -mlxbc -fenable-matrix -O2 -mllvm -enable-all-vector-as-tilereg=true -std=c++20
Target: Linx64 V5

Development Guide

Adding an operator

Add header-only kernel under benchmark/<arch>/kernels/<operator>/.
Create test dir under benchmark/<arch>/test/kernel/<operator>/ with Makefile, compile.all, src/.
Add the operator to compile_all.sh.

Conventions

Header-only kernels; PTO tile-programming paradigm.
Build artifacts not tracked (.gitignore).

License

See LICENSE.

Navigation Menu

Repository files navigation

SuperNPUBench

Repository Structure

Architecture Backends

two-level-arch (LinxISA)

one-level-arch (PTO ISA)

Operator Overview

Setup Environment

1. Clone the build repo

2. Install host build tools

3. Initialize component sources

4. Build the toolchain

5. Point SuperNPUBench at the toolchain

(Optional) Package

Quick Start

1. Environment

2. Compile an operator

3. Batch / full compilation

Microbenchmark

Running on the Models

Tile-op kernels: single-tier gfsim mode

Build System

Makefile parameters

Build targets

Documentation

Toolchain

Development Guide

Adding an operator

Conventions

Related Links

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
benchmark		benchmark
docs		docs
microbenchmark		microbenchmark
.gitignore		.gitignore
README.md		README.md
compile_all.sh		compile_all.sh

Latest commit

History

Folders and files

Repository files navigation

SuperNPUBench

Repository Structure

Architecture Backends

two-level-arch (LinxISA)

one-level-arch (PTO ISA)

Operator Overview

Setup Environment

1. Clone the build repo

2. Install host build tools

3. Initialize component sources

4. Build the toolchain

5. Point SuperNPUBench at the toolchain

(Optional) Package

Quick Start

1. Environment

2. Compile an operator

3. Batch / full compilation

Microbenchmark

Running on the Models

Tile-op kernels: single-tier gfsim mode

Build System

Makefile parameters

Build targets

Documentation

Toolchain

Development Guide

Adding an operator

Conventions

Related Links

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages