Load quantized weights per component by xiaoyu-work · Pull Request #669 · onnxruntime/mobius

xiaoyu-work · 2026-08-27T02:31:36Z

Summary

parse an authoritative component_quantization plan for each package component
map component-local module paths to candidate HuggingFace full module names
apply exact and re: modules_to_not_convert rules per projection, rather than collapsing the whole component
keep excluded per_layer_input_gate and per_layer_projection modules floating point while quantizing the rest of the decoder
normalize existing Olive/GPTQ/AWQ packed sidecars through the typed codec layer; never quantize float weights
fail closed for specialized quantized subclasses, packed experts/tied tables awaiting model adapters, and unrouted packed weights

Stack

Design: Design per-component quantized checkpoint loading #664
Component manifest: Add canonical component manifests #665
Depends on typed weight pipeline: Add typed checkpoint weight records #666
This PR: per-component loader
Next: model-specific adapter migration

Validation

331 focused loader, config, component, Gemma4 exclusion, and all-model matrix tests
104 independent-layout builds across VL, speech, and seq2seq model configurations
77 non-Base config and diffusers compatibility tests

github-actions · 2026-08-27T02:33:35Z

Performance Comparison

Comparing 15c0dfc → e5ce365

Model	Metric	Baseline	Current	Delta
bert (feature-extraction)	model_size_bytes	359 KB	359 KB	+0.0%	⚪
bert (feature-extraction)	num_nodes	68	68	+0.0%	⚪
falcon	model_size_bytes	364 KB	364 KB	+0.0%	⚪
falcon	num_nodes	66	66	+0.0%	⚪
gemma2	model_size_bytes	428 KB	428 KB	+0.0%	⚪
gemma2	num_nodes	105	105	+0.0%	⚪
gpt2	model_size_bytes	324 KB	324 KB	+0.0%	⚪
gpt2	num_nodes	54	54	+0.0%	⚪
llama	model_size_bytes	425 KB	425 KB	+0.0%	⚪
llama	num_nodes	60	60	+0.0%	⚪
llama (static-cache)	model_size_bytes	425 KB	425 KB	+0.0%	⚪
llama (static-cache)	num_nodes	56	56	+0.0%	⚪
mamba (ssm-text-generation)	model_size_bytes	296 KB	296 KB	+0.0%	⚪
mamba (ssm-text-generation)	num_nodes	94	94	+0.0%	⚪
phi3	model_size_bytes	421 KB	421 KB	+0.0%	⚪
phi3	num_nodes	58	58	+0.0%	⚪
phi3 (static-cache)	model_size_bytes	421 KB	421 KB	+0.0%	⚪
phi3 (static-cache)	num_nodes	54	54	+0.0%	⚪
qwen2	model_size_bytes	425 KB	425 KB	+0.0%	⚪
qwen2	num_nodes	60	60	+0.0%	⚪
qwen2 (static-cache)	model_size_bytes	425 KB	425 KB	+0.0%	⚪
qwen2 (static-cache)	num_nodes	56	56	+0.0%	⚪
qwen3_5_moe (hybrid-text-generation)	model_size_bytes	506 KB	506 KB	+0.0%	⚪
qwen3_5_moe (hybrid-text-generation)	num_nodes	265	265	+0.0%	⚪
qwen3_5_text (hybrid-text-generation)	model_size_bytes	458 KB	458 KB	+0.0%	⚪
qwen3_5_text (hybrid-text-generation)	num_nodes	127	127	+0.0%	⚪
qwen3_5_vl (hybrid-qwen-vl)	model_size_bytes	977 KB	977 KB	+0.0%	⚪
qwen3_5_vl (hybrid-qwen-vl)	num_nodes	450	450	+0.0%	⚪
t5 (seq2seq)	model_size_bytes	836 KB	836 KB	+0.0%	⚪
t5 (seq2seq)	num_nodes	176	176	+0.0%	⚪
whisper (speech-to-text)	model_size_bytes	1008 KB	1008 KB	+0.0%	⚪
whisper (speech-to-text)	num_nodes	128	128	+0.0%	⚪

No performance regressions.

github-actions · 2026-08-27T02:36:47Z

🏗️ Architecture Diff

Comparing 15c0dfc → e5ce365

Model	Sub-model	Status
bert (feature-extraction)	model	⚪
falcon	model	⚪
gemma2	model	⚪
gemma4 (gemma4)	decoder	⚪
gemma4 (gemma4)	embedding	⚪
gemma4 (gemma4)	vision_encoder	⚪
gemma4_text	model	⚪
gpt2	model	⚪
llama	model	⚪
llama (static-cache)	model	⚪
mamba (ssm-text-generation)	model	⚪
phi3	model	⚪
phi3 (static-cache)	model	⚪
qwen	model	⚪
qwen (static-cache)	model	⚪
qwen2	model	⚪
qwen2 (static-cache)	model	⚪
qwen2_moe	model	⚪
qwen2_moe (static-cache)	model	⚪
qwen3	model	⚪
qwen3 (static-cache)	model	⚪
qwen3_5_moe (hybrid-text-generation)	model	⚪
qwen3_5_text (hybrid-text-generation)	model	⚪
qwen3_5_vl (hybrid-qwen-vl)	decoder	⚪
qwen3_5_vl (hybrid-qwen-vl)	embedding	⚪
qwen3_5_vl (hybrid-qwen-vl)	vision_encoder	⚪
qwen3_moe	model	⚪
qwen3_moe (static-cache)	model	⚪
qwen3_next (hybrid-text-generation)	model	⚪
t5 (seq2seq)	decoder	⚪
t5 (seq2seq)	encoder	⚪
whisper (speech-to-text)	decoder	⚪
whisper (speech-to-text)	encoder	⚪

No architecture changes detected. ✅

Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

Parse authoritative component quantization plans, map component-local modules to HuggingFace names, honor exact and regex float exclusions per projection, and normalize existing packed sidecars through typed codecs. Mobius does not quantize float weights. Signed-off-by: Xiaoyu Zhang <xiaoyuzhang@microsoft.com>

Apply Ruff formatting preserved across the manifest field rename rebase. Signed-off-by: Xiaoyu Zhang <xiaoyuzhang@microsoft.com>

xiaoyu-work · 2026-09-04T21:39:32Z

@copilot resolve the merge conflicts in this pull request

…into feat/per-component-quantization-loader # Conflicts: # CHANGELOG.md # src/mobius/__main__.py # src/mobius/_builder.py # src/mobius/_component_quantization.py # src/mobius/_component_quantization_test.py # src/mobius/_configs/_base.py # src/mobius/_configs/_quantization.py # src/mobius/_configs_test.py # src/mobius/components/__init__.py # src/mobius/components/_quantized_linear.py # src/mobius/integrations/transformers/_builder.py # src/mobius/integrations/transformers/_builder_test.py # tests/build_graph_test.py Co-authored-by: xiaoyu-work <85524621+xiaoyu-work@users.noreply.github.com>

…ilder Co-authored-by: xiaoyu-work <85524621+xiaoyu-work@users.noreply.github.com>

Copilot · 2026-09-04T22:11:35Z

@copilot resolve the merge conflicts in this pull request

Resolved merge conflicts with refactor/typed-weight-pipeline in commits e35b7cd and c6acc01. All tests have been verified and passed.

CLAassistant · 2026-09-04T22:11:36Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ xiaoyu-work
❌ Copilot
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from fabb835 to b1ea32f Compare August 27, 2026 02:41

xiaoyu-work force-pushed the feat/per-component-quantization-loader branch from e3efe95 to e409b2d Compare August 27, 2026 02:41

xiaoyu-work mentioned this pull request Aug 27, 2026

Migrate composite model weight adapters #670

Open

xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from b1ea32f to e02875f Compare August 27, 2026 21:06

xiaoyu-work force-pushed the feat/per-component-quantization-loader branch from e409b2d to a8a5c5b Compare August 27, 2026 21:08

xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from e02875f to ea9c144 Compare August 27, 2026 21:13

xiaoyu-work force-pushed the feat/per-component-quantization-loader branch from a8a5c5b to 7562458 Compare August 27, 2026 21:22

github-advanced-security Bot found potential problems Aug 27, 2026

View reviewed changes

Comment thread src/mobius/_component_quantization.py Fixed

xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from ea9c144 to 15c0dfc Compare August 27, 2026 21:30

xiaoyu-work added 2 commits August 27, 2026 14:30

Format component loader after restack

e5ce365

Apply Ruff formatting preserved across the manifest field rename rebase. Signed-off-by: Xiaoyu Zhang <xiaoyuzhang@microsoft.com>

xiaoyu-work force-pushed the feat/per-component-quantization-loader branch from 7562458 to e5ce365 Compare August 27, 2026 21:30

xiaoyu-work marked this pull request as ready for review September 4, 2026 20:54

Copilot started work on behalf of xiaoyu-work September 4, 2026 21:39 View session

Copilot AI and others added 2 commits September 4, 2026 21:51

Fix post-merge config and quantization resolution for transformers bu…

c6acc01

…ilder Co-authored-by: xiaoyu-work <85524621+xiaoyu-work@users.noreply.github.com>

Copilot finished work on behalf of xiaoyu-work September 4, 2026 22:11

Navigation Menu

Uh oh!

Load quantized weights per component - #669

Load quantized weights per component#669
xiaoyu-work wants to merge 4 commits into
refactor/typed-weight-pipelinefrom
feat/per-component-quantization-loader

xiaoyu-work commented Aug 27, 2026

Uh oh!

github-actions Bot commented Aug 27, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 27, 2026 •

edited

Loading

Uh oh!

Uh oh!

xiaoyu-work commented Sep 4, 2026

Uh oh!

Copilot AI commented Sep 4, 2026

Uh oh!

CLAassistant commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

xiaoyu-work commented Aug 27, 2026

Summary

Stack

Validation

Uh oh!

github-actions Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Comparison

Uh oh!

github-actions Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🏗️ Architecture Diff

Uh oh!

Uh oh!

xiaoyu-work commented Sep 4, 2026

Uh oh!

Copilot AI commented Sep 4, 2026

Uh oh!

CLAassistant commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Aug 27, 2026 •

edited

Loading

github-actions Bot commented Aug 27, 2026 •

edited

Loading