feat(qwen3): genai bundle generation#996

Merged

DingmaomaoBJTU merged 30 commits into

mainfrom

pr/836/feature/qwen3-quant

Jul 3, 2026

Commits on Jul 1, 2026

feat(qwen3): add genai bundle generation and inference script
github-actions[bot]
committed
refactor(qwen3/genai): generic build_genai_config with ONNX introspection
github-actions[bot]
committed
feat(session): add GenaiSession for onnxruntime-genai inference
github-actions[bot]
committed
feat(session): add GenaiSession.apply_chatml_template static method
github-actions[bot]
committed
feat(qwen3/genai): NPU+CPU hybrid EP support in genai_config
github-actions[bot]
committed
fix(genai_session): let genai_config.json drive EP routing, add mixed EP
github-actions[bot]
committed
feat: add --compile flag to infer_genai.py for EPContext pre-compilation
github-actions[bot]
committed
fix: resolve prefill compilation hang by forcing htp_graph_finalization_optimization_mode=0
github-actions[bot]
committed
fix: move _do_compile to module-level _qnn_compile_worker for Windows spawn
github-actions[bot]
committed
perf: use configured htp_graph_finalization_optimization_mode for gen stages
github-actions[bot]
committed
simplify: remove htp_graph_finalization_optimization_mode override in _compile_stage
github-actions[bot]
committed
refactor(genai): move generic bundle logic to utils/genai, qwen3/genai as shim
github-actions[bot]
committed
fix: address code review issues in genai bundle and session
github-actions[bot]
committed
fix: import rule violations, fallback-stage path bug, and div-by-zero in perf print
github-actions[bot]
committed
feat(genai): auto-build embeddings+lm_head in export script using new model types
github-actions[bot]
committed
fix(genai): patch embeddings+lm_head seq_len to dynamic; revert cpu-override; fix shape_config
github-actions[bot]
committed
fix(genai): mirror non-QNN ONNX files into compiled bundle
github-actions[bot]
committed
refactor(scripts): unify Qwen3 export + inference into qwen3.py
github-actions[bot]
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(qwen3): genai bundle generation#996

feat(qwen3): genai bundle generation#996
DingmaomaoBJTU merged 30 commits into
mainfrom
pr/836/feature/qwen3-quant

Commits on Jul 1, 2026

feat(qwen3): add genai bundle generation and inference script

refactor(qwen3/genai): generic build_genai_config with ONNX introspection

feat(session): add GenaiSession for onnxruntime-genai inference

feat(session): add GenaiSession.apply_chatml_template static method

feat(qwen3/genai): NPU+CPU hybrid EP support in genai_config

fix(genai_session): let genai_config.json drive EP routing, add mixed EP

feat: add --compile flag to infer_genai.py for EPContext pre-compilation

fix: resolve prefill compilation hang by forcing htp_graph_finalization_optimization_mode=0

fix: move _do_compile to module-level _qnn_compile_worker for Windows spawn

perf: use configured htp_graph_finalization_optimization_mode for gen stages

simplify: remove htp_graph_finalization_optimization_mode override in _compile_stage

refactor(genai): move generic bundle logic to utils/genai, qwen3/genai as shim

fix: address code review issues in genai bundle and session

fix: import rule violations, fallback-stage path bug, and div-by-zero in perf print

feat(genai): auto-build embeddings+lm_head in export script using new model types

fix(genai): patch embeddings+lm_head seq_len to dynamic; revert cpu-override; fix shape_config

fix(genai): mirror non-QNN ONNX files into compiled bundle

refactor(scripts): unify Qwen3 export + inference into qwen3.py

Commits on Jul 3, 2026

refactor(genai): make bundle machinery EP-agnostic; move QNN into qwen3

fix(quant): use uint8 activations for transformer-only w8a8 (matches reference model)

revert(quant): restore w8a16 (uint16 activations) for transformer-only

Strip exporter-injected default GQA attrs from transformer bundle ONNX

Truncate cos/sin rope cache to max_cache_len at export time

fix: pin rope cache to static Python int to fix symbolic tracing bug

fix(scripts): drop broken genai session import and dead _SUPPORTED_EPS

Merge origin/main into qwen3 genai bundle branch

fix(qwen3): forward transformer_onnx_passes through genai bundle wrapper

fix(genai): preserve pad_token_id==0 and clear CodeQL import/statement alerts

docs(qwen3): note deferred max_rope_len wiring at the transformer-only wrapper

fix(qwen3): route export CLI via set_defaults(func) dispatch