Model name
Qwen4Exp / Qwen3.8-Flash-Next
Hugging Face model or repository link
https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
Evidence revision: d3bc75ee6ccef3efc1e228ec00a6cc2cdb1e2249
Primary task
Text generation
Use case
Complete Mobius support for executing the released Qwen4Exp GGUF checkpoint. Mobius already supports the exact qwen4exp header, configuration, split-shard closure, 1,224-tensor shape/qtype contract, tensor-name mapping, and text graph. Payload conversion currently fails closed because there is no faithful executable import route.
The implementation should add one of these truthful bounded-memory routes:
- Native quantized execution for the
IQ4_NL per-layer token embedding and rank-3 routed expert banks using mixed IQ1_S gate/up and IQ4_NL down tensors; or
- Streaming/lazy dequantization and serialization that never materializes the approximately 191 GiB dense payload in memory.
Do not reinterpret these tensors as ordinary affine MatMulNBits; that would change their semantics.
Completion requires immutable-artifact weight parity, bounded-memory conversion evidence, ORT execution, cache/decode validation, and multi-token generation parity. Runtime/export metadata must remain fail closed until the required bindings and execution semantics are representable.
Execution providers
Target hardware
The conversion path must have bounded host memory and should support machines that cannot hold the approximately 191 GiB dense expansion. CUDA execution is the primary performance target; CPU is required as a correctness reference where practical.
Quantization requirements
Preserve the released GGUF semantics: IQ4_NL per-layer token embedding, rank-3 routed experts with IQ1_S gate/up tensors, and IQ4_NL down tensors. Any dequantized route must stream or serialize incrementally rather than materializing the full dense model.
Contribution
Additional context
The current design and fail-closed boundary are documented in docs/design/qwen4-exp-text-core.md. PR #698 only makes preflight/download diagnostics truthful; it does not implement GGUF payload execution.
Known downstream packaging gap: released ONNX Runtime GenAI metadata cannot yet represent ple_input_ids, four-axis position state, and heterogeneous per-layer PLE/QSA cache membership. This must not be reported as runtime-supported until the schema and bindings are faithful.
Model name
Qwen4Exp / Qwen3.8-Flash-Next
Hugging Face model or repository link
https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
Evidence revision:
d3bc75ee6ccef3efc1e228ec00a6cc2cdb1e2249Primary task
Text generation
Use case
Complete Mobius support for executing the released Qwen4Exp GGUF checkpoint. Mobius already supports the exact
qwen4expheader, configuration, split-shard closure, 1,224-tensor shape/qtype contract, tensor-name mapping, and text graph. Payload conversion currently fails closed because there is no faithful executable import route.The implementation should add one of these truthful bounded-memory routes:
IQ4_NLper-layer token embedding and rank-3 routed expert banks using mixedIQ1_Sgate/up andIQ4_NLdown tensors; orDo not reinterpret these tensors as ordinary affine
MatMulNBits; that would change their semantics.Completion requires immutable-artifact weight parity, bounded-memory conversion evidence, ORT execution, cache/decode validation, and multi-token generation parity. Runtime/export metadata must remain fail closed until the required bindings and execution semantics are representable.
Execution providers
Target hardware
The conversion path must have bounded host memory and should support machines that cannot hold the approximately 191 GiB dense expansion. CUDA execution is the primary performance target; CPU is required as a correctness reference where practical.
Quantization requirements
Preserve the released GGUF semantics:
IQ4_NLper-layer token embedding, rank-3 routed experts withIQ1_Sgate/up tensors, andIQ4_NLdown tensors. Any dequantized route must stream or serialize incrementally rather than materializing the full dense model.Contribution
Additional context
The current design and fail-closed boundary are documented in
docs/design/qwen4-exp-text-core.md. PR #698 only makes preflight/download diagnostics truthful; it does not implement GGUF payload execution.Known downstream packaging gap: released ONNX Runtime GenAI metadata cannot yet represent
ple_input_ids, four-axis position state, and heterogeneous per-layer PLE/QSA cache membership. This must not be reported as runtime-supported until the schema and bindings are faithful.