Conversation
… fallback Three comments name Blackwell sm_120 as an architecture Triton/Inductor does not support. Measured on an sm_120 device with the pinned torch 2.8.0+cu128 and triton 3.4.0: torch.compile default and reduce-overhead (the cudagraph_trees path debpalash#278 names), an attention module over growing sequence lengths, and a raw Triton kernel all run, with compiled output matching eager (maxdiff 1.19e-06 and 0.0). The app's own probe agrees — arch_unsupported returns None, so compile is already attempted there. The error text debpalash#278 quotes is also misattributed. "Detected that you are using FX to symbolically trace a dynamo-optimized function" reproduces with CUDA unavailable: Dynamo raises it whenever FX traces a compiled function, regardless of device. It belongs in the compile-stack classifier, not in the evidence for a missing-architecture failure. Comments only. The fallback contract and the arch-list gate are unchanged and still correct: the gate is generic rather than a Blackwell blocklist, and on a build whose arch list lacks the device the described mechanism holds. Only the example and the FX attribution are stale. Signed-off-by: Moep90 <volleyballlive@googlemail.com> Signed-off-by: moep90 <volleyballlive@googlemail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 10 included reviews per hour; 5 remain after this review. 📝 WalkthroughWalkthroughThe pull request updates service documentation, explanatory comments, and test documentation. It records current ChangesCompile fallback documentation
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~5 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to This PR updates compile-fallback documentation and comments without changing runtime behavior or test logic, so it is mergeable. 🚥 Pre-merge checks | ✅ 8 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (8 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
 The PR should not merge until the documented architecture-mismatch failure path is aligned with the actual pre-compile gate.
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| On a GPU architecture missing from the running torch build's arch list, | ||
| `torch.compile` succeeds at load time but the *first generation* dies inside | ||
| the Dynamo/FX/Inductor stack (an AssertionError out of | ||
| torch/_inductor/cudagraph_trees.py) and was mislabeled as an OOM. |
There was a problem hiding this comment.
The revised narrative says a missing architecture reaches first generation and triggers the cudagraph failure, but should_torch_compile() rejects that mismatch before torch.compile() is applied, so this cannot occur during normal execution. Describe the assertion as an independent compile-stack failure, or explicitly limit this scenario to configurations that force compilation past the architecture gate.
Knowledge Base Used:
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Three comments justify the #278 compile fallback by naming Blackwell
sm_120as an architecture Triton/Inductor does not support. On the pinned build it is supported, and the error text they quote is not an architecture symptom at all.Evidence
Measured on an sm_120 Blackwell GPU (capability
(12, 0)) with the repo's pinned environment,torch 2.8.0+cu128/triton 3.4.0:torch.compiledefault, incl. recompile on a new shapemode="reduce-overhead"×3 — thecudagraph_trees.pypath #278 namesallclose=True, maxdiff1.19e-06and0.0The app's own gate agrees:
arch_unsupported(torch)returnsNoneand_cuda_arch_supported_for_compile()returns(True, ""), so VoiceStudio already attempts compile on sm_120 with the pinned build — and per the table that is fine.The quoted error is device-independent
#278 quotes:
That still reproduces — and reproduces identically with
CUDA_VISIBLE_DEVICES=""and CUDA unavailable. Dynamo raises it unconditionally whenever FX traces a compiled function (torch/_dynamo/eval_frame.py). It is a compile-stack error the classifier should keep catching, but it was never evidence of a missing-architecture problem.What changes
tests/test_compile_fallback.pybackend/services/model_manager.py_is_compile_runtime_failurebackend/services/engine_env.py_cuda_arch_supported_for_compileWhat does not change
No behaviour, no test logic, no gate. The fallback contract — compile is an optimization, never a point of failure — is correct regardless of which architecture exposed it, and the arch-list gate is already generic rather than a Blackwell blocklist. I am not claiming the #278 reports were invalid: on a build whose arch list lacks the device, the described mechanism is exactly right. What is stale is the example and the FX attribution.
Scope limit: the tests exercise synthetic modules, not the shipped TTS engines, and a
cudagraph_treesAssertionError not reproducing here is not proof it cannot occur with a specific model. That is precisely why the fallback stays.Verification
Follows #2084, which corrects the neighbouring claim that the pinned torch lacks sm_120 kernels. Independent of it — different issue (#278 vs #1931), different files, no overlap.
Updated compile-fallback documentation to state that pinned
torch 2.8.0+cu128andtriton 3.4.0support Blackwellsm_120, and that the FX tracing error is device-independent. No behavior or architecture gates changed; review risk is limited to documentation accuracy.