Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
6337861
feat(sana): add SANA-WM integration shim
aidanfnv Jul 15, 2026
f504eac
docs(sana): document upstream env install flow
aidanfnv Jul 15, 2026
41ee2bb
docs(sana): clarify boolean CLI values
aidanfnv Jul 15, 2026
7e6af88
docs(sana): record validated BF16 smoke tests
aidanfnv Jul 16, 2026
de39905
feat(sana): add FP8 and FP4 precision options
aidanfnv Jul 16, 2026
a710bd0
Add native SANA-WM Stage-1 FlashDreams integration
aidanfnv Jul 16, 2026
5c87bfd
Reduce native SANA-WM Stage-1 memory use
aidanfnv Jul 17, 2026
c00b3c5
Fix SANA-WM VAE retry temporal stride
aidanfnv Jul 17, 2026
7de18d6
Run native SANA VAE decode in inference mode
aidanfnv Jul 17, 2026
46b6e66
Implement native SANA-WM refinement
aidanfnv Jul 17, 2026
0f77ccf
Run native SANA refiner under inference mode
aidanfnv Jul 17, 2026
52617e8
Clean up SANA-WM integration surface
aidanfnv Jul 17, 2026
26a66be
Harden SANA-WM FlashDreams integration boundaries
aidanfnv Jul 17, 2026
80ef815
Keep SANA-WM models resident across reuse without inflating peak memory
aidanfnv Jul 17, 2026
4e4a8cd
Derive SANA-WM intrinsics from the first frame when none provided
aidanfnv Jul 17, 2026
5449238
Make SANA integration obey FlashDreams contracts
aidanfnv Jul 17, 2026
5a82a1e
Fit SANA trajectories to requested frame count
aidanfnv Jul 17, 2026
2368785
Merge remote-tracking branch 'origin/main' into dev/aidanf/sana_integ…
aidanfnv Jul 20, 2026
3f4bbfd
Adapt runner for refactor
aidanfnv Jul 20, 2026
eb6f6e9
Merge branch 'main' into dev/aidanf/sana_integration
aidanfnv Jul 21, 2026
e26d882
Prune dead env-var code
aidanfnv Jul 21, 2026
2fe96af
Add WIP model card
aidanfnv Jul 22, 2026
ec6031e
Fix FP4 quality using NVFP4 scaling
aidanfnv Jul 22, 2026
87099d2
Merge branch 'main' into dev/aidanf/sana_integration
aidanfnv Jul 23, 2026
8c6a6e4
Add benchmarking scripts
aidanfnv Jul 24, 2026
69c4daa
Remove local docs
aidanfnv Jul 24, 2026
8a3ef24
More benchmarking scripts, missing from prev commits
aidanfnv Jul 24, 2026
9b48f41
Optmizations to catch up to upstream
aidanfnv Jul 24, 2026
a9f56a4
Split benchmarks by GPU, chnage metrics, update model card
aidanfnv Jul 24, 2026
9372087
Remove uupstream TE dep
aidanfnv Jul 24, 2026
1435fc0
Update data after Claude review
aidanfnv Jul 27, 2026
edf9166
Uaw GB300 numbers for model card
aidanfnv Jul 27, 2026
fd6faa6
Fix bench labels
aidanfnv Jul 27, 2026
67ed409
Update result to use refiner
aidanfnv Jul 27, 2026
3af585f
Add some logging for debug
aidanfnv Jul 27, 2026
bbfec5c
Clean old bench runs
aidanfnv Jul 27, 2026
51767ea
Strip out upstream fused kernels
aidanfnv Jul 27, 2026
36d2d8a
Optimizations
aidanfnv Jul 28, 2026
4c1d857
Optimize SANA-WM Stage 1 kernels
aidanfnv Jul 29, 2026
85c7065
Merge branch 'main' into dev/aidanf/sana_integration
aidanfnv Jul 29, 2026
3eaaa8f
Update benchmark scripts for _streaming upstream
aidanfnv Jul 29, 2026
0091ce1
Add integration for SANA-WM_streaming variant
aidanfnv Jul 29, 2026
ec508c5
Rename SANA benchmark native helper to FlashDreams
aidanfnv Jul 29, 2026
a89e40f
Fix benchmarks hopefully
aidanfnv Jul 30, 2026
48b6f8b
Wire up streaming variant to benchmark scripts
aidanfnv Jul 30, 2026
1d4fdd8
Tighten scripts, split streaming and bidirectional model cards
aidanfnv Jul 30, 2026
9a1760d
Merge branch 'main' into dev/aidanf/sana_integration
aidanfnv Jul 30, 2026
8cc3bf7
Add note to model cards
aidanfnv Jul 30, 2026
3cad01d
Optimize streaming
aidanfnv Jul 30, 2026
2c5304f
Handle TE depedencies, re-enforce naming convention
aidanfnv Jul 30, 2026
22347c2
Run benchmark runs in proper sequence
aidanfnv Jul 30, 2026
38e84be
Update model card data
aidanfnv Jul 30, 2026
174e4b5
Benchmark changes/fixes
aidanfnv Jul 31, 2026
3fede3f
Fix issues running on fresh machine?
aidanfnv Jul 31, 2026
416fbb5
Revert "Fix issues running on fresh machine?"
aidanfnv Jul 31, 2026
284202d
Fix issues running on fresh machine (for real)
aidanfnv Jul 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SANA-WM_bidirectional BF16 Benchmark Data (ms)

| device | official | flashdreams |
| --- | ---: | ---: |
| GB202 | 101874.59 | 96353.88 |
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SANA-WM_streaming BF16 Benchmark Data (ms/chunk)

| device | official | flashdreams |
| --- | ---: | ---: |
| GB202 | 1849.65 | 3660.15 |
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SANA-WM_streaming FP4 Benchmark Data (ms/chunk)

| device | official | flashdreams |
| --- | ---: | ---: |
| GB202 | 1308.22 | 6520.72 |
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SANA-WM_streaming FP8 Benchmark Data (ms/chunk)

| device | official | flashdreams |
| --- | ---: | ---: |
| GB202 | 1552.48 | 3993.07 |
15 changes: 15 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,13 @@ invocation, the checkpoint source, and the per-implementation knobs.

Camera-controllable image-to-video world model.

.. grid-item-card:: SANA-WM_streaming
:class-card: fd-feature
:link: models/sana_wm_streaming
:link-type: doc

Chunk-causal camera-controlled world model.

.. grid-item-card:: FlashVSR
:class-card: fd-feature
:link: models/flashvsr
Expand All @@ -261,6 +268,14 @@ invocation, the checkpoint source, and the per-implementation knobs.

Bidirectional Cosmos-Predict2 reference implementations (T2V / I2V, 2B).

.. grid-item-card:: SANA-WM_bidirectional
:class-card: fd-feature
:link: models/sana_wm_bidirectional
:link-type: doc

Bidirectional camera-controlled world model (Stage-1 DiT + LTX-2
refiner, 2.6B).

.. Master toctree: one flat entry per top-level navbar item. Order
here = order in the navbar.

Expand Down
18 changes: 18 additions & 0 deletions docs/source/models/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ Models
flashvsr
hy_worldplay
lingbot_world
sana_wm_streaming
sana_wm_bidirectional
wan21

FlashDreams runs a growing family of world and video models (text-to-video,
Expand Down Expand Up @@ -138,6 +140,14 @@ uses, and the settings you can tune.

Action- and camera-controllable image-to-video world model.

.. grid-item-card:: SANA-WM_streaming
:class-card: fd-feature
:link: /models/sana_wm_streaming
:link-type: doc

Chunk-causal camera-controlled world model with streaming Stage-1,
refiner, and VAE paths.

.. container:: fd-eyebrow

Bidirectional Video Generation
Expand Down Expand Up @@ -172,6 +182,14 @@ uses, and the settings you can tune.

Bidirectional Cosmos-Predict2 reference implementations (T2V / I2V, 2B).

.. grid-item-card:: SANA-WM_bidirectional
:class-card: fd-feature
:link: /models/sana_wm_bidirectional
:link-type: doc

Bidirectional camera-controlled world model (Stage-1 DiT + LTX-2
refiner, 2.6B).

.. container:: fd-eyebrow

Super-resolution
Expand Down
171 changes: 171 additions & 0 deletions docs/source/models/sana_wm_bidirectional.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
.. SPDX-License-Identifier: Apache-2.0
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.

SANA-WM_bidirectional
===================================

.. container:: fd-cta-row

.. button-link:: https://nvlabs.github.io/Sana/
:color: primary

Project page

.. button-link:: https://arxiv.org/abs/2410.10629
:color: primary

arXiv paper

.. button-link:: https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional
:color: primary

Checkpoint

.. button-link:: https://github.com/NVlabs/Sana
:color: primary

Official code

``SANA-WM_bidirectional`` is the full-sequence, camera-controlled
`NVlabs/Sana <https://github.com/NVlabs/Sana>`_ world model release. Given a
first frame, a text prompt, and a camera trajectory, it renders a video clip in
a single bidirectional pass. FlashDreams runs it through the
``sana-wm-bidirectional`` runner with a native Stage-1 DiT and an LTX-2 refiner.

The sibling streaming release has a separate model card:
:doc:`sana_wm_streaming`.

Requirements
------------

- **PyTorch**: >= 2.9.
- **Precision**: BF16 by default. The FlashDreams runner also exposes opt-in
FP8 and FP4 execution paths, but the upstream-vs-FlashDreams benchmark for
``SANA-WM_bidirectional`` is BF16-only because upstream
``SANA-WM_bidirectional`` does not support those precision flags.

Installation
------------

.. code-block:: bash

# from the repo root
uv sync --package flashdreams-sana-wm --extra dev

Running the method
------------------

Launch the ``sana-wm-bidirectional`` runner with a first-frame image, a prompt,
and a camera trajectory:

.. code-block:: bash

uv run flashdreams-run sana-wm-bidirectional \
--image-path ../Sana/asset/sana_wm/demo_0.png \
--prompt-path ../Sana/asset/sana_wm/demo_0.txt \
--camera-path ../Sana/asset/sana_wm/demo_0_pose.npy \
--intrinsics-path ../Sana/asset/sana_wm/demo_0_intrinsics.npy \
--num-frames 161 \
--output-dir outputs/sana_wm_bidirectional_bf16

The demo image, prompt, camera, and intrinsics files ship with the
``SANA-WM_bidirectional`` release; equivalent inputs with matching shapes work
as well.

Optional inputs and knobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- ``--intrinsics-path`` is optional. When omitted, intrinsics are derived from
the first-frame size, assuming a centered principal point and a horizontal
field of view of ``--intrinsics-hfov-deg``.
- ``--camera-path`` can be replaced by an ``--action`` DSL string:

.. code-block:: bash

uv run flashdreams-run sana-wm-bidirectional \
--image-path my_frame.png \
--prompt "a scene description; describe the world's own motion" \
--action "w-100,dw-60,w-101" \
--num-frames 161 \
--output-dir outputs/mine

- ``--no-refiner True`` runs Stage-1 only, for diagnostics.

To inspect all supported CLI arguments and their default values, run:

.. code-block:: bash

uv run flashdreams-run sana-wm-bidirectional --help

What to expect
--------------

- **Model checkpoint**: pulled from
``huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional`` on first run.
- **First launch**: a few minutes for download and warmup; subsequent launches
reuse local caches.
- **Outputs**: ``outputs/<output-dir>/sana-wm-bidirectional.mp4``.

Profiling benchmark
-------------------

The BF16 chart below compares steady-state in-process generation latency per
generated clip for FlashDreams ``SANA-WM_bidirectional`` and the official
``SANA-WM_bidirectional`` implementation under matched settings. On GB202,
FlashDreams measured 96,353.88 ms per clip versus 101,874.59 ms for the
official implementation.

In this chart, ``Official Impl`` means the pinned NVlabs/Sana upstream
implementation measured by the FlashDreams parity harness under matched
settings. It is not the SANA-WM 80-scene benchmark result published by the
model authors.

.. raw:: html

<figure class="benchmark-figure-wrap">
<div
id="sana-wm-bidirectional-bf16-benchmark-chart"
class="benchmark-figure"
data-benchmark-md-url="../_static/performance/sana_wm_bidirectional/perf-bf16-gb202.md"
data-benchmark-series="official:Official Impl:#3b82f6;flashdreams:FlashDreams:#76B900"
data-chart-aria-label="SANA-WM bidirectional BF16 benchmark chart"
></div>
<figcaption>
<p class="model-footnote">
This chart shows steady-state in-process generation latency per generated clip in milliseconds for a
121-frame full-pipeline BF16 run (Stage-1 DiT + LTX-2 refiner + SANA VAE decode).
The measured row used one NVIDIA GB202 GPU, one live warmup generation, and three measured generations.
Model construction, checkpoint loading, video writing, and frame dumps are outside the timing boundary.
The upstream checkout was pinned to commit 6298508.
</p>
</figcaption>
</figure>
<script src="../_static/js/benchmark_chart.js"></script>

Citation
--------

If you use SANA-WM, please cite the original SANA work:

.. code-block:: bibtex

@misc{xie2024sana,
title={SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers},
author={Enze Xie and Junsong Chen and Junyu Chen and Han Cai and Haotian Tang and Yujun Lin and Zhekai Zhang and Muyang Li and Ligeng Zhu and Yao Lu and Song Han},
year={2024},
eprint={2410.10629},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Loading