Wan-Animate-2 (authored by @kelseyee)

.ai/testing.md

@@ -36,7 +36,7 @@ Follow the style introduced in [#14113](https://github.com/huggingface/diffusers
 
 - Location: `tests/modular_pipelines/<model>/test_modular_pipeline_<model>.py` (one test class per blockset / pipeline variant).
 - Subclass `ModularPipelineTesterMixin` (from `..test_modular_pipelines_common`) — it runs the pipeline end-to-end (call signature, batch consistency, float16, device placement) against a tiny checkpoint.
-- Set `pipeline_class`, `pipeline_blocks_class`, `pretrained_model_name_or_path`, `params` / `batch_params`, and implement `get_dummy_inputs(seed=0)`. Set `expected_workflow_blocks` to pin the block name → class ordering per workflow.
+- Set `pipeline_class`, `pipeline_blocks_class`, `pretrained_model_name_or_path`, `params` / `batch_params`, and implement `get_dummy_inputs(seed=0)`. Set `expected_workflow_blocks` to pin the block name → class ordering per workflow (only for blocksets with a `_workflow_map` — with a single workflow the list would just restate the class definition), and `expected_workflow_defaults` to pin each workflow's components, pipeline configs, and inputs — required ones by name, optional ones with their defaults. A pipeline without workflows pins its full blockset under the `None` key. An optional `component_configs` entry pins config values of `from_config` components against their creating spec (e.g. the guider scale that tells a base and a distilled preset apart); pretrained components take their config from the repo, so there is nothing block-level to pin.
 - `pretrained_model_name_or_path` is a tiny repo with real components (tiny transformer, real scheduler / VAE / tokenizer configs). Develop against a personal repo; tiny repos ultimately live under `hf-internal-testing/` — not merge-blocking, a maintainer moves it before or after merge.
 - **The tiny repo must mirror the real checkpoint's shape** — same index file type, same pipeline-level config keys, a scheduler configured like the real one. A fixture that doesn't look like the published repos tests a loading/config path no user will ever hit, while the path users *do* hit stays uncovered. If the model ships variants with different configs (base/distilled, different schedules), make one tiny repo and test class per variant — see the flux2 klein base/distilled split.
 - **Bespoke tests go on the tester class as methods**, not as module-level functions — the mixin is pytest-style, so fixtures (`tmp_path`, `pytest.raises`, parametrize) all work in methods.

docs/source/en/_toctree.yml

@@ -405,6 +405,8 @@
         title: Transformer2DModel
       - local: api/models/transformer_temporal
         title: TransformerTemporalModel
+      - local: api/models/wan_animate_2_transformer_3d
+        title: WanAnimate2Transformer3DModel
       - local: api/models/wan_animate_transformer_3d
         title: WanAnimateTransformer3DModel
       - local: api/models/wan_transformer_3d
@@ -711,6 +713,8 @@
         title: Stable Video Diffusion
       - local: api/pipelines/wan
         title: Wan
+      - local: api/pipelines/wan_animate_2
+        title: Wan-Animate-2
       title: Video
     title: Pipelines
   - sections:

docs/source/en/api/models/wan_animate_2_transformer_3d.md

@@ -0,0 +1,30 @@
+<!-- Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License. -->
+
+# WanAnimate2Transformer3DModel
+
+A Diffusion Transformer model for 3D video-like data used in [Wan-Animate-2](https://github.com/Wan-Video/Wan2.2) by the Alibaba Wan Team. It animates a character image with the motion of a driving video through an in-context reference mechanism: each segment first runs a reference pass (`kv_cache_mode="extract"`) that caches every layer's reference K/V, then the denoising passes (`kv_cache_mode="cached"`) attend jointly over the generation tokens and the cached reference tokens through a flex `BlockMask`.
+
+The model can be loaded with the following code snippet.
+
+```python
+from diffusers import WanAnimate2Transformer3DModel
+
+transformer = WanAnimate2Transformer3DModel.from_pretrained("Wan-AI/Wan2.2-Animate-2-14B-Diffusers", subfolder="transformer", dtype=torch.bfloat16)
+```
+
+## WanAnimate2Transformer3DModel
+
+[[autodoc]] WanAnimate2Transformer3DModel
+
+## Transformer2DModelOutput
+
+[[autodoc]] models.modeling_outputs.Transformer2DModelOutput

docs/source/en/api/pipelines/wan_animate_2.md

@@ -0,0 +1,77 @@
+<!-- Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License. -->
+
+# Wan-Animate-2
+
+[Wan-Animate-2](https://github.com/Wan-Video/Wan2.2) by the Alibaba Wan Team animates a reference character image with the motion of a driving video. The driving video is processed in fixed-length segments: each segment runs a reference-extraction pass that caches the driving segment's K/V in every transformer layer, denoises against that cache, and is decoded inside the loop because the next segment conditions on the previous segment's decoded tail frames.
+
+Two presets are available: the base checkpoint samples with classifier-free guidance, and the distilled checkpoint samples in few steps without it (its guider is pinned to `guidance_scale=1.0`).
+
+```python
+import torch
+from diffusers import ModularPipeline
+from diffusers.utils import export_to_video, load_image, load_video
+
+pipe = ModularPipeline.from_pretrained("Wan-AI/Wan2.2-Animate-2-14B-Diffusers")
+pipe.load_components(dtype=torch.bfloat16)
+
+# The transformer weights and the per-segment reference KV cache do not co-reside on one 80 GB
+# card at the default resolution, so stream the transformer's blocks. Compiling the blocks is
+# required as the in-context attention runs on the flex backend
+pipe.transformer.enable_group_offload(
+    onload_device=torch.device("cuda"),
+    offload_device=torch.device("cpu"),
+    offload_type="block_level",
+    use_stream=True,
+)
+pipe.text_encoder.to("cuda")
+pipe.image_encoder.to("cuda")
+pipe.vae.to("cuda")
+pipe.transformer.compile_repeated_blocks(fullgraph=False)
+
+# The first demo from the official repository: https://github.com/Wan-Video/Wan-Animate-2
+demo = "https://raw.githubusercontent.com/Wan-Video/Wan-Animate-2/main/examples/demo1"
+image = load_image(f"{demo}/reference.png")
+driving_video, driving_video_fps = load_video(f"{demo}/template.mp4", return_fps=True)
+prompt = "人物外观描述：一只银灰色虎斑纹的小猫，拥有圆润的脸庞、竖立的耳朵和巨大的圆形眼睛。它身穿一套深蓝色的制服套装，包括一件带有金色纽扣的西装外套和一条百褶裙。外套里面搭配着白色衬衫，领口处系着一个红色的蝴蝶结，袖口露出白色的衬衫边缘。背景描述：背景为纯白色，光线均匀明亮，无其他杂物或装饰。"
+
+videos = pipe(
+    image=image,
+    driving_video=driving_video,
+    driving_video_fps=driving_video_fps,
+    prompt=prompt,
+    output="videos",
+)
+export_to_video(videos[0], "output.mp4", fps=24)
+```
+
+For the distilled checkpoint, load `Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers` the same way — nothing else changes. Each preset carries its own sampling defaults (40 steps for the base checkpoint, 10 for the distilled one), and no `guidance_scale` argument exists anywhere: guidance is owned by the pipeline's guider component (classifier-free guidance at 3.0 for the base preset, disabled for the distilled one).
+
+`height` and `width` (defaults 800 and 640) set the target *area* of the generated video; the actual frame size keeps the reference image's aspect ratio, and the driving frames are letterboxed to it. Inputs that already sit at the target letterbox size pass through the preprocessing untouched, so preprocessing can also be done entirely outside the pipeline.
+
+## WanAnimate2ModularPipeline
+
+[[autodoc]] WanAnimate2ModularPipeline
+
+## WanAnimate2DistilledModularPipeline
+
+[[autodoc]] WanAnimate2DistilledModularPipeline
+
+## WanAnimate2Blocks
+
+[[autodoc]] WanAnimate2Blocks
+
+## WanAnimate2DistilledBlocks
+
+[[autodoc]] WanAnimate2DistilledBlocks

src/diffusers/__init__.py

@@ -352,6 +352,7 @@
             "UNetSpatioTemporalConditionModel",
             "UVit2DModel",
             "VQModel",
+            "WanAnimate2Transformer3DModel",
             "WanAnimateTransformer3DModel",
             "WanTransformer3DModel",
             "WanVACETransformer3DModel",
@@ -558,6 +559,10 @@
             "Wan22Image2VideoBlocks",
             "Wan22Image2VideoModularPipeline",
             "Wan22ModularPipeline",
+            "WanAnimate2Blocks",
+            "WanAnimate2DistilledBlocks",
+            "WanAnimate2DistilledModularPipeline",
+            "WanAnimate2ModularPipeline",
             "WanBlocks",
             "WanImage2VideoAutoBlocks",
             "WanImage2VideoModularPipeline",
@@ -1204,6 +1209,7 @@
             UNetSpatioTemporalConditionModel,
             UVit2DModel,
             VQModel,
+            WanAnimate2Transformer3DModel,
             WanAnimateTransformer3DModel,
             WanTransformer3DModel,
             WanVACETransformer3DModel,
@@ -1390,6 +1396,10 @@
             Wan22Image2VideoBlocks,
             Wan22Image2VideoModularPipeline,
             Wan22ModularPipeline,
+            WanAnimate2Blocks,
+            WanAnimate2DistilledBlocks,
+            WanAnimate2DistilledModularPipeline,
+            WanAnimate2ModularPipeline,
             WanBlocks,
             WanImage2VideoAutoBlocks,
             WanImage2VideoModularPipeline,

src/diffusers/loaders/single_file_model.py

@@ -54,6 +54,7 @@
     convert_sana_transformer_to_diffusers,
     convert_sd3_transformer_checkpoint_to_diffusers,
     convert_stable_cascade_unet_single_file_to_diffusers,
+    convert_wan_animate_2_transformer_to_diffusers,
     convert_wan_transformer_to_diffusers,
     convert_wan_vae_to_diffusers,
     convert_z_image_controlnet_checkpoint_to_diffusers,
@@ -172,6 +173,10 @@
         "checkpoint_mapping_fn": convert_wan_transformer_to_diffusers,
         "default_subfolder": "transformer",
     },
+    "WanAnimate2Transformer3DModel": {
+        "checkpoint_mapping_fn": convert_wan_animate_2_transformer_to_diffusers,
+        "default_subfolder": "transformer",
+    },
     "AutoencoderKLWan": {
         "checkpoint_mapping_fn": convert_wan_vae_to_diffusers,
         "default_subfolder": "vae",

src/diffusers/loaders/single_file_utils.py

@@ -3289,6 +3289,38 @@ def reshape_bias_handler(key, state_dict):
     return converted_state_dict
 
 
+def convert_wan_animate_2_transformer_to_diffusers(checkpoint, **kwargs):
+    r"""
+    Converts the state dict of the Wan-Animate-2 transformer from the official checkpoint format to the diffusers
+    format.
+    """
+    attention_renames = {
+        ".q.": ".to_q.",
+        ".k.": ".to_k.",
+        ".v.": ".to_v.",
+        ".o.": ".to_out.0.",
+        ".k_img.": ".add_k_proj.",
+        ".v_img.": ".add_v_proj.",
+        ".norm_k_img.": ".norm_added_k.",
+    }
+
+    converted_state_dict = {}
+    for key in list(checkpoint.keys()):
+        new_key = key.replace("model.diffusion_model.", "")
+        # The official checkpoint wraps every transformer block in an in-context module the
+        # diffusers layout does not have: `blocks.N.block.X` -> `blocks.N.X`.
+        if new_key.startswith("blocks."):
+            new_key = new_key.replace(".block.", ".", 1)
+        if ".self_attn." in new_key or ".cross_attn." in new_key:
+            for old, new in attention_renames.items():
+                if old in new_key:
+                    new_key = new_key.replace(old, new)
+                    break
+        converted_state_dict[new_key] = checkpoint.pop(key)
+
+    return converted_state_dict
+
+
 def convert_wan_vae_to_diffusers(checkpoint, **kwargs):
     converted_state_dict = {}
 

src/diffusers/models/__init__.py

@@ -145,6 +145,7 @@
     _import_structure["transformers.transformer_temporal"] = ["TransformerTemporalModel"]
     _import_structure["transformers.transformer_wan"] = ["WanTransformer3DModel"]
     _import_structure["transformers.transformer_wan_animate"] = ["WanAnimateTransformer3DModel"]
+    _import_structure["transformers.transformer_wan_animate_2"] = ["WanAnimate2Transformer3DModel"]
     _import_structure["transformers.transformer_wan_vace"] = ["WanVACETransformer3DModel"]
     _import_structure["transformers.transformer_z_image"] = ["ZImageTransformer2DModel"]
     _import_structure["unets.unet_1d"] = ["UNet1DModel"]
@@ -285,6 +286,7 @@
             T5FilmDecoder,
             Transformer2DModel,
             TransformerTemporalModel,
+            WanAnimate2Transformer3DModel,
             WanAnimateTransformer3DModel,
             WanTransformer3DModel,
             WanVACETransformer3DModel,

src/diffusers/models/transformers/__init__.py

@@ -64,5 +64,6 @@
     from .transformer_temporal import TransformerTemporalModel
     from .transformer_wan import WanTransformer3DModel
     from .transformer_wan_animate import WanAnimateTransformer3DModel
+    from .transformer_wan_animate_2 import WanAnimate2Transformer3DModel
     from .transformer_wan_vace import WanVACETransformer3DModel
     from .transformer_z_image import ZImageTransformer2DModel