Skip to content

Add Echo modular pipeline - #14696

Open
Gelercatty wants to merge 1 commit into
huggingface:mainfrom
Echo-Team-Joy-Future-Academy-JD:echo
Open

Add Echo modular pipeline#14696
Gelercatty wants to merge 1 commit into
huggingface:mainfrom
Echo-Team-Joy-Future-Academy-JD:echo

Conversation

@Gelercatty

@Gelercatty Gelercatty commented Sep 3, 2026

Copy link
Copy Markdown

What does this PR do?

Adds official Diffusers Modular Pipeline support for Echo, a long-video model that generates synchronized video and audio with cross-shot memory.

This supersedes #13910 with a modular implementation that incorporates the earlier review feedback:

  • adds the standalone diffusers.modular_pipelines.echo package and the public EchoModularPipeline / EchoBlocks APIs
  • reuses existing lower-level Diffusers components without exposing LTX in Echo's public API
  • keeps checkpoint conversion in scripts/ instead of adding an original-checkpoint runtime pipeline
  • supports an optional first frame and up to seven paired image/audio memory slots
  • supports variable-length memory audio and crops audio longer than 9.62 seconds to its highest-response window
  • implements Echo's positive-only conditioning and stochastic 8-step DMD denoising workflow
  • adds mixed-precision-safe audio decoding, documentation, and tests using a public tiny fixture

Official resources:

Closes #13909
Supersedes #13910

Testing

  • make style
  • make fix-copies
  • make quality
  • PYTHONPATH=src python -m pytest -q tests/modular_pipelines/echo/test_modular_pipeline_echo.py
    • 25 passed, 12 skipped

AI-assisted self-review

Codex was used to help review, refactor, test, and prepare this contribution. The final diff was reviewed using the repository's model-integration and self-review guidance.

The final self-review found no blocking correctness issues. Echo has its own public package and naming throughout; existing lower-level Diffusers component types are reused only as implementation dependencies. Memory-slot count, short-audio handling, variable audio lengths, response-aware 9.62-second cropping, deterministic DMD re-noising, mixed-precision decoding, loading, save/load, workflow, and memory behavior are covered by the test suite.

Validation scope: the final pass used the public Echo-Team/tiny-echo-modular-pipe fixture on CPU. A full production-checkpoint conversion and end-to-end GPU inference run was not repeated in this final pass.

Before submitting

Who can review?

@asomoza @yiyixuxu

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests modular-pipelines utils size/L PR with diff > 200 LOC labels Sep 3, 2026
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation modular-pipelines size/L PR with diff > 200 LOC tests utils

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

[New Pipeline/Model] Add JoyAI-Echo multi-shot audio-video generation pipeline

3 participants