Skip to content

Generated videos are pure noise without parameter changes; possible model class mismatch? #5

@Anastasiawd

Description

@Anastasiawd

Hi, thank you for releasing this project.

I am encountering an issue where all generated videos are pure noise, even though I did not modify any parameters in the provided scripts. The problem occurs consistently across runs.

During inference, I also see the following warnings/errors:
`Expected types for vae:
(<class 'diffusers.models.autoencoders.autoencoder_kl_cogvideox.AutoencoderKLCogVideoX'>,),
got <class 'robomaster.models.autoencoder_magvit.AutoencoderKLCogVideoX'>.

Expected types for transformer:
(<class 'diffusers.models.transformers.cogvideox_transformer_3d.CogVideoXTransformer3DModel'>,),
got <class 'robomaster.models.transformer3d.CogVideoXTransformer3DModel'>.`

I am not sure whether this type mismatch between diffusers and robomaster implementations is the root cause of the noisy outputs, or if the warnings can be safely ignored. Below is one of the output video:

close.mp4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions