Skip to content

[New feature] A Noise Injection Method for Flux #10071

Open
@SahilCarterr

Description

@SahilCarterr
Contributor

New Feature for FluxPipeline

Paper: Link
Code : Link
workflow : Link
This paper introduces Sample-specific Score-aware Noise Injection (SSNI) to improve diffusion-based purification (DBP) methods. Unlike existing approaches that use a fixed noise level (t*) for all samples, SSNI adapts t* based on how noisy or clean each sample is. Using a pre-trained score network, SSNI estimates a sample's deviation from the clean data and adjusts the noise level accordingly.

This have stunning Results with Flux

spider_with_noise_injection
spider_sh

@sayakpaul @yiyixuxu

Activity

sayakpaul

sayakpaul commented on Dec 2, 2024

@sayakpaul
Member

@SahilCarterr consider avoiding human subjects :-)

Also, feel free to start a community pipeline.

SahilCarterr

SahilCarterr commented on Dec 2, 2024

@SahilCarterr
ContributorAuthor

I Found this in a reddit Post.I think this should be included in the main FluxPipeline.

John6666cat

John6666cat commented on Dec 2, 2024

@John6666cat

I'm joining this issue too.
#10080

John6666cat

John6666cat commented on Dec 2, 2024

@John6666cat

The code of FluxPipeline is based on StableDiffusionXLPipeline (or its original?), so I think it is extremely unlikely that bugs will occur, for better or for worse, even if we add sigmas option.

Rather, I think the branching is currently buggy. This is because FluxPipeline forcibly assigns a default value to sigmas, which should be None by default in StableDiffusionXLPipeline and other pipelines.😅

if timesteps is not None and sigmas is not None:

    if timesteps is not None and sigmas is not None:
        raise ValueError("Only one of `timesteps` or `sigmas` can be passed. Please choose one to set custom values")

Fix in my test commit

ad3344e

sigmas: List[float] = None, # option

#sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps)
sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps) if sigmas is None and timesteps is None else sigmas
added
consider-for-modular-diffusersThings to consider adding support for in Modular Diffusers (with the help of community)
on Dec 2, 2024
github-actions

github-actions commented on Jan 1, 2025

@github-actions
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

removed
staleIssues that haven't received updates
on Jan 2, 2025
github-actions

github-actions commented on Jan 27, 2025

@github-actions
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

vladmandic

vladmandic commented on Jan 27, 2025

@vladmandic
Contributor

ping to remove stale...

github-actions

github-actions commented on Feb 21, 2025

@github-actions
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

vladmandic

vladmandic commented on Feb 21, 2025

@vladmandic
Contributor

ping to remove stale...

added and removed
staleIssues that haven't received updates
on Feb 21, 2025
John6666cat

John6666cat commented on Feb 26, 2025

@John6666cat

This function has been implemented. We can use it as follows, so I think it's okay to close the issue.

import numpy as np
def calc_sigmas(num_inference_steps: int, sigmas_factor: float):
    sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps)
    sigmas = sigmas * sigmas_factor
    return sigmas

sigmas_factor = 0.95
sigmas = calc_sigmas(steps, sigmas_factor)
pipe(... , sigmas=sigmas) # FluxPipeline instance
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    consider-for-modular-diffusersThings to consider adding support for in Modular Diffusers (with the help of community)wip

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @yiyixuxu@sayakpaul@vladmandic@a-r-r-o-w@hlky

        Issue actions

          [New feature] A Noise Injection Method for Flux · Issue #10071 · huggingface/diffusers