[Installation]: FlashInfer Dependency issue due to pre-release apache-tvm-ffi

[robertgshaw2-redhat]

Your current environment

On Top of Tree:

(vllm) [robertgshaw2-redhat@nma-h200-isolated-0-preserve vllm]$ VLLM_USE_PRECOMPILED=1 uv pip install -e .
    Updated https://github.com/triton-lang/triton.git (c3c476f357f1e9768ea4e45aa5c17528449ab9ef)
  × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of apache-tvm-ffi==0.1.0b15 and flashinfer-python==0.4.1 depends on apache-tvm-ffi==0.1.0b15, we can conclude that flashinfer-python==0.4.1 cannot be used.
      And because vllm==0.11.1rc3.dev57+g6454afec9.precompiled depends on flashinfer-python==0.4.1, we can conclude that vllm==0.11.1rc3.dev57+g6454afec9.precompiled cannot be used.
      And because only vllm==0.11.1rc3.dev57+g6454afec9.precompiled is available and you require vllm, we can conclude that your requirements are unsatisfiable.

      hint: `apache-tvm-ffi` was requested with a pre-release marker (e.g., apache-tvm-ffi==0.1.0b15), but pre-releases weren't enabled (try: `--prerelease=allow`)

How you are installing vllm

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[mgoin]

Tracking issue in flashinfer-ai/flashinfer#1962

We were waiting for the stable release of tvm-ffi: flashinfer-ai/flashinfer#1939 (comment)
Given that v0.1.0 was released yesterday we will merge flashinfer-ai/flashinfer#1960 soon.

[sniper35]

+1 here,
When install vllm using existing pytorch and encountered this issue, waiting for upstream flashinfer to resolve the issue asap^^

[huachenheli]

+1

[noooop]

Without deleting flashinfer-python==0.4.1, in requirements/cuda.txt, the latest vllm cannot be installed properly.

VLLM_USE_PRECOMPILED=1 uv pip install -v --editable .

  × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of apache-tvm-ffi==0.1.0b15 and flashinfer-python==0.4.1 depends on apache-tvm-ffi==0.1.0b15, we can conclude that flashinfer-python==0.4.1 cannot be
      used.
      And because vllm==0.1.dev10781+gfb5fdfad0.d20251028.precompiled depends on flashinfer-python==0.4.1, we can conclude that vllm==0.1.dev10781+gfb5fdfad0.d20251028.precompiled cannot
      be used.
      And because only vllm==0.1.dev10781+gfb5fdfad0.d20251028.precompiled is available and you require vllm, we can conclude that your requirements are unsatisfiable.

      hint: `apache-tvm-ffi` was requested with a pre-release marker (e.g., apache-tvm-ffi==0.1.0b15), but pre-releases weren't enabled (try: `--prerelease=allow`)

cc @hmellor

[jmkuebler]

As a temporary workaround to unblock development you mind want to add --prerelease=allow

For me VLLM_USE_PRECOMPILED=1 uv pip install --editable . --prerelease=allow worked and serves its purpose so far.

(Disclaimer: There might be reasons to not do this that I am not aware of)

[noooop]

#26859

We have already disabled FlashInfer sampler by default, I think it's best not to install it by default. At least in the upcoming 0.11.1 release.

[hmellor]

Yes currently to install locally you must manually install that beta version first. This is what we do in the Dockerfile.

I'm keeping an eye on Flashinfer releases so we can remove the workaround ASAP.

[noooop]

Anyhow, users should not need to use any workarounds to install smoothly when the upcoming 0.11.1 release.

[hmellor]

I agree.

I've left a comment on the Flashinfer issue indicating that we are considering removing it. If that doesn't encourage them to create a new release ASAP, we'll have to make it an optional dependency.