Skip to content

Conversation

@yehudit1987
Copy link
Contributor

@yehudit1987 yehudit1987 commented Nov 10, 2025

Disable Gemma embedding model in CI tests since it's a gated model
requiring HF_TOKEN. Tests now use Qwen3-Embedding-0.6B exclusively.

This approach was discussed and approved by maintainers who decided
to focus on Qwen3 (non-gated) for CI tests.
See: https://github.com/vllm-project/semantic-router/issues/573#issuecomment-3607352121

Changes in candle-binding/semantic-router_test.go:

  • Set GemmaEmbeddingModelPath to empty string (disable Gemma)
  • Update dimension expectations from 768/1024 to 1024 (Qwen3)
  • Skip InitGemmaOnly test with clear explanation
  • Remove conditional skip logic that was masking test failures
  • Update comments to clarify Qwen3-only and Matryoshka usage

Changes in tools/make/models.mk:

  • Add Qwen3-Embedding-0.6B to minimal download target for CI
  • Remove Gemma from lora download target
  • Improve download tracking with .downloaded marker files

Qwen3-Embedding-0.6B is fully open (no gating) and supports
Matryoshka dimension truncation (768/512/256/128) from its
native 1024 dimensions.

Resolves #573 (Section 5: Embedding Model Tests)

@netlify
Copy link

netlify bot commented Nov 10, 2025

Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit ae7829d
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/6931c9b32768ee0008659da0
😎 Deploy Preview https://deploy-preview-623--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link

github-actions bot commented Nov 10, 2025

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 candle-binding

Owners: @rootfs
Files changed:

  • candle-binding/semantic-router_test.go
  • candle-binding/src/classifiers/unified.rs
  • candle-binding/src/ffi/embedding.rs

📁 tools

Owners: @yuluo-yx, @rootfs, @Xunzhuo
Files changed:

  • tools/make/models.mk
  • tools/make/rust.mk

vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@yehudit1987 yehudit1987 force-pushed the fix_skipped_tests_5 branch 2 times, most recently from cb26fa5 to 5cc12a3 Compare November 10, 2025 14:56
@github-actions github-actions bot deleted a comment Nov 15, 2025
@yehudit1987 yehudit1987 force-pushed the fix_skipped_tests_5 branch 2 times, most recently from a925516 to ba72fb3 Compare December 4, 2025 11:55
@yehudit1987 yehudit1987 marked this pull request as ready for review December 4, 2025 12:51
@yehudit1987 yehudit1987 marked this pull request as draft December 4, 2025 14:09
@yehudit1987 yehudit1987 marked this pull request as ready for review December 4, 2025 17:37
Signed-off-by: Yehudit Kerido <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Testing] improve and enable the skipped testing cases

3 participants