Skip to content

Support multi images for vlm benchmarking in samples and llm_bench #2197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 34 commits into
base: master
Choose a base branch
from

Conversation

wgzintel
Copy link
Contributor

@wgzintel wgzintel commented May 12, 2025

  1. tools/llm_bench supports multiple images for benchmarking of vlm models.
  2. samples/cpp/visual_language_chat supports multiple images for benchmarking of vlm models.
  3. samples/cpp/text_generation supports to read prompt from file.

@github-actions github-actions bot added category: llm_bench Label for tool/llm_bench folder category: LLM samples GenAI LLM samples category: VLM samples GenAI VLM samples labels May 12, 2025
@wgzintel wgzintel requested a review from peterchen-intel May 12, 2025 09:55
@wgzintel wgzintel changed the title Support multi images for vlm test Support multi images for vlm benchmarking in samples and llm_bench May 13, 2025
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately I forgot that we also have python equivalent: benchmark_genai.py. Please update it and its readme as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, pls review again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Wovchena Do you have any suggestions for this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: cmake / build Cmake scripts category: llm_bench Label for tool/llm_bench folder category: LLM samples GenAI LLM samples category: VLM samples GenAI VLM samples
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants