[VLM] Optimize video frame preprocessing for LLaVA-NeXT-Video-7B on GPU

[andrew-k-park]

Description

Optimize video frame preprocessing for LLaVA-NeXT-Video-7B model on GPU by creating an OpenVINO preprocessing model to move preprocessing operations from CPU to GPU

Ticket: CVS-177558

Average 1st token latency (1280x720 5s video (32 frames) + 100 input tokens -> generate 128 tokens)

CPP preprocessing (GPU)  2906.118 ms
OV preprocessing (GPU)   845.6711 ms
CPP preprocessing (CPU)  15321.59 ms
OV preprocessing (CPU)   14327.6  ms

WWB results with video input:

CPP preprocessing (GPU)  0.880806
OV preprocessing (GPU)   0.860603
CPP preprocessing (CPU)  0.918247
OV preprocessing (CPU)   0.906167

Checklist:

• Tests have been updated or added to cover the new code.
• This patch fully addresses the ticket.
• I have made corresponding changes to the documentation.

[Copilot]

Pull request overview

This PR optimizes video frame preprocessing for the LLaVA-NeXT-Video-7B model by implementing GPU-accelerated preprocessing using OpenVINO operations instead of CPU-based preprocessing. The change provides significant performance improvements, reducing first token latency from ~15s (CPU) to ~845ms (GPU with OV preprocessing).

Key changes:

• Added OpenVINO-based preprocessing model that performs resize, crop, and normalization on GPU
• Implemented environment variable control to switch between CPU and GPU preprocessing
• Refactored preprocessing logic to support both CPU (preprocess_frames_cpp) and GPU (preprocess_frames_ov) paths

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
src/cpp/src/visual_language/llava_next_video/classes.hpp	Added new methods for CPU and GPU preprocessing, added preprocessing model infrastructure and use flag
src/cpp/src/visual_language/llava_next_video/classes.cpp	Implemented OpenVINO preprocessing model creation and GPU-accelerated frame preprocessing logic

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The explicit casts to float in the division operations are unnecessary. Both config.crop_size_height and config.crop_size_width will be integer-divided by m_patch_size (which is size_t), and the result is multiplied together. The float casts don't change the outcome since intermediate results are still integers. Consider removing the casts or applying them to the entire expression if floating-point division is intended.

Suggested change

	size_t num_video_tokens = ((float)config.crop_size_height / m_patch_size) *
	((float)config.crop_size_width / m_patch_size);
	num_video_tokens = num_video_tokens / 4 * num_frames;
	size_t num_video_tokens = (static_cast<float>(config.crop_size_height) / static_cast<float>(m_patch_size)) *
	(static_cast<float>(config.crop_size_width) / static_cast<float>(m_patch_size));
	num_video_tokens = static_cast<size_t>(num_video_tokens / 4 * num_frames);

[Copilot]

[nitpick] The double negation logic is harder to read. Consider inverting the condition for clarity: return !env || std::string(env) != \"CPP\"; or better yet, return env == nullptr || std::string(env) != \"CPP\";

Suggested change

	return !(env && std::string(env) == "CPP");
	return env == nullptr || std::string(env) != "CPP";

[Copilot]

Corrected spelling of 'concatinated_frames' to 'concatenated_frames'.

[Copilot]

Corrected spelling of 'concatinated_frames' to 'concatenated_frames'.

[Copilot]

Corrected spelling of 'prepprocessed_frames' to 'preprocessed_frames'.