VOID on L40S GPU is bottlenecked by loading the 5.5B CogVideoX transformer, which takes about 40-50 seconds before inference begins.

[ganeshmohane-aisolo]

[ErenAta16]

I ran a clean cold single-sequence benchmark on Colab A100-SXM4-40GB to check whether the memory mode is responsible for the startup delay.

Test controls:

• scenario: cold_single_seq
• sequence: my_video
• repeats: 3 per mode
• num_inference_steps: 30
• sample_size: 384x672
• max_video_length: 197
• temporal_window_size: 85
• sequence directory cleaned before each run
• skip_if_exists=False

Median results:

mode	total wall time	process to first sequence	init before transformer log	after first sequence log	peak VRAM
model_cpu_offload_and_qfloat8	153.5s	75.1s	65.2s	78.5s	18.9 GiB
model_full_load	134.4s	74.6s	59.0s	59.6s	38.0 GiB
model_cpu_offload	147.9s	68.9s	59.0s	79.0s	18.9 GiB

Takeaway: model_full_load is useful on A100 for total runtime. It reduced the median end-to-end run from 153.5s to 134.4s, mostly by making the work after the sequence starts faster.

It did not materially fix the cold-start delay before inference begins. The first sequence log still appears around 75s with model_full_load, and the largest wait remains before/around transformer initialization.

So I would not treat model_full_load as the fix for this issue. It is a good high-VRAM runtime option, but the startup bottleneck needs finer profiling inside load_pipeline(), especially around base CogVideoX transformer loading, VOID checkpoint loading, VAE/T5 loading, and memory-mode setup.

The benchmark note is now recorded on issue-1-l40s-load-bottleneck in benchmark/results/2026-06-21-colab-a100-cold-single-seq.md.

This is A100 data only. A direct L40S run is still needed before making L40S-specific claims.