LLM Benchmark crashes on llava:13b when used on Nvidia GPU

[synchronic1]

llm_benchmark has completed successful runs on a CPU only machine. But once I've tried to run the program in a dual Nvidia 3060 gpu configuration, it crashes in llava:13b. I've tried both in Python 3.10 and 3.12 with the same result. I've also tried to run it in CPU only mode with the GPU's flagged off in environment variables, where only one GPU takes the load, and where the GPU loads are distributed between two GPUs with the same result. Is there something wrong being done here?

Edit: I need to add that it isn't only the particular model where it crashes. It will crash on various models unpredictably. Of note, it will not work at all on an AVX dual Xeon machine I have been using in testing. The test run has only completed on a CPU only machine with an AMD 3950x.

`model_name = llava:13b
prompt = Describe the image, C:\Users\Asus\AppData\Local\Programs\Python\Python310\lib\site-packages\llm_benchmark\data\img\sample1.jpg
┌─────────────────────────────── Traceback (most recent call last) ────────────────────────────────┐
│ C:\Users\Asus\AppData\Local\Programs\Python\Python310\lib\site-packages\llm_benchmark\main.py:55 │
│ in run │
│ │
│ 52 │ │ bench_results_info.update(result1) │
│ 53 │ │ result2 = run_benchmark.run_benchmark(models_file_path,benchmark_file_path, 'que │
│ 54 │ │ bench_results_info.update(result2) │
│ > 55 │ │ result3 = run_benchmark.run_benchmark(models_file_path,benchmark_file_path, 'vis │
│ 56 │ │ bench_results_info.update(result3) │
│ 57 │ else: │
│ 58 │ │ bench_results_info.update({"llama2:7b":7.65}) │
│ │
│ ┌─────────────────────────────────────────── locals ───────────────────────────────────────────┐ │
│ │ bench_results_info = { │ │
│ │ │ 'mistral:7b': '54.40', │ │
│ │ │ 'llama3.1:8b': '48.97', │ │
│ │ │ 'phi3:3.8b': '79.95', │ │
│ │ │ 'qwen2:7b': '50.46', │ │
│ │ │ 'gemma2:9b': '35.88' │ │
│ │ } │ │
│ │ benchmark_file_path = 'C:\Users\Asus\AppData\Local\Programs\Python\Python310\lib\s… │ │
│ │ ft_mem_size = 15.94 │ │
│ │ is_simulation = False │ │
│ │ models_file_path = 'C:\Users\Asus\AppData\Local\Programs\Python\Python310\lib\s… │ │
│ │ ollama_version = '0.3.6' │ │
│ │ ollamabin = 'ollama' │ │
│ │ result1 = { │ │
│ │ │ 'mistral:7b': '54.40', │ │
│ │ │ 'llama3.1:8b': '48.97', │ │
│ │ │ 'phi3:3.8b': '79.95', │ │
│ │ │ 'qwen2:7b': '50.46' │ │
│ │ } │ │
│ │ result2 = {'gemma2:9b': '35.88'} │ │
│ │ sendinfo = True │ │
│ │ sys_info = { │ │
│ │ │ 'system': 'Windows', │ │
│ │ │ 'memory': 15.940048217773438, │ │
│ │ │ 'cpu': 'Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz', │ │
│ │ │ 'gpu': 'NVIDIA GeForce RTX 3060 NVIDIA GeForce RTX 3060', │ │
│ │ │ 'os_version': 'Microsoft Windows 10 Pro\n', │ │
│ │ │ 'system_name': 'Windows', │ │
│ │ │ 'run_in': 'Unknown' │ │
│ │ } │ │
│ └──────────────────────────────────────────────────────────────────────────────────────────────┘ │

llm benchmark crash.txt
`