4.57.3 and 5.0.0 from main: MistralTokenizer object has no attribute 'convert_tokens_to_ids'

[dgouju]

System Info

• transformers version: 5.0.0.dev0
• Platform: Linux-6.14.0-1021-gcp-x86_64-with-glibc2.39
• Python version: 3.12.3
• Huggingface_hub version: 1.2.3
• Safetensors version: 0.7.0
• Accelerate version: 1.12.0
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (accelerator?): 2.9.0+cu130 (CUDA)
• Using distributed or parallel set-up in script?: No, TP=1
• Using GPU in script?: Yes
• GPU type: NVIDIA RTX PRO 6000 Blackwell Server Edition

Who can help?

@zucchini-nlp @ArthurZucker @itazap
When running VLLM with above Mistral Model, including this VLLM PR for 5.0.0, I have the following error:

(EngineCore_DP0 pid=13140)   File "/home/user/vllm-venv/lib/python3.12/site-packages/transformers/models/pixtral/processing_pixtral.py", line 124, in __init__
(EngineCore_DP0 pid=13140)     self.image_token_id = tokenizer.convert_tokens_to_ids(self.image_token)
(EngineCore_DP0 pid=13140)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=13140) AttributeError: 'MistralTokenizer' object has no attribute 'convert_tokens_to_ids'. Did you mean: 'convert_tokens_to_string'?

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Use case:

1. Install:

uv pip install setuptools packaging
uv pip install https://github.com/vllm-project/vllm/releases/download/v0.13.0/vllm-0.13.0+cu130-cp38-abi3-manylinux_2_35_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu130
uv pip install datasets psutil pandas torch-c-dlpack-ext

2. Run: VLLM_USE_FLASHINFER_MOE_FP4=1 ENABLE_NVFP4_SM120=1 VLLM_ATTENTION_BACKEND=FLASHINFER vllm serve RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4 --tensor-parallel-size=1 --no-enable-prefix-caching --kv-cache-dtype=fp8 --seed=42 --disable-log-requests --download_dir /home/user/models --max-model-len=8192 --max-num-batched-tokens=2048 --max-num-seqs=256 --gpu-memory-utilization=0.97

Expected behavior

Having VLLM properly running with this model.

[adi776borate]

The issue seems to be on vLLM's side. Their MistralTokenizer wrapper holds the real HF backend but it's missing the convert_tokens_to_ids method. Since the PixtralProcessor in v5 relies on that method being available, it crashes when it hits the wrapper.
Creating the proxy call to the backend should fix the error.

Does this sound good?

[dgouju]

@adi776borate I followed up with VLLM team, they pushed a fix on their end (PR), but I still have the following error:

VLLM_USE_FLASHINFER_MOE_FP4=1 ENABLE_NVFP4_SM120=1 VLLM_ATTENTION_BACKEND=FLASHINFER vllm serve RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4 --tensor-parallel-size=1 --no-enable-prefix-caching --kv-cache-dtype=fp8 --seed=42 --disable-log-requests --download_dir /home/user/models/ --max-model-len=8192 --max-num-batched-tokens=2048 --max-num-seqs=256 --gpu-memory-utilization=0.97
WARNING 01-06 12:02:08 [attention.py:82] Using VLLM_ATTENTION_BACKEND environment variable is deprecated and will be removed in v0.14.0 or v1.0.0, whichever is soonest. Please use --attention-config.backend command line argument or AttentionConfig(backend=...) config field instead.
(APIServer pid=557989) INFO 01-06 12:02:08 [api_server.py:1278] vLLM API server version 0.1.dev12681+g0e212bce8.d20260106
(APIServer pid=557989) INFO 01-06 12:02:08 [utils.py:253] non-default args: {'model_tag': 'RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', 'model': 'RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', 'seed': 42, 'max_model_len': 8192, 'download_dir': '/home/user/models/', 'gpu_memory_utilization': 0.97, 'kv_cache_dtype': 'fp8', 'enable_prefix_caching': False, 'max_num_batched_tokens': 2048, 'max_num_seqs': 256}
(APIServer pid=557989) INFO 01-06 12:02:10 [model.py:522] Resolved architecture: Mistral3ForConditionalGeneration
(APIServer pid=557989) INFO 01-06 12:02:10 [model.py:1508] Using max model len 8192
(APIServer pid=557989) WARNING 01-06 12:02:10 [vllm.py:1447] Current vLLM config is not set.
(APIServer pid=557989) INFO 01-06 12:02:10 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=557989) INFO 01-06 12:02:10 [vllm.py:635] Disabling NCCL for DP synchronization when using async scheduling.
(APIServer pid=557989) INFO 01-06 12:02:10 [vllm.py:640] Asynchronous scheduling is enabled.
(APIServer pid=557989) INFO 01-06 12:02:10 [cache.py:205] Using fp8 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor.
(APIServer pid=557989) INFO 01-06 12:02:10 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=557989) [2026-01-06 12:02:10] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(APIServer pid=557989) [2026-01-06 12:02:10] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-06 12:02:16 [vllm.py:1447] Current vLLM config is not set.
INFO 01-06 12:02:16 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
WARNING 01-06 12:02:16 [attention.py:82] Using VLLM_ATTENTION_BACKEND environment variable is deprecated and will be removed in v0.14.0 or v1.0.0, whichever is soonest. Please use --attention-config.backend command line argument or AttentionConfig(backend=...) config field instead.
INFO 01-06 12:02:16 [vllm.py:635] Disabling NCCL for DP synchronization when using async scheduling.
INFO 01-06 12:02:16 [vllm.py:640] Asynchronous scheduling is enabled.
(EngineCore_DP0 pid=558096) INFO 01-06 12:02:16 [core.py:96] Initializing a V1 LLM engine (v0.1.dev12681+g0e212bce8.d20260106) with config: model='RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', speculative_config=None, tokenizer='RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=8192, download_dir='/home/user/models/', load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=fp8, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False), seed=42, served_model_name=RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=558096) [2026-01-06 12:02:16] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=558096) [2026-01-06 12:02:16] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=558096) /home/user/source/vllm-src-venv/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.)
(EngineCore_DP0 pid=558096)   _C._set_float32_matmul_precision(precision)
(EngineCore_DP0 pid=558096) INFO 01-06 12:02:17 [parallel_state.py:1214] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.0.106:49403 backend=nccl
(EngineCore_DP0 pid=558096) INFO 01-06 12:02:17 [parallel_state.py:1425] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A
(EngineCore_DP0 pid=558096) [2026-01-06 12:02:20] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=558096) [2026-01-06 12:02:20] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=558096) [2026-01-06 12:02:23] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=558096) [2026-01-06 12:02:23] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888] EngineCore failed to start.
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888] Traceback (most recent call last):
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 879, in run_engine_core
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 644, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     super().__init__(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 105, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/executor/abstract.py", line 101, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     self._init_executor()
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/executor/uniproc_executor.py", line 47, in _init_executor
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     self.driver_worker.init_device()
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/worker_base.py", line 326, in init_device
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     self.worker.init_device()  # type: ignore
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/gpu_worker.py", line 264, in init_device
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     self.model_runner = GPUModelRunnerV1(self.vllm_config, self.device)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/gpu_model_runner.py", line 611, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     MultiModalBudget(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/utils.py", line 48, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     max_tokens_by_modality = mm_registry.get_max_tokens_per_item_by_modality(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/registry.py", line 171, in get_max_tokens_per_item_by_modality
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     return profiler.get_mm_max_tokens(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 350, in get_mm_max_tokens
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 263, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     processor_inputs = factory.get_dummy_processor_inputs(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 120, in get_dummy_processor_inputs
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     dummy_text = self.get_dummy_text(mm_counts)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/model_executor/models/mistral3.py", line 228, in get_dummy_text
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     processor = self.info.get_hf_processor()
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/model_executor/models/mistral3.py", line 257, in get_hf_processor
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     return self.ctx.get_hf_processor(PixtralProcessor, **kwargs)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1189, in get_hf_processor
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     return cached_processor_from_config(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/transformers_utils/processor.py", line 251, in cached_processor_from_config
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     return cached_get_processor_without_dynamic_kwargs(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/transformers_utils/processor.py", line 224, in cached_get_processor_without_dynamic_kwargs
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     final_processor = cached_get_processor(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                       ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm/vllm/transformers_utils/processor.py", line 128, in get_processor
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     processor = processor_cls.from_pretrained(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1414, in from_pretrained
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     return cls.from_args_and_dict(args, processor_dict, **instantiation_kwargs)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1182, in from_args_and_dict
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     processor = cls(*args, **valid_kwargs)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/models/pixtral/processing_pixtral.py", line 131, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     super().__init__(image_processor, tokenizer, chat_template=chat_template)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 614, in __init__
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     self.check_argument_for_proper_class(attribute_name, arg)
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 697, in check_argument_for_proper_class
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888]     raise TypeError(
(EngineCore_DP0 pid=558096) ERROR 01-06 12:02:23 [core.py:888] TypeError: Received a MistralTokenizer for argument tokenizer, but a ('PreTrainedTokenizerBase', 'MistralCommonBackend') was expected.
(EngineCore_DP0 pid=558096) Process EngineCore_DP0:
(EngineCore_DP0 pid=558096) Traceback (most recent call last):
(EngineCore_DP0 pid=558096)   File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=558096)     self.run()
(EngineCore_DP0 pid=558096)   File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=558096)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 892, in run_engine_core
(EngineCore_DP0 pid=558096)     raise e
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 879, in run_engine_core
(EngineCore_DP0 pid=558096)     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore_DP0 pid=558096)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 644, in __init__
(EngineCore_DP0 pid=558096)     super().__init__(
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 105, in __init__
(EngineCore_DP0 pid=558096)     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=558096)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/executor/abstract.py", line 101, in __init__
(EngineCore_DP0 pid=558096)     self._init_executor()
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/executor/uniproc_executor.py", line 47, in _init_executor
(EngineCore_DP0 pid=558096)     self.driver_worker.init_device()
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/worker/worker_base.py", line 326, in init_device
(EngineCore_DP0 pid=558096)     self.worker.init_device()  # type: ignore
(EngineCore_DP0 pid=558096)     ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/worker/gpu_worker.py", line 264, in init_device
(EngineCore_DP0 pid=558096)     self.model_runner = GPUModelRunnerV1(self.vllm_config, self.device)
(EngineCore_DP0 pid=558096)                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/worker/gpu_model_runner.py", line 611, in __init__
(EngineCore_DP0 pid=558096)     MultiModalBudget(
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/v1/worker/utils.py", line 48, in __init__
(EngineCore_DP0 pid=558096)     max_tokens_by_modality = mm_registry.get_max_tokens_per_item_by_modality(
(EngineCore_DP0 pid=558096)                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/multimodal/registry.py", line 171, in get_max_tokens_per_item_by_modality
(EngineCore_DP0 pid=558096)     return profiler.get_mm_max_tokens(
(EngineCore_DP0 pid=558096)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 350, in get_mm_max_tokens
(EngineCore_DP0 pid=558096)     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(EngineCore_DP0 pid=558096)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 263, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=558096)     processor_inputs = factory.get_dummy_processor_inputs(
(EngineCore_DP0 pid=558096)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 120, in get_dummy_processor_inputs
(EngineCore_DP0 pid=558096)     dummy_text = self.get_dummy_text(mm_counts)
(EngineCore_DP0 pid=558096)                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/model_executor/models/mistral3.py", line 228, in get_dummy_text
(EngineCore_DP0 pid=558096)     processor = self.info.get_hf_processor()
(EngineCore_DP0 pid=558096)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/model_executor/models/mistral3.py", line 257, in get_hf_processor
(EngineCore_DP0 pid=558096)     return self.ctx.get_hf_processor(PixtralProcessor, **kwargs)
(EngineCore_DP0 pid=558096)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1189, in get_hf_processor
(EngineCore_DP0 pid=558096)     return cached_processor_from_config(
(EngineCore_DP0 pid=558096)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/transformers_utils/processor.py", line 251, in cached_processor_from_config
(EngineCore_DP0 pid=558096)     return cached_get_processor_without_dynamic_kwargs(
(EngineCore_DP0 pid=558096)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/transformers_utils/processor.py", line 224, in cached_get_processor_without_dynamic_kwargs
(EngineCore_DP0 pid=558096)     final_processor = cached_get_processor(
(EngineCore_DP0 pid=558096)                       ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm/vllm/transformers_utils/processor.py", line 128, in get_processor
(EngineCore_DP0 pid=558096)     processor = processor_cls.from_pretrained(
(EngineCore_DP0 pid=558096)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1414, in from_pretrained
(EngineCore_DP0 pid=558096)     return cls.from_args_and_dict(args, processor_dict, **instantiation_kwargs)
(EngineCore_DP0 pid=558096)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1182, in from_args_and_dict
(EngineCore_DP0 pid=558096)     processor = cls(*args, **valid_kwargs)
(EngineCore_DP0 pid=558096)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/models/pixtral/processing_pixtral.py", line 131, in __init__
(EngineCore_DP0 pid=558096)     super().__init__(image_processor, tokenizer, chat_template=chat_template)
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 614, in __init__
(EngineCore_DP0 pid=558096)     self.check_argument_for_proper_class(attribute_name, arg)
(EngineCore_DP0 pid=558096)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 697, in check_argument_for_proper_class
(EngineCore_DP0 pid=558096)     raise TypeError(
(EngineCore_DP0 pid=558096) TypeError: Received a MistralTokenizer for argument tokenizer, but a ('PreTrainedTokenizerBase', 'MistralCommonBackend') was expected.
[rank0]:[W106 12:02:24.980891838 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(APIServer pid=557989) Traceback (most recent call last):
(APIServer pid=557989)   File "/home/user/source/vllm-src-venv/bin/vllm", line 10, in <module>
(APIServer pid=557989)     sys.exit(main())
(APIServer pid=557989)              ^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=557989)     args.dispatch_function(args)
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=557989)     uvloop.run(run_server(args))
(APIServer pid=557989)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=557989)     return __asyncio.run(
(APIServer pid=557989)            ^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=557989)     return runner.run(main)
(APIServer pid=557989)            ^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=557989)     return self._loop.run_until_complete(task)
(APIServer pid=557989)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=557989)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=557989)     return await main
(APIServer pid=557989)            ^^^^^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 1325, in run_server
(APIServer pid=557989)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 1344, in run_server_worker
(APIServer pid=557989)     async with build_async_engine_client(
(APIServer pid=557989)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=557989)     return await anext(self.gen)
(APIServer pid=557989)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 171, in build_async_engine_client
(APIServer pid=557989)     async with build_async_engine_client_from_engine_args(
(APIServer pid=557989)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=557989)     return await anext(self.gen)
(APIServer pid=557989)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 212, in build_async_engine_client_from_engine_args
(APIServer pid=557989)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=557989)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=557989)     return cls(
(APIServer pid=557989)            ^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/async_llm.py", line 134, in __init__
(APIServer pid=557989)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=557989)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/core_client.py", line 122, in make_async_mp_client
(APIServer pid=557989)     return AsyncMPClient(*client_args)
(APIServer pid=557989)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/core_client.py", line 824, in __init__
(APIServer pid=557989)     super().__init__(
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/core_client.py", line 479, in __init__
(APIServer pid=557989)     with launch_core_engines(vllm_config, executor_class, log_stats) as (
(APIServer pid=557989)   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
(APIServer pid=557989)     next(self.gen)
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/utils.py", line 921, in launch_core_engines
(APIServer pid=557989)     wait_for_engine_startup(
(APIServer pid=557989)   File "/home/user/source/vllm/vllm/v1/engine/utils.py", line 980, in wait_for_engine_startup
(APIServer pid=557989)     raise RuntimeError(
(APIServer pid=557989) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

[ArthurZucker]

Arf tell us if we can do anything / if its on our side, very happy to fix!

[dgouju]

@ArthurZucker yes, it seems that the next step is on your side

[adi776borate]

Hey, I have tried to fix the new error . Please review it and let me know if this fixes the issue.

[dgouju]

Hi @adi776borate ! I tried, and a new error was raised:

VLLM_USE_FLASHINFER_MOE_FP4=1 ENABLE_NVFP4_SM120=1 VLLM_ATTENTION_BACKEND=FLASHINFER vllm serve RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4 --tensor-parallel-size=1 --no-enable-prefix-caching --kv-cache-dtype=fp8 --seed=42 --disable-log-requests --download_dir /home/user/models/ --max-model-len=8192 --max-num-batched-tokens=2048 --max-num-seqs=256 --gpu-memory-utilization=0.97
WARNING 01-07 07:40:11 [attention.py:82] Using VLLM_ATTENTION_BACKEND environment variable is deprecated and will be removed in v0.14.0 or v1.0.0, whichever is soonest. Please use --attention-config.backend command line argument or AttentionConfig(backend=...) config field instead.
(APIServer pid=632249) INFO 01-07 07:40:11 [api_server.py:1278] vLLM API server version 0.1.dev12692+g05ce285bc.d20260106
(APIServer pid=632249) INFO 01-07 07:40:11 [utils.py:253] non-default args: {'model_tag': 'RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', 'model': 'RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', 'seed': 42, 'max_model_len': 8192, 'download_dir': '/home/user/models/', 'gpu_memory_utilization': 0.97, 'kv_cache_dtype': 'fp8', 'enable_prefix_caching': False, 'max_num_batched_tokens': 2048, 'max_num_seqs': 256}
(APIServer pid=632249) INFO 01-07 07:40:13 [model.py:522] Resolved architecture: Mistral3ForConditionalGeneration
(APIServer pid=632249) INFO 01-07 07:40:13 [model.py:1508] Using max model len 8192
(APIServer pid=632249) WARNING 01-07 07:40:13 [vllm.py:1447] Current vLLM config is not set.
(APIServer pid=632249) INFO 01-07 07:40:13 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=632249) INFO 01-07 07:40:13 [vllm.py:635] Disabling NCCL for DP synchronization when using async scheduling.
(APIServer pid=632249) INFO 01-07 07:40:13 [vllm.py:640] Asynchronous scheduling is enabled.
(APIServer pid=632249) INFO 01-07 07:40:13 [cache.py:205] Using fp8 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor.
(APIServer pid=632249) INFO 01-07 07:40:13 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=632249) [2026-01-07 07:40:13] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(APIServer pid=632249) [2026-01-07 07:40:13] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-07 07:40:18 [vllm.py:1447] Current vLLM config is not set.
INFO 01-07 07:40:18 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
WARNING 01-07 07:40:18 [attention.py:82] Using VLLM_ATTENTION_BACKEND environment variable is deprecated and will be removed in v0.14.0 or v1.0.0, whichever is soonest. Please use --attention-config.backend command line argument or AttentionConfig(backend=...) config field instead.
INFO 01-07 07:40:18 [vllm.py:635] Disabling NCCL for DP synchronization when using async scheduling.
INFO 01-07 07:40:18 [vllm.py:640] Asynchronous scheduling is enabled.
(EngineCore_DP0 pid=632375) INFO 01-07 07:40:18 [core.py:96] Initializing a V1 LLM engine (v0.1.dev12692+g05ce285bc.d20260106) with config: model='RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', speculative_config=None, tokenizer='RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=8192, download_dir='/home/user/models/', load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=fp8, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False), seed=42, served_model_name=RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:18] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:18] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=632375) /home/user/source/vllm-src-venv/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.)
(EngineCore_DP0 pid=632375)   _C._set_float32_matmul_precision(precision)
(EngineCore_DP0 pid=632375) INFO 01-07 07:40:19 [parallel_state.py:1214] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.0.106:38403 backend=nccl
(EngineCore_DP0 pid=632375) INFO 01-07 07:40:19 [parallel_state.py:1425] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:23] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:23] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:25] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:25] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:26] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=632375) [2026-01-07 07:40:26] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] EngineCore failed to start.
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] Traceback (most recent call last):
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1261, in call_hf_processor
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/models/pixtral/processing_pixtral.py", line 220, in __call__
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     self._check_special_mm_tokens(prompt_strings, text_inputs, modalities=["image"])
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1912, in _check_special_mm_tokens
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     raise ValueError(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] ValueError: Mismatch in `image` token count between text and `input_ids`. Got ids=[0] and text=[1]. Likely due to `truncation='max_length'`. Please disable truncation or increase `max_length`.
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] 
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] 
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] Traceback (most recent call last):
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 879, in run_engine_core
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 644, in __init__
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     super().__init__(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 105, in __init__
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/executor/abstract.py", line 101, in __init__
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     self._init_executor()
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/executor/uniproc_executor.py", line 47, in _init_executor
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     self.driver_worker.init_device()
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/worker_base.py", line 326, in init_device
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     self.worker.init_device()  # type: ignore
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/gpu_worker.py", line 264, in init_device
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     self.model_runner = GPUModelRunnerV1(self.vllm_config, self.device)
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/gpu_model_runner.py", line 611, in __init__
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     MultiModalBudget(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/v1/worker/utils.py", line 48, in __init__
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     max_tokens_by_modality = mm_registry.get_max_tokens_per_item_by_modality(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/registry.py", line 171, in get_max_tokens_per_item_by_modality
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     return profiler.get_mm_max_tokens(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 350, in get_mm_max_tokens
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 267, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     return self.processor.apply(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 2361, in apply
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     ) = self._cached_apply_hf_processor(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 2138, in _cached_apply_hf_processor
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     ) = self._apply_hf_processor_main(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1868, in _apply_hf_processor_main
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     prompt_ids = self._apply_hf_processor_text_only(prompt, tokenization_kwargs)
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1788, in _apply_hf_processor_text_only
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     prompt_ids, _, _ = self._apply_hf_processor_text_mm(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1757, in _apply_hf_processor_text_mm
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     processed_data = self._call_hf_processor(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                      ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/model_executor/models/mistral3.py", line 268, in _call_hf_processor
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     processed_outputs = super()._call_hf_processor(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1717, in _call_hf_processor
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     return self.info.ctx.call_hf_processor(
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1290, in call_hf_processor
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888]     raise ValueError(msg) from exc
(EngineCore_DP0 pid=632375) ERROR 01-07 07:40:26 [core.py:888] ValueError: Failed to apply PixtralProcessor on data={'text': '[IMG]'} with kwargs={'truncation': False}
(EngineCore_DP0 pid=632375) Process EngineCore_DP0:
(EngineCore_DP0 pid=632375) Traceback (most recent call last):
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1261, in call_hf_processor
(EngineCore_DP0 pid=632375)     output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(EngineCore_DP0 pid=632375)              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/models/pixtral/processing_pixtral.py", line 220, in __call__
(EngineCore_DP0 pid=632375)     self._check_special_mm_tokens(prompt_strings, text_inputs, modalities=["image"])
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1912, in _check_special_mm_tokens
(EngineCore_DP0 pid=632375)     raise ValueError(
(EngineCore_DP0 pid=632375) ValueError: Mismatch in `image` token count between text and `input_ids`. Got ids=[0] and text=[1]. Likely due to `truncation='max_length'`. Please disable truncation or increase `max_length`.
(EngineCore_DP0 pid=632375) 
(EngineCore_DP0 pid=632375) The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=632375) 
(EngineCore_DP0 pid=632375) Traceback (most recent call last):
(EngineCore_DP0 pid=632375)   File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=632375)     self.run()
(EngineCore_DP0 pid=632375)   File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=632375)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 892, in run_engine_core
(EngineCore_DP0 pid=632375)     raise e
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 879, in run_engine_core
(EngineCore_DP0 pid=632375)     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore_DP0 pid=632375)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 644, in __init__
(EngineCore_DP0 pid=632375)     super().__init__(
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/engine/core.py", line 105, in __init__
(EngineCore_DP0 pid=632375)     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=632375)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/executor/abstract.py", line 101, in __init__
(EngineCore_DP0 pid=632375)     self._init_executor()
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/executor/uniproc_executor.py", line 47, in _init_executor
(EngineCore_DP0 pid=632375)     self.driver_worker.init_device()
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/worker/worker_base.py", line 326, in init_device
(EngineCore_DP0 pid=632375)     self.worker.init_device()  # type: ignore
(EngineCore_DP0 pid=632375)     ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/worker/gpu_worker.py", line 264, in init_device
(EngineCore_DP0 pid=632375)     self.model_runner = GPUModelRunnerV1(self.vllm_config, self.device)
(EngineCore_DP0 pid=632375)                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/worker/gpu_model_runner.py", line 611, in __init__
(EngineCore_DP0 pid=632375)     MultiModalBudget(
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/v1/worker/utils.py", line 48, in __init__
(EngineCore_DP0 pid=632375)     max_tokens_by_modality = mm_registry.get_max_tokens_per_item_by_modality(
(EngineCore_DP0 pid=632375)                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/registry.py", line 171, in get_max_tokens_per_item_by_modality
(EngineCore_DP0 pid=632375)     return profiler.get_mm_max_tokens(
(EngineCore_DP0 pid=632375)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 350, in get_mm_max_tokens
(EngineCore_DP0 pid=632375)     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(EngineCore_DP0 pid=632375)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/profiling.py", line 267, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=632375)     return self.processor.apply(
(EngineCore_DP0 pid=632375)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 2361, in apply
(EngineCore_DP0 pid=632375)     ) = self._cached_apply_hf_processor(
(EngineCore_DP0 pid=632375)         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 2138, in _cached_apply_hf_processor
(EngineCore_DP0 pid=632375)     ) = self._apply_hf_processor_main(
(EngineCore_DP0 pid=632375)         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1868, in _apply_hf_processor_main
(EngineCore_DP0 pid=632375)     prompt_ids = self._apply_hf_processor_text_only(prompt, tokenization_kwargs)
(EngineCore_DP0 pid=632375)                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1788, in _apply_hf_processor_text_only
(EngineCore_DP0 pid=632375)     prompt_ids, _, _ = self._apply_hf_processor_text_mm(
(EngineCore_DP0 pid=632375)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1757, in _apply_hf_processor_text_mm
(EngineCore_DP0 pid=632375)     processed_data = self._call_hf_processor(
(EngineCore_DP0 pid=632375)                      ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/model_executor/models/mistral3.py", line 268, in _call_hf_processor
(EngineCore_DP0 pid=632375)     processed_outputs = super()._call_hf_processor(
(EngineCore_DP0 pid=632375)                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1717, in _call_hf_processor
(EngineCore_DP0 pid=632375)     return self.info.ctx.call_hf_processor(
(EngineCore_DP0 pid=632375)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=632375)   File "/home/user/source/vllm/vllm/multimodal/processing.py", line 1290, in call_hf_processor
(EngineCore_DP0 pid=632375)     raise ValueError(msg) from exc
(EngineCore_DP0 pid=632375) ValueError: Failed to apply PixtralProcessor on data={'text': '[IMG]'} with kwargs={'truncation': False}
[rank0]:[W107 07:40:27.510240726 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(APIServer pid=632249) Traceback (most recent call last):
(APIServer pid=632249)   File "/home/user/source/vllm-src-venv/bin/vllm", line 10, in <module>
(APIServer pid=632249)     sys.exit(main())
(APIServer pid=632249)              ^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=632249)     args.dispatch_function(args)
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=632249)     uvloop.run(run_server(args))
(APIServer pid=632249)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=632249)     return __asyncio.run(
(APIServer pid=632249)            ^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=632249)     return runner.run(main)
(APIServer pid=632249)            ^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=632249)     return self._loop.run_until_complete(task)
(APIServer pid=632249)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=632249)   File "/home/user/source/vllm-src-venv/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=632249)     return await main
(APIServer pid=632249)            ^^^^^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 1325, in run_server
(APIServer pid=632249)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 1344, in run_server_worker
(APIServer pid=632249)     async with build_async_engine_client(
(APIServer pid=632249)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=632249)     return await anext(self.gen)
(APIServer pid=632249)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 171, in build_async_engine_client
(APIServer pid=632249)     async with build_async_engine_client_from_engine_args(
(APIServer pid=632249)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=632249)     return await anext(self.gen)
(APIServer pid=632249)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/entrypoints/openai/api_server.py", line 212, in build_async_engine_client_from_engine_args
(APIServer pid=632249)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=632249)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=632249)     return cls(
(APIServer pid=632249)            ^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/async_llm.py", line 134, in __init__
(APIServer pid=632249)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=632249)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/core_client.py", line 122, in make_async_mp_client
(APIServer pid=632249)     return AsyncMPClient(*client_args)
(APIServer pid=632249)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/core_client.py", line 824, in __init__
(APIServer pid=632249)     super().__init__(
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/core_client.py", line 479, in __init__
(APIServer pid=632249)     with launch_core_engines(vllm_config, executor_class, log_stats) as (
(APIServer pid=632249)   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
(APIServer pid=632249)     next(self.gen)
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/utils.py", line 921, in launch_core_engines
(APIServer pid=632249)     wait_for_engine_startup(
(APIServer pid=632249)   File "/home/user/source/vllm/vllm/v1/engine/utils.py", line 980, in wait_for_engine_startup
(APIServer pid=632249)     raise RuntimeError(
(APIServer pid=632249) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

[adi776borate]

The Pixtral Repo doesn't have tokenizer.json and tokenizer_config.json. That's why the special token [IMG] (and several others) are getting split instead of treated as a single token.

@ArthurZucker Should Mistral add the missing tokenizer files to the Pixtral repo?
Or should MistralConverter be more robust and not require the special_tokens key in tekken.json?

Minimal script to check:

from transformers import AutoTokenizer

model_id = "mistralai/Pixtral-12B-2409"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

text = "[IMG]"
ids = tokenizer.encode(text, add_special_tokens=False)

print(f"Model: {model_id}")
print(f"Token: {text}")
print(f"IDs:   {ids}")
print(f"Count: {len(ids)}")

[dgouju]

@adi776borate the model I'm using does have tokenizer.json and tokenizer_config.json : https://huggingface.co/RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4/tree/main

[dgouju]

Python 3.12.3 (main, Nov  6 2025, 13:44:16) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import AutoTokenizer
>>> model_id="RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4"
>>> tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
>>> 
>>> text = "[IMG]"
>>> ids = tokenizer.encode(text, add_special_tokens=False)
>>> 
>>> print(f"Model: {model_id}")
Model: RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-NVFP4
>>> print(f"Token: {text}")
Token: [IMG]
>>> print(f"IDs:   {ids}")
IDs:   [10]
>>> print(f"Count: {len(ids)}")
Count: 1
>>> exit()

[ArthurZucker]

@adi776borate no in general we should have automatic extraction. Do you want to open a PR to update the conversion? Its in here:

transformers/src/transformers/integrations/mistral.py

Line 8 in 57278c9

class MistralConverter:

[adi776borate]

@adi776borate no in general we should have automatic extraction. Do you want to open a PR to update the conversion? Its in here:

transformers/src/transformers/integrations/mistral.py

Line 8 in 57278c9

class MistralConverter:

Yes, I would love to work on that.