Your current environment
it can use in ds4-sm120-preview-dev in 2026-06-05
but i update in [2026-07-05] vllm run with error report
How you are installing vllm
(vllm-ds2) wuwen@wuwen-MZ73-LM0-000:/mnt/AI-Acer4T/AI-Chat/vllm$ export SAFETENSORS_FAST_GPU=1
export TMPDIR=/tmp
export TEMP=/tmp
export TMP=/tmp
export CUDA_DEVICE_ORDER=PCI_BUS_ID
export CUDA_VISIBLE_DEVICES=0,1
vllm serve /mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/ --trust-remote-code --kv-cache-dtype fp8 --block-size 256 --tensor-parallel-size 2 --host 0.0.0.0 --port 8005 --served-model-name VLLM-MODEL deepseek-v4-flash --gpu-memory-utilization 0.95 --max-model-len 32000 --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE", "custom_ops":["all"]}' --load-format auto --pipeline-parallel-size 1 --trust-remote-code --async-scheduling --enable-prefix-caching --max-num-seqs 8 --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --enable-auto-tool-choice --reasoning-parser deepseek_v4 --default-chat-template-kwargs '{"thinking": true}' --override-generation-config '{"temperature": 1.0,"top_p": 1.0}' --speculative_config '{"method":"mtp","num_speculative_tokens":2}' --enable-expert-parallel
WARNING 07-06 12:28:06 [argparse_utils.py:422] Found duplicate keys --trust-remote-code
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339]
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] █ █ █▄ ▄█
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 20260706.dev3+gfe486bf62
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] █▄█▀ █ █ █ █ model /mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339]
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:273] non-default args: {'model_tag': '/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', 'default_chat_template_kwargs': {'thinking': True}, 'enable_auto_tool_choice': True, 'tool_call_parser': 'deepseek_v4', 'host': '0.0.0.0', 'port': 8005, 'model': '/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', 'tokenizer_mode': 'deepseek_v4', 'trust_remote_code': True, 'max_model_len': 32000, 'served_model_name': ['VLLM-MODEL', 'deepseek-v4-flash'], 'override_generation_config': {'temperature': 1.0, 'top_p': 1.0}, 'reasoning_parser': 'deepseek_v4', 'tensor_parallel_size': 2, 'enable_expert_parallel': True, 'block_size': 256, 'gpu_memory_utilization': 0.95, 'kv_cache_dtype': 'fp8', 'enable_prefix_caching': True, 'max_num_seqs': 8, 'async_scheduling': True, 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 2}, 'compilation_config': {'mode': None, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'ir_enable_torch_wrap': None, 'splitting_ops': None, 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': None, 'compile_ranges_endpoints': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': None, 'static_all_moe_layers': []}}
(APIServer pid=83220) INFO 07-06 12:28:06 [config.py:741] Detected quantization_config.scale_fmt=ue8m0; enabling UE8M0 for DeepGEMM.
(APIServer pid=83220) INFO 07-06 12:28:06 [model.py:606] Resolved architecture: DeepseekV4ForCausalLM
(APIServer pid=83220) INFO 07-06 12:28:06 [model.py:1736] Using max model len 32000
(APIServer pid=83220) INFO 07-06 12:28:08 [cache.py:280] Using fp8 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor
(APIServer pid=83220) INFO 07-06 12:28:08 [model.py:606] Resolved architecture: DeepSeekV4MTPModel
(APIServer pid=83220) INFO 07-06 12:28:08 [model.py:1736] Using max model len 1048576
(APIServer pid=83220) INFO 07-06 12:28:37 [speculative.py:1046] Overriding draft model max model len from 1048576 to 32000
(APIServer pid=83220) INFO 07-06 12:28:37 [scheduler.py:252] Chunked prefill is enabled with max_num_batched_tokens=8192.
(APIServer pid=83220) INFO 07-06 12:28:37 [vllm.py:1062] Asynchronous scheduling is enabled.
(APIServer pid=83220) INFO 07-06 12:28:37 [kernel.py:286] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=83220) WARNING 07-06 12:28:37 [vllm.py:1661] max_num_scheduled_tokens is set to 8184 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens or max_num_seqs.
(APIServer pid=83220) WARNING 07-06 12:28:38 [vllm.py:1499] Auto-initialization of reasoning token IDs failed. Please check whether your reasoning parser has implemented the reasoning_start_str and reasoning_end_str.
(APIServer pid=83220) INFO 07-06 12:28:38 [compilation.py:312] Enabled custom fusions: norm_quant, act_quant
(EngineCore pid=83768) INFO 07-06 12:29:03 [core.py:114] Initializing a V1 LLM engine (v20260706.dev3+gfe486bf62) with config: model='/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', speculative_config=SpeculativeConfig(method='dspark', model='/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', num_spec_tokens=2), tokenizer='/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', skip_tokenizer_init=False, tokenizer_mode=deepseek_v4, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32000, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=deepseek_v4_fp8, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=fp8, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='deepseek_v4', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=VLLM-MODEL, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all', '+quant_fp8', '+quant_fp8'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::qwen_gdn_attention_core', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 48, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, moe_backend='auto', linear_backend='auto')
(EngineCore pid=83768) WARNING 07-06 12:29:03 [multiproc_executor.py:1067] Reducing Torch parallelism from 32 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
(EngineCore pid=83768) INFO 07-06 12:29:03 [multiproc_executor.py:140] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=192.168.31.143 (local), world_size=2, local_world_size=2
(Worker pid=83965) INFO 07-06 12:29:24 [parallel_state.py:1588] world_size=2 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:46393 backend=nccl
(Worker pid=83966) INFO 07-06 12:29:24 [parallel_state.py:1588] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:46393 backend=nccl
(Worker pid=83965) INFO 07-06 12:29:27 [pynccl.py:113] vLLM is using nccl==2.28.9
(Worker pid=83965) WARNING 07-06 12:29:27 [symm_mem.py:66] SymmMemCommunicator: Device capability 12.0 not supported, communicator is not available.
(Worker pid=83966) WARNING 07-06 12:29:27 [symm_mem.py:66] SymmMemCommunicator: Device capability 12.0 not supported, communicator is not available.
(Worker pid=83965) INFO 07-06 12:29:27 [cuda_communicator.py:246] Using ['CUSTOM', 'PYNCCL'] all-reduce backends (in dispatch order) for group 'tp:0' out of potential backends: ['NCCL_SYMM_MEM', 'QUICK_REDUCE', 'FLASHINFER', 'CUSTOM', 'SYMM_MEM', 'PYNCCL'].
(Worker pid=83965) INFO 07-06 12:29:27 [cuda_communicator.py:246] Using ['PYNCCL'] all-reduce backends (in dispatch order) for group 'ep:0' out of potential backends: ['NCCL_SYMM_MEM', 'QUICK_REDUCE', 'FLASHINFER', 'CUSTOM', 'SYMM_MEM', 'PYNCCL'].
(Worker pid=83965) INFO 07-06 12:29:27 [parallel_state.py:1923] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
(Worker pid=83965) INFO 07-06 12:29:27 [topk_topp_sampler.py:55] Using FlashInfer for top-p & top-k sampling.
(Worker pid=83966) WARNING 07-06 12:29:27 [init.py:204] min_p and logit_bias parameters won't work with speculative decoding.
(Worker pid=83965) WARNING 07-06 12:29:27 [init.py:204] min_p and logit_bias parameters won't work with speculative decoding.
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:27 [gpu_model_runner.py:5169] Starting to load model /mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/...
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:28 [quant_config.py:75] DeepSeek V4 expert_dtype resolved to 'fp4'
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:28 [init.py:563] Selected TritonFp8BlockScaledMMKernel for Fp8LinearMethod
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:28 [attention.py:88] Using DeepSeek's fp8_ds_mla KV cache format.
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] WorkerProc failed to start.
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] Traceback (most recent call last):
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 865, in worker_main
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] worker = WorkerProc(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 634, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.worker.load_model()
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_worker.py", line 413, in load_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_model_runner.py", line 5185, in load_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = model_loader.load_model(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = initialize_model(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/utils.py", line 63, in initialize_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1532, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = self.model_cls(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1145, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.start_layer, self.end_layer, self.layers = make_layers(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 711, in make_layers
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] + get_offloader().wrap_modules(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/offloader/base.py", line 104, in wrap_modules
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return list(modules_generator)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 712, in
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] layer_fn(prefix=f"{prefix}.{idx}") for idx in range(start_layer, end_layer)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1147, in
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] lambda prefix: DeepseekV4DecoderLayer(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 964, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.attn = _select_dsv4_attn_cls(vllm_config)(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/flashinfer_sm120_decode.py", line 142, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] from flashinfer.mla._sparse_mla_sm120 import _SparseMLAPagedAttentionRunner
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ModuleNotFoundError: No module named 'flashinfer.mla._sparse_mla_sm120'
(EngineCore pid=83768) INFO 07-06 12:29:28 [multiproc_executor.py:426] [shutdown] Executor: waiting for worker exit count=2
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] WorkerProc failed to start.
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] Traceback (most recent call last):
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 865, in worker_main
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] worker = WorkerProc(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 634, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.worker.load_model()
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_worker.py", line 413, in load_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_model_runner.py", line 5185, in load_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = model_loader.load_model(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = initialize_model(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/utils.py", line 63, in initialize_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1532, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = self.model_cls(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1145, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.start_layer, self.end_layer, self.layers = make_layers(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 711, in make_layers
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] + get_offloader().wrap_modules(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/offloader/base.py", line 104, in wrap_modules
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return list(modules_generator)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 712, in
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] layer_fn(prefix=f"{prefix}.{idx}") for idx in range(start_layer, end_layer)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1147, in
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] lambda prefix: DeepseekV4DecoderLayer(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 964, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.attn = _select_dsv4_attn_cls(vllm_config)(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/flashinfer_sm120_decode.py", line 142, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] from flashinfer.mla._sparse_mla_sm120 import _SparseMLAPagedAttentionRunner
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ModuleNotFoundError: No module named 'flashinfer.mla._sparse_mla_sm120'
[rank0]:[W706 12:29:29.915183184 ProcessGroupNCCL.cpp:1575] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(EngineCore pid=83768) INFO 07-06 12:29:30 [multiproc_executor.py:433] [shutdown] Executor: all workers exited gracefully
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] EngineCore failed to start.
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] Traceback (most recent call last):
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 1200, in run_engine_core
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] return func(*args, **kwargs)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 966, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] super().init(
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 123, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] self.model_executor = executor_class(vllm_config)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 108, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] super().init(vllm_config)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] return func(*args, **kwargs)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/abstract.py", line 109, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] self._init_executor()
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 201, in _init_executor
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 762, in wait_for_ready
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] raise e from None
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore pid=83768) Process EngineCore:
(EngineCore pid=83768) Traceback (most recent call last):
(EngineCore pid=83768) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=83768) self.run()
(EngineCore pid=83768) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore pid=83768) self._target(*self._args, **self._kwargs)
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 1235, in run_engine_core
(EngineCore pid=83768) raise e
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 1200, in run_engine_core
(EngineCore pid=83768) engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) return func(*args, **kwargs)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 966, in init
(EngineCore pid=83768) super().init(
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 123, in init
(EngineCore pid=83768) self.model_executor = executor_class(vllm_config)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 108, in init
(EngineCore pid=83768) super().init(vllm_config)
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) return func(*args, **kwargs)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/abstract.py", line 109, in init
(EngineCore pid=83768) self._init_executor()
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 201, in _init_executor
(EngineCore pid=83768) self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 762, in wait_for_ready
(EngineCore pid=83768) raise e from None
(EngineCore pid=83768) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=83220) Traceback (most recent call last):
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/bin/vllm", line 6, in
(APIServer pid=83220) sys.exit(main())
(APIServer pid=83220) ^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/cli/main.py", line 95, in main
(APIServer pid=83220) args.dispatch_function(args)
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/cli/serve.py", line 148, in cmd
(APIServer pid=83220) uvloop.run(run_server(args))
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/site-packages/uvloop/init.py", line 96, in run
(APIServer pid=83220) return __asyncio.run(
(APIServer pid=83220) ^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=83220) return runner.run(main)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=83220) return self._loop.run_until_complete(task)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=83220) return await main
(APIServer pid=83220) ^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 692, in run_server
(APIServer pid=83220) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 706, in run_server_worker
(APIServer pid=83220) async with build_async_engine_client(
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/contextlib.py", line 204, in aenter
(APIServer pid=83220) return await anext(self.gen)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=83220) async with build_async_engine_client_from_engine_args(
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/contextlib.py", line 204, in aenter
(APIServer pid=83220) return await anext(self.gen)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 136, in build_async_engine_client_from_engine_args
(APIServer pid=83220) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/async_llm.py", line 217, in from_vllm_config
(APIServer pid=83220) return cls(
(APIServer pid=83220) ^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/async_llm.py", line 146, in init
(APIServer pid=83220) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=83220) return func(*args, **kwargs)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core_client.py", line 132, in make_async_mp_client
(APIServer pid=83220) return AsyncMPClient(*client_args)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=83220) return func(*args, **kwargs)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core_client.py", line 963, in init
(APIServer pid=83220) super().init(
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core_client.py", line 573, in init
(APIServer pid=83220) with launch_core_engines(
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/contextlib.py", line 144, in exit
(APIServer pid=83220) next(self.gen)
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/utils.py", line 1213, in launch_core_engines
(APIServer pid=83220) wait_for_engine_startup(
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/utils.py", line 1272, in wait_for_engine_startup
(APIServer pid=83220) raise RuntimeError(
(APIServer pid=83220) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Before submitting a new issue...
Your current environment
it can use in ds4-sm120-preview-dev in 2026-06-05
but i update in [2026-07-05] vllm run with error report
How you are installing vllm
(vllm-ds2) wuwen@wuwen-MZ73-LM0-000:/mnt/AI-Acer4T/AI-Chat/vllm$ export SAFETENSORS_FAST_GPU=1
export TMPDIR=/tmp
export TEMP=/tmp
export TMP=/tmp
export CUDA_DEVICE_ORDER=PCI_BUS_ID
export CUDA_VISIBLE_DEVICES=0,1
vllm serve /mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/ --trust-remote-code --kv-cache-dtype fp8 --block-size 256 --tensor-parallel-size 2 --host 0.0.0.0 --port 8005 --served-model-name VLLM-MODEL deepseek-v4-flash --gpu-memory-utilization 0.95 --max-model-len 32000 --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE", "custom_ops":["all"]}' --load-format auto --pipeline-parallel-size 1 --trust-remote-code --async-scheduling --enable-prefix-caching --max-num-seqs 8 --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --enable-auto-tool-choice --reasoning-parser deepseek_v4 --default-chat-template-kwargs '{"thinking": true}' --override-generation-config '{"temperature": 1.0,"top_p": 1.0}' --speculative_config '{"method":"mtp","num_speculative_tokens":2}' --enable-expert-parallel
WARNING 07-06 12:28:06 [argparse_utils.py:422] Found duplicate keys --trust-remote-code
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339]
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] █ █ █▄ ▄█
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 20260706.dev3+gfe486bf62
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] █▄█▀ █ █ █ █ model /mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:339]
(APIServer pid=83220) INFO 07-06 12:28:06 [api_utils.py:273] non-default args: {'model_tag': '/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', 'default_chat_template_kwargs': {'thinking': True}, 'enable_auto_tool_choice': True, 'tool_call_parser': 'deepseek_v4', 'host': '0.0.0.0', 'port': 8005, 'model': '/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', 'tokenizer_mode': 'deepseek_v4', 'trust_remote_code': True, 'max_model_len': 32000, 'served_model_name': ['VLLM-MODEL', 'deepseek-v4-flash'], 'override_generation_config': {'temperature': 1.0, 'top_p': 1.0}, 'reasoning_parser': 'deepseek_v4', 'tensor_parallel_size': 2, 'enable_expert_parallel': True, 'block_size': 256, 'gpu_memory_utilization': 0.95, 'kv_cache_dtype': 'fp8', 'enable_prefix_caching': True, 'max_num_seqs': 8, 'async_scheduling': True, 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 2}, 'compilation_config': {'mode': None, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'ir_enable_torch_wrap': None, 'splitting_ops': None, 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': None, 'compile_ranges_endpoints': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': None, 'static_all_moe_layers': []}}
(APIServer pid=83220) INFO 07-06 12:28:06 [config.py:741] Detected quantization_config.scale_fmt=ue8m0; enabling UE8M0 for DeepGEMM.
(APIServer pid=83220) INFO 07-06 12:28:06 [model.py:606] Resolved architecture: DeepseekV4ForCausalLM
(APIServer pid=83220) INFO 07-06 12:28:06 [model.py:1736] Using max model len 32000
(APIServer pid=83220) INFO 07-06 12:28:08 [cache.py:280] Using fp8 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor
(APIServer pid=83220) INFO 07-06 12:28:08 [model.py:606] Resolved architecture: DeepSeekV4MTPModel
(APIServer pid=83220) INFO 07-06 12:28:08 [model.py:1736] Using max model len 1048576
(APIServer pid=83220) INFO 07-06 12:28:37 [speculative.py:1046] Overriding draft model max model len from 1048576 to 32000
(APIServer pid=83220) INFO 07-06 12:28:37 [scheduler.py:252] Chunked prefill is enabled with max_num_batched_tokens=8192.
(APIServer pid=83220) INFO 07-06 12:28:37 [vllm.py:1062] Asynchronous scheduling is enabled.
(APIServer pid=83220) INFO 07-06 12:28:37 [kernel.py:286] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=83220) WARNING 07-06 12:28:37 [vllm.py:1661] max_num_scheduled_tokens is set to 8184 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens or max_num_seqs.
(APIServer pid=83220) WARNING 07-06 12:28:38 [vllm.py:1499] Auto-initialization of reasoning token IDs failed. Please check whether your reasoning parser has implemented the
reasoning_start_strandreasoning_end_str.(APIServer pid=83220) INFO 07-06 12:28:38 [compilation.py:312] Enabled custom fusions: norm_quant, act_quant
(EngineCore pid=83768) INFO 07-06 12:29:03 [core.py:114] Initializing a V1 LLM engine (v20260706.dev3+gfe486bf62) with config: model='/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', speculative_config=SpeculativeConfig(method='dspark', model='/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', num_spec_tokens=2), tokenizer='/mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/', skip_tokenizer_init=False, tokenizer_mode=deepseek_v4, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32000, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=deepseek_v4_fp8, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=fp8, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='deepseek_v4', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=VLLM-MODEL, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all', '+quant_fp8', '+quant_fp8'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::qwen_gdn_attention_core', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 48, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, moe_backend='auto', linear_backend='auto')
(EngineCore pid=83768) WARNING 07-06 12:29:03 [multiproc_executor.py:1067] Reducing Torch parallelism from 32 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
(EngineCore pid=83768) INFO 07-06 12:29:03 [multiproc_executor.py:140] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=192.168.31.143 (local), world_size=2, local_world_size=2
(Worker pid=83965) INFO 07-06 12:29:24 [parallel_state.py:1588] world_size=2 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:46393 backend=nccl
(Worker pid=83966) INFO 07-06 12:29:24 [parallel_state.py:1588] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:46393 backend=nccl
(Worker pid=83965) INFO 07-06 12:29:27 [pynccl.py:113] vLLM is using nccl==2.28.9
(Worker pid=83965) WARNING 07-06 12:29:27 [symm_mem.py:66] SymmMemCommunicator: Device capability 12.0 not supported, communicator is not available.
(Worker pid=83966) WARNING 07-06 12:29:27 [symm_mem.py:66] SymmMemCommunicator: Device capability 12.0 not supported, communicator is not available.
(Worker pid=83965) INFO 07-06 12:29:27 [cuda_communicator.py:246] Using ['CUSTOM', 'PYNCCL'] all-reduce backends (in dispatch order) for group 'tp:0' out of potential backends: ['NCCL_SYMM_MEM', 'QUICK_REDUCE', 'FLASHINFER', 'CUSTOM', 'SYMM_MEM', 'PYNCCL'].
(Worker pid=83965) INFO 07-06 12:29:27 [cuda_communicator.py:246] Using ['PYNCCL'] all-reduce backends (in dispatch order) for group 'ep:0' out of potential backends: ['NCCL_SYMM_MEM', 'QUICK_REDUCE', 'FLASHINFER', 'CUSTOM', 'SYMM_MEM', 'PYNCCL'].
(Worker pid=83965) INFO 07-06 12:29:27 [parallel_state.py:1923] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
(Worker pid=83965) INFO 07-06 12:29:27 [topk_topp_sampler.py:55] Using FlashInfer for top-p & top-k sampling.
(Worker pid=83966) WARNING 07-06 12:29:27 [init.py:204] min_p and logit_bias parameters won't work with speculative decoding.
(Worker pid=83965) WARNING 07-06 12:29:27 [init.py:204] min_p and logit_bias parameters won't work with speculative decoding.
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:27 [gpu_model_runner.py:5169] Starting to load model /mnt/AI-Acer4T/AI-Chat/models/Deepseek/DeepSeek-V4-Flash-DSpark/...
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:28 [quant_config.py:75] DeepSeek V4 expert_dtype resolved to 'fp4'
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:28 [init.py:563] Selected TritonFp8BlockScaledMMKernel for Fp8LinearMethod
(Worker_TP0_EP0 pid=83965) INFO 07-06 12:29:28 [attention.py:88] Using DeepSeek's fp8_ds_mla KV cache format.
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] WorkerProc failed to start.
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] Traceback (most recent call last):
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 865, in worker_main
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] worker = WorkerProc(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 634, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.worker.load_model()
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_worker.py", line 413, in load_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_model_runner.py", line 5185, in load_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = model_loader.load_model(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = initialize_model(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/utils.py", line 63, in initialize_model
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1532, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = self.model_cls(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1145, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.start_layer, self.end_layer, self.layers = make_layers(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 711, in make_layers
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] + get_offloader().wrap_modules(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/offloader/base.py", line 104, in wrap_modules
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return list(modules_generator)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 712, in
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] layer_fn(prefix=f"{prefix}.{idx}") for idx in range(start_layer, end_layer)
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1147, in
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] lambda prefix: DeepseekV4DecoderLayer(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 964, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.attn = _select_dsv4_attn_cls(vllm_config)(
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/flashinfer_sm120_decode.py", line 142, in init
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] from flashinfer.mla._sparse_mla_sm120 import _SparseMLAPagedAttentionRunner
(Worker_TP1_EP1 pid=83966) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ModuleNotFoundError: No module named 'flashinfer.mla._sparse_mla_sm120'
(EngineCore pid=83768) INFO 07-06 12:29:28 [multiproc_executor.py:426] [shutdown] Executor: waiting for worker exit count=2
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] WorkerProc failed to start.
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] Traceback (most recent call last):
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 865, in worker_main
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] worker = WorkerProc(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 634, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.worker.load_model()
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_worker.py", line 413, in load_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/worker/gpu_model_runner.py", line 5185, in load_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = model_loader.load_model(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = initialize_model(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/model_loader/utils.py", line 63, in initialize_model
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1532, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.model = self.model_cls(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1145, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.start_layer, self.end_layer, self.layers = make_layers(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 711, in make_layers
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] + get_offloader().wrap_modules(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/offloader/base.py", line 104, in wrap_modules
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] return list(modules_generator)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/model_executor/models/utils.py", line 712, in
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] layer_fn(prefix=f"{prefix}.{idx}") for idx in range(start_layer, end_layer)
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1147, in
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] lambda prefix: DeepseekV4DecoderLayer(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 964, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] self.attn = _select_dsv4_attn_cls(vllm_config)(
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/models/deepseek_v4/nvidia/flashinfer_sm120_decode.py", line 142, in init
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] from flashinfer.mla._sparse_mla_sm120 import _SparseMLAPagedAttentionRunner
(Worker_TP0_EP0 pid=83965) ERROR 07-06 12:29:28 [multiproc_executor.py:898] ModuleNotFoundError: No module named 'flashinfer.mla._sparse_mla_sm120'
[rank0]:[W706 12:29:29.915183184 ProcessGroupNCCL.cpp:1575] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(EngineCore pid=83768) INFO 07-06 12:29:30 [multiproc_executor.py:433] [shutdown] Executor: all workers exited gracefully
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] EngineCore failed to start.
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] Traceback (most recent call last):
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 1200, in run_engine_core
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] return func(*args, **kwargs)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 966, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] super().init(
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 123, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] self.model_executor = executor_class(vllm_config)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 108, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] super().init(vllm_config)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] return func(*args, **kwargs)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/abstract.py", line 109, in init
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] self._init_executor()
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 201, in _init_executor
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 762, in wait_for_ready
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] raise e from None
(EngineCore pid=83768) ERROR 07-06 12:29:30 [core.py:1231] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore pid=83768) Process EngineCore:
(EngineCore pid=83768) Traceback (most recent call last):
(EngineCore pid=83768) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=83768) self.run()
(EngineCore pid=83768) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore pid=83768) self._target(*self._args, **self._kwargs)
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 1235, in run_engine_core
(EngineCore pid=83768) raise e
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 1200, in run_engine_core
(EngineCore pid=83768) engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) return func(*args, **kwargs)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 966, in init
(EngineCore pid=83768) super().init(
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core.py", line 123, in init
(EngineCore pid=83768) self.model_executor = executor_class(vllm_config)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 108, in init
(EngineCore pid=83768) super().init(vllm_config)
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=83768) return func(*args, **kwargs)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/abstract.py", line 109, in init
(EngineCore pid=83768) self._init_executor()
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 201, in _init_executor
(EngineCore pid=83768) self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=83768) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=83768) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/executor/multiproc_executor.py", line 762, in wait_for_ready
(EngineCore pid=83768) raise e from None
(EngineCore pid=83768) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=83220) Traceback (most recent call last):
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/bin/vllm", line 6, in
(APIServer pid=83220) sys.exit(main())
(APIServer pid=83220) ^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/cli/main.py", line 95, in main
(APIServer pid=83220) args.dispatch_function(args)
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/cli/serve.py", line 148, in cmd
(APIServer pid=83220) uvloop.run(run_server(args))
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/site-packages/uvloop/init.py", line 96, in run
(APIServer pid=83220) return __asyncio.run(
(APIServer pid=83220) ^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=83220) return runner.run(main)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=83220) return self._loop.run_until_complete(task)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=83220) return await main
(APIServer pid=83220) ^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 692, in run_server
(APIServer pid=83220) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 706, in run_server_worker
(APIServer pid=83220) async with build_async_engine_client(
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/contextlib.py", line 204, in aenter
(APIServer pid=83220) return await anext(self.gen)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=83220) async with build_async_engine_client_from_engine_args(
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/contextlib.py", line 204, in aenter
(APIServer pid=83220) return await anext(self.gen)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/entrypoints/openai/api_server.py", line 136, in build_async_engine_client_from_engine_args
(APIServer pid=83220) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/async_llm.py", line 217, in from_vllm_config
(APIServer pid=83220) return cls(
(APIServer pid=83220) ^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/async_llm.py", line 146, in init
(APIServer pid=83220) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=83220) return func(*args, **kwargs)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core_client.py", line 132, in make_async_mp_client
(APIServer pid=83220) return AsyncMPClient(*client_args)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=83220) return func(*args, **kwargs)
(APIServer pid=83220) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core_client.py", line 963, in init
(APIServer pid=83220) super().init(
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/core_client.py", line 573, in init
(APIServer pid=83220) with launch_core_engines(
(APIServer pid=83220) File "/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/contextlib.py", line 144, in exit
(APIServer pid=83220) next(self.gen)
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/utils.py", line 1213, in launch_core_engines
(APIServer pid=83220) wait_for_engine_startup(
(APIServer pid=83220) File "/mnt/AI-Acer4T/AI-Chat/vllm/vllm/v1/engine/utils.py", line 1272, in wait_for_engine_startup
(APIServer pid=83220) raise RuntimeError(
(APIServer pid=83220) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
/mnt/AI-Acer4T/miniconda3/envs/vllm-ds2/lib/python3.12/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Before submitting a new issue...