[warn] vcvars64.bat not found at C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat - FlashInfer JIT may fail.
[info] MSVC env not available; setting VLLM_USE_FLASHINFER_SAMPLER=0 to skip flashinfer sampler JIT (would crash without cl.exe). Native PyTorch sampler is used instead, slightly slower but reliable.
============================================================
vLLM serve: qwen3.6-27b-autoround (Blackwell snapshot)
Model : E:\ModelScope\vllm\Qwen3.6-27B-int4-AutoRound
Ctx : 240000 | TP: 1 | PP: 1
KV dtype: fp8_e4m3 | MTP: True (n=6)
Listen : http://0.0.0.0:5001
============================================================
H:\qwen3.6-windows-server\python\python.exe -c from vllm.entrypoints.cli.main import main; main() serve E:\ModelScope\vllm\Qwen3.6-27B-int4-AutoRound --served-model-name=qwen3.6-27b-autoround --quantization=autoround --max-model-len=240000 --max-num-seqs=1 --max-num-batched-tokens=4128 --block-size=32 --enable-prefix-caching --enable-chunked-prefill --enable-auto-tool-choice --tool-call-parser=qwen3_coder --reasoning-parser=qwen3 --chat-template=H:\qwen3.6-windows-server\templates\qwen3.5-enhanced.jinja --default-chat-template-kwargs={"preserve_thinking": false} --kv-cache-dtype=fp8_e4m3 --tensor-parallel-size=1 --pipeline-parallel-size=1 --gpu-memory-utilization=0.93 --trust-remote-code --attention-backend=auto --no-use-tqdm-on-load --host=0.0.0.0 --port=5001 --data-parallel-rpc-port=8991 --limit-mm-per-prompt={"image":0,"video":0} --speculative-config={"method":"mtp","num_speculative_tokens":6}
============================================================
[launcher] tee stdout -> H:\qwen3.6-windows-server\logs\vllm_server.5001.log (also streaming to this terminal)
[launcher] runtime manifest -> H:\qwen3.6-windows-server\logs\runtime\5001.json
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:299]
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:299] █ █ █▄ ▄█
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:299] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.20.0
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:299] █▄█▀ █ █ █ █ model E:\ModelScope\vllm\Qwen3.6-27B-int4-AutoRound
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:299] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:299]
(APIServer pid=76280) INFO 05-13 21:34:51 [utils.py:233] non-default args: {'model_tag': 'E:\\ModelScope\\vllm\\Qwen3.6-27B-int4-AutoRound', 'chat_template': 'H:\\qwen3.6-windows-server\\templates\\qwen3.5-enhanced.jinja', 'default_chat_template_kwargs': {'preserve_thinking': False}, 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_coder', 'host': '0.0.0.0', 'port': 5001, 'model': 'E:\\ModelScope\\vllm\\Qwen3.6-27B-int4-AutoRound', 'trust_remote_code': True, 'max_model_len': 240000, 'quantization': 'autoround', 'served_model_name': ['qwen3.6-27b-autoround'], 'use_tqdm_on_load': False, 'attention_backend': 'auto', 'reasoning_parser': 'qwen3', 'data_parallel_rpc_port': 8991, 'block_size': 32, 'gpu_memory_utilization': 0.93, 'kv_cache_dtype': 'fp8_e4m3', 'enable_prefix_caching': True, 'limit_mm_per_prompt': {'image': 0, 'video': 0}, 'max_num_batched_tokens': 4128, 'max_num_seqs': 1, 'enable_chunked_prefill': True, 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 6}}
(APIServer pid=76280) INFO 05-13 21:34:51 [system_utils.py:284] Windows detected, skipping ulimit adjustment.
(APIServer pid=76280) WARNING 05-13 21:34:51 [envs.py:1821] Unknown vLLM environment variable detected: VLLM_DISABLE_CUSTOM_OPS
(APIServer pid=76280) WARNING 05-13 21:34:51 [envs.py:1821] Unknown vLLM environment variable detected: VLLM_MODEL_DIR
(APIServer pid=76280) WARNING 05-13 21:34:51 [envs.py:1821] Unknown vLLM environment variable detected: VLLM_NO_MARLIN
(APIServer pid=76280) WARNING 05-13 21:34:51 [envs.py:1821] Unknown vLLM environment variable detected: VLLM_SLEEP_WHEN_IDLE
(APIServer pid=76280) INFO 05-13 21:34:51 [model.py:563] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=76280) INFO 05-13 21:34:51 [model.py:1692] Using max model len 240000
(APIServer pid=76280) INFO 05-13 21:34:52 [nixl_utils.py:20] Setting UCX_RCACHE_MAX_UNRELEASED to '1024' to avoid a rare memory leak in UCX when using NIXL.
(APIServer pid=76280) WARNING 05-13 21:34:52 [nixl_utils.py:34] NIXL is not available
(APIServer pid=76280) WARNING 05-13 21:34:52 [nixl_utils.py:44] NIXL agent config is not available
(APIServer pid=76280) INFO 05-13 21:34:53 [cache.py:261] Using fp8_e4m3 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor
(APIServer pid=76280) INFO 05-13 21:34:53 [model.py:563] Resolved architecture: Qwen3_5MTP
(APIServer pid=76280) INFO 05-13 21:34:53 [model.py:1692] Using max model len 262144
(APIServer pid=76280) WARNING 05-13 21:34:53 [speculative.py:654] Enabling num_speculative_tokens > 1 will run multiple times of forward on same MTP layer,which may result in lower acceptance rate
(APIServer pid=76280) INFO 05-13 21:34:53 [scheduler.py:239] Chunked prefill is enabled with max_num_batched_tokens=4128.
(APIServer pid=76280) WARNING 05-13 21:34:53 [config.py:367] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(APIServer pid=76280) INFO 05-13 21:34:53 [config.py:387] Warning: Prefix caching in Mamba cache 'align' mode is currently enabled. Its support for Mamba layers is experimental. Please report any issues you may observe.
(APIServer pid=76280) INFO 05-13 21:34:53 [vllm.py:840] Asynchronous scheduling is enabled.
(APIServer pid=76280) INFO 05-13 21:34:53 [kernel.py:205] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'])
(APIServer pid=76280) WARNING 05-13 21:34:53 [vllm.py:1402] max_num_scheduled_tokens is set to 4128 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens or max_num_seqs.
(APIServer pid=76280) [transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
(APIServer pid=76280) INFO 05-13 21:34:54 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(EngineCore pid=44048) INFO 05-13 21:35:00 [core.py:109] Initializing a V1 LLM engine (v0.20.0) with config: model='E:\\ModelScope\\vllm\\Qwen3.6-27B-int4-AutoRound', speculative_config=SpeculativeConfig(method='mtp', model='E:\\ModelScope\\vllm\\Qwen3.6-27B-int4-AutoRound', num_spec_tokens=6), tokenizer='E:\\ModelScope\\vllm\\Qwen3.6-27B-int4-AutoRound', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=240000, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=inc, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=fp8_e4m3, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='qwen3', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=qwen3.6-27b-autoround, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [4128], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 8, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native']), enable_flashinfer_autotune=True, moe_backend='auto')
(EngineCore pid=44048) WARNING 05-13 21:35:00 [nixl_utils.py:34] NIXL is not available
(EngineCore pid=44048) WARNING 05-13 21:35:00 [nixl_utils.py:44] NIXL agent config is not available
(EngineCore pid=44048) [transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
(EngineCore pid=44048) INFO 05-13 21:35:02 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(EngineCore pid=44048) INFO 05-13 21:35:02 [parallel_state.py:1440] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://192.168.50.198:4041 backend=gloo
[W513 21:35:02.000000000 socket.cpp:764] [c10d] The client socket has failed to connect to [prometheusultra]:4041 (system error: 10049 - �����������У�������ĵ�ַ��Ч��).
(EngineCore pid=44048) INFO 05-13 21:35:02 [parallel_state.py:1753] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=44048) H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\spec_decode\llm_base_proposer.py:145: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
(EngineCore pid=44048) self.input_ids = torch.zeros(
(EngineCore pid=44048) WARNING 05-13 21:35:02 [__init__.py:206] min_p and logit_bias parameters won't work with speculative decoding.
(EngineCore pid=44048) INFO 05-13 21:35:02 [gpu_model_runner.py:4777] Starting to load model E:\ModelScope\vllm\Qwen3.6-27B-int4-AutoRound...
(EngineCore pid=44048) INFO 05-13 21:35:02 [cuda.py:423] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=44048) INFO 05-13 21:35:03 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=44048) INFO 05-13 21:35:04 [gptq_marlin.py:387] Using MarlinLinearKernel for GPTQMarlinLinearMethod
(EngineCore pid=44048) INFO 05-13 21:35:04 [gdn_linear_attn.py:153] Using Triton/FLA GDN prefill kernel
(EngineCore pid=44048) INFO 05-13 21:35:04 [cuda.py:368] Using FLASHINFER attention backend out of potential backends: ['FLASHINFER', 'TRITON_ATTN'].
(EngineCore pid=44048) INFO 05-13 21:35:04 [weight_utils.py:904] Filesystem type for checkpoints: unknown. Checkpoint size: 17.69 GiB. Available RAM: 17.94 GiB.
(EngineCore pid=44048) INFO 05-13 21:35:04 [weight_utils.py:934] Auto-prefetch is disabled because the filesystem (unknown) is not a recognized network FS (NFS/Lustre) and the checkpoint size (17.69 GiB) exceeds 90% of available RAM (17.94 GiB).
(EngineCore pid=44048) INFO 05-13 21:35:13 [default_loader.py:384] Loading weights took 8.62 seconds
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] EngineCore failed to start.
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] Traceback (most recent call last):
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 1110, in run_engine_core
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] return func(*args, **kwargs)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 876, in __init__
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] super().__init__(
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 118, in __init__
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self.model_executor = executor_class(vllm_config)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] return func(*args, **kwargs)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\abstract.py", line 109, in __init__
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self._init_executor()
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\uniproc_executor.py", line 52, in _init_executor
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self.driver_worker.load_model()
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\gpu_worker.py", line 324, in load_model
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] return func(*args, **kwargs)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\gpu_model_runner.py", line 4793, in load_model
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self.model = model_loader.load_model(
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] return func(*args, **kwargs)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\model_executor\model_loader\base_loader.py", line 80, in load_model
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] process_weights_after_loading(model, model_config, target_device)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\model_executor\model_loader\utils.py", line 111, in process_weights_after_loading
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] quant_method.process_weights_after_loading(module)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\model_executor\layers\quantization\gptq_marlin.py", line 486, in process_weights_after_loading
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self.kernel.process_weights_after_loading(layer)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\model_executor\kernels\linear\mixed_precision\marlin.py", line 167, in process_weights_after_loading
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] self._transform_param(layer, self.w_q_name, transform_w_q)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\model_executor\kernels\linear\mixed_precision\MPLinearKernel.py", line 74, in _transform_param
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] new_param = fn(old_param)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\model_executor\kernels\linear\mixed_precision\marlin.py", line 99, in transform_w_q
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] x.data = ops.gptq_marlin_repack(
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\vllm\_custom_ops.py", line 1279, in gptq_marlin_repack
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] return torch.ops._C.gptq_marlin_repack(
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] File "H:\qwen3.6-windows-server\python\Lib\site-packages\torch\_ops.py", line 1269, in __call__
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] return self._op(*args, **kwargs)
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] torch.AcceleratorError: CUDA error: the provided PTX was compiled with an unsupported toolchain.
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] Search for `cudaErrorUnsupportedPtxVersion' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] For debugging consider passing CUDA_LAUNCH_BLOCKING=1
(EngineCore pid=44048) ERROR 05-13 21:35:13 [core.py:1136] Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
(e.g. dropped to enforce-eager, disabled MTP, ran the coherence check, etc.)
What happened
(Describe the failure in 1–3 sentences.)
Hardware + OS
GPU(s), model and driver version (
nvidia-smi -q | head -25)Wed May 13 21:12:35 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 591.86 Driver Version: 591.86 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 5090 WDDM | 00000000:02:00.0 On | N/A |
| 0% 49C P8 27W / 600W | 2502MiB / 32607MiB | 1% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
Windows build (
winver, paste the build number)版本 Windows 11 专业工作站版
版本号 25H2
安装日期 2025/11/1
操作系统版本 26200.8117
体验 Windows 功能体验包 1000.26100.297.0
CPU + RAM (relevant only for very large prefills)
处理器 Intel(R) Core(TM) Ultra 7 265K (3.90 GHz)
机带 RAM 64.0 GB (63.4 GB 可用)
Single GPU? Is the display attached to it?
Install path
H:\qwen3.6-windows-serve
vLLM version (
python -c "import vllm; print(vllm.__version__)"):H:\qwen3.6-windows-server\python>python -c "import vllm; print(vllm.version)
0.20.0
Snapshot you tried
H:\qwen3.6-windows-server\snapshots\start_5090.bat
Py
H:\qwen3.6-windows-server\snapshots\start_5090.py
start_speed/start_127k/start_pp2_160k/ other (specify):Reproducer
(The exact command or click sequence that triggered the failure.)
Logs
Paste the boot section of
logs\vllm_server.<port>.log(first ~50 lines)plus the last ~50 lines around the failure. Don't worry about size, we'd
rather have too much than too little.
Verifier output
What you've already tried
(e.g. dropped to enforce-eager, disabled MTP, ran the coherence check, etc.)