Skip to content

[Bug] RuntimeError: yaml-cpp: error at line 13, column 30: bad conversion #3544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
wwwwk123 opened this issue May 13, 2025 · 7 comments · Fixed by #3575
Closed
3 tasks done

[Bug] RuntimeError: yaml-cpp: error at line 13, column 30: bad conversion #3544

wwwwk123 opened this issue May 13, 2025 · 7 comments · Fixed by #3575

Comments

@wwwwk123
Copy link

wwwwk123 commented May 13, 2025

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

使用0.8.0运行qwen3-32b时报错,此问题在相同启动命令的0.7.3中没有遇到过。
RuntimeError: yaml-cpp: error at line 13, column 30: bad conversion

经过确认,--rope-scaling-factor 无论输入多少都会报错,尝试了1.0 1.5 2.0 4.0 4,都获得了上方相同的报错,去除该参数能正常启动。

尝试了pyyaml==6.0.1与6.0.2,无效果
尝试了conda install conda-forge::yaml-cpp,无效果

2025-05-13 08:05:36,413 - lmdeploy - �[37mINFO�[0m - async_engine.py:259 - input backend=turbomind, backend_config=TurbomindEngineConfig(dtype='auto', model_format=None, tp=2, dp=1, device_num=None, attn_tp_size=None, attn_dp_size=None, mlp_tp_size=None, mlp_dp_size=None, outer_dp_size=None, session_len=131072, max_batch_size=512, cache_max_entry_count=0.85, cache_chunk_size=-1, cache_block_seq_len=128, enable_prefix_caching=True, quant_policy=0, rope_scaling_factor=4.0, use_logn_attn=False, download_dir=None, revision=None, max_prefill_token_num=32768, num_tokens_per_iter=0, max_prefill_iters=1, communicator='nccl')
2025-05-13 08:05:36,413 - lmdeploy - �[37mINFO�[0m - async_engine.py:260 - input chat_template_config=ChatTemplateConfig(model_name='qwen2d5', system=None, meta_instruction=None, eosys=None, user=None, eoh=None, assistant=None, eoa=None, tool=None, eotool=None, separator=None, capability=None, stop_words=None)
2025-05-13 08:05:36,418 - lmdeploy - �[37mINFO�[0m - async_engine.py:269 - updated chat_template_onfig=ChatTemplateConfig(model_name='qwen2d5', system=None, meta_instruction=None, eosys=None, user=None, eoh=None, assistant=None, eoa=None, tool=None, eotool=None, separator=None, capability=None, stop_words=None)
2025-05-13 08:05:36,939 - lmdeploy - �[37mINFO�[0m - turbomind.py:312 - model_source: ModelSource.HF_MODEL
2025-05-13 08:05:36,977 - lmdeploy - �[37mINFO�[0m - turbomind.py:226 - turbomind model config:

{
"model_config": {
"model_name": "",
"chat_template": "",
"model_arch": "Qwen3ForCausalLM",
"head_num": 64,
"kv_head_num": 8,
"hidden_units": 5120,
"vocab_size": 151936,
"embedding_size": 151936,
"num_layer": 64,
"inter_size": [
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600,
25600
],
"norm_eps": 1e-06,
"attn_bias": 0,
"qk_norm": true,
"size_per_head": 128,
"group_size": 64,
"weight_type": "bfloat16",
"session_len": 131072,
"attn_tp_size": 2,
"mlp_tp_size": 2,
"model_format": "hf",
"expert_num": [],
"expert_inter_size": 0,
"experts_per_token": 0,
"moe_shared_gate": false,
"norm_topk_prob": false,
"routed_scale": 1.0,
"topk_group": 1,
"topk_method": "greedy",
"moe_group_num": 1,
"q_lora_rank": 0,
"kv_lora_rank": 0,
"qk_rope_dim": 0,
"v_head_dim": 0,
"tune_layer_num": 1
},
"attention_config": {
"softmax_scale": 0.0,
"cache_block_seq_len": 128,
"use_logn_attn": 0,
"max_position_embeddings": 40960,
"rope_param": {
"type": "dynamic",
"base": 1000000.0,
"dim": 128,
"factor": 4.0,
"max_position_embeddings": null,
"attention_factor": 1.0,
"beta_fast": 32,
"beta_slow": 1,
"low_freq_factor": null,
"high_freq_factor": null,
"original_max_position_embeddings": null
}
},
"lora_config": {
"lora_policy": "",
"lora_r": 0,
"lora_scale": 0.0,
"lora_max_wo_r": 0,
"lora_rank_pattern": "",
"lora_scale_pattern": ""
},
"engine_config": {
"dtype": "auto",
"model_format": null,
"tp": 2,
"dp": 1,
"device_num": 2,
"attn_tp_size": 2,
"attn_dp_size": 1,
"mlp_tp_size": 2,
"mlp_dp_size": 1,
"outer_dp_size": 1,
"session_len": 131072,
"max_batch_size": 512,
"cache_max_entry_count": 0.85,
"cache_chunk_size": -1,
"cache_block_seq_len": 128,
"enable_prefix_caching": true,
"quant_policy": 0,
"rope_scaling_factor": 4.0,
"use_logn_attn": false,
"download_dir": null,
"revision": null,
"max_prefill_token_num": 32768,
"num_tokens_per_iter": 32768,
"max_prefill_iters": 4,
"communicator": "nccl"
}
}
Traceback (most recent call last):
File "/home/wk/miniconda3/envs/lmdeploy/bin/lmdeploy", line 8, in
sys.exit(run())
^^^^^
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/cli/entrypoint.py", line 39, in run
args.run(args)
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/cli/serve.py", line 333, in api_server
run_api_server(args.model_path,
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/serve/openai/api_server.py", line 1121, in serve
VariableInterface.async_engine = pipeline_class(model_path=model_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/serve/async_engine.py", line 277, in init
self._build_turbomind(model_path=model_path, backend_config=backend_config, **kwargs)
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/serve/async_engine.py", line 328, in _build_turbomind
self.engine = tm.TurboMind.from_pretrained(model_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/turbomind/turbomind.py", line 313, in from_pretrained
return cls(model_path=pretrained_model_name_or_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/turbomind/turbomind.py", line 149, in init
self.model_comm = self._from_hf(model_source=model_source,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wk/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/turbomind/turbomind.py", line 242, in _from_hf
model_comm = _tm.AbstractTransformerModel.create_llama_model(model_dir='',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: yaml-cpp: error at line 13, column 30: bad conversion

Reproduction

lmdeploy serve api_server "Qwen3-32B"
--model-name "Qwen3-32B"
--server-name 0.0.0.0
--server-port $PORT
--tp 2
--cache-max-entry-count 0.85
--tool-call-parser "qwen"
--chat-template qwen2d5

--max-prefill-token-num 32768
--rope-scaling-factor 4.0
--session-len 131072

--cache-block-seq-len 128
--enable-prefix-caching

--log-level "INFO"
>

Environment

lmdeploy check_env
sys.platform: linux
Python: 3.12.9 | packaged by Anaconda, Inc. | (main, Feb  6 2025, 18:56:27) [GCC 11.2.0]
CUDA available: True
MUSA available: False
numpy_random_seed: 2147483648
GPU 0,1,2,3,4,5: NVIDIA H100 80GB HBM3
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 12.8, V12.8.93
GCC: gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
PyTorch: 2.6.0+cu124
PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201703
  - Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.5.3 (Git Hash 66f0cb9eb66affd2da3bf5f8d897376f04aae6af)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX512
  - CUDA Runtime 12.4
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
  - CuDNN 90.1
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=2236df1770800ffea5697b11b0bb0d910b2e59e1, CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.6.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 

TorchVision: 0.21.0+cu124
LMDeploy: 0.8.0+8ce41ec
transformers: 4.51.3
gradio: Not Found
fastapi: 0.115.12
pydantic: 2.11.3
triton: 3.2.0
NVIDIA Topology: 
        GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    NIC0    NIC1    NIC2    NIC3    NIC4  NIC5     NIC6    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      NV18    NV18    NV18    NV18    NV18    PIX     NODE    NODE    SYS     SYS   SYS      SYS     0-23,96-119     0               N/A
GPU1    NV18     X      NV18    NV18    NV18    NV18    NODE    PIX     NODE    SYS     SYS   SYS      SYS     0-23,96-119     0               N/A
GPU2    NV18    NV18     X      NV18    NV18    NV18    NODE    NODE    PIX     SYS     SYS   SYS      SYS     0-23,96-119     0               N/A
GPU3    NV18    NV18    NV18     X      NV18    NV18    SYS     SYS     SYS     PIX     SYS   SYS      NODE    24-47,120-143   1               N/A
GPU4    NV18    NV18    NV18    NV18     X      NV18    SYS     SYS     SYS     SYS     PIX   SYS      SYS     48-71,144-167   2               N/A
GPU5    NV18    NV18    NV18    NV18    NV18     X      SYS     SYS     SYS     SYS     SYS   PIX      SYS     72-95,168-191   3               N/A
NIC0    PIX     NODE    NODE    SYS     SYS     SYS      X      NODE    NODE    SYS     SYS   SYS      SYS
NIC1    NODE    PIX     NODE    SYS     SYS     SYS     NODE     X      NODE    SYS     SYS   SYS      SYS
NIC2    NODE    NODE    PIX     SYS     SYS     SYS     NODE    NODE     X      SYS     SYS   SYS      SYS
NIC3    SYS     SYS     SYS     PIX     SYS     SYS     SYS     SYS     SYS      X      SYS   SYS      NODE
NIC4    SYS     SYS     SYS     SYS     PIX     SYS     SYS     SYS     SYS     SYS      X    SYS      SYS
NIC5    SYS     SYS     SYS     SYS     SYS     PIX     SYS     SYS     SYS     SYS     SYS    X       SYS
NIC6    SYS     SYS     SYS     NODE    SYS     SYS     SYS     SYS     SYS     NODE    SYS   SYS       X 

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1
  NIC2: mlx5_2
  NIC3: mlx5_5
  NIC4: mlx5_6
  NIC5: mlx5_7
  NIC6: mlx5_bond_0

Error traceback

@lvhan028
Copy link
Collaborator

没能复现这个问题

@lvhan028
Copy link
Collaborator

yaml-cpp 是以 csrc 源码方式编译到 turbomind中的。

FetchContent_Declare(
  yaml-cpp
  GIT_REPOSITORY https://github.com/jbeder/yaml-cpp.git
  GIT_TAG 0.8.0
)

0.7.3 也是用的 yaml-cpp 0.8.0,这个一直没有变过

@lvhan028
Copy link
Collaborator

建议在 LlamaTritonModel::LlamaTritonModel 中增加下关于 yaml config的日志,试试debug源码

@nstl-zyb
Copy link

加载qwen3-30b-A3B,遇到同样的问题,当设置rope_scaling_factor时,会出现 :RuntimeError: yaml-cpp: error at line 13, column 30: bad conversion。当rope_scaling_factor默认为0时,可以正常加载

报错如下:
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | Traceback (most recent call last):
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | return _run_code(code, main_globals, None,
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | exec(code, run_globals)
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/opt/py3/lib/python3.10/site-packages/fastchat/serve/lmdeploy_worker.py", line 560, in
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | lm_engine = async_engine.AsyncEngine(model_path=args.model_path,
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/opt/py3/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 277, in init
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | self._build_turbomind(model_path=model_path, backend_config=backend_config, **kwargs)
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/opt/py3/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 328, in _build_turbomind
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | self.engine = tm.TurboMind.from_pretrained(model_path,
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/opt/py3/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 313, in from_pretrained
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | return cls(model_path=pretrained_model_name_or_path,
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/opt/py3/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 149, in init
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | self.model_comm = self._from_hf(model_source=model_source,
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | File "/opt/py3/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 242, in _from_hf
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | model_comm = _tm.AbstractTransformerModel.create_llama_model(model_dir='',
[2025-05-19 10:39:10] 2025-05-19 02:39:10 | ERROR | stderr | RuntimeError: yaml-cpp: error at line 13, column 30: bad conversion
[2025-05-19 10:39:11] time="2025-05-19T02:39:11Z" level=info msg="program stopped with status:exit status 1" program=/bin/sh

调用代码如下:
backend_config = TurbomindEngineConfig(
tp=args.tp_size,
cache_max_entry_count=args.cache_max_entry_count,
cache_chunk_size=args.cache_chunk_size,
enable_prefix_caching=args.enable_prefix_caching,
session_len=args.session_len,
# max_batch_size=args.max_batch_size,
quant_policy=args.quant_policy,
rope_scaling_factor=args.rope_scaling_factor,
use_logn_attn=args.use_logn_attn,
max_prefill_token_num = args.max_prefill_token_num,
num_tokens_per_iter=args.num_tokens_per_iter,
max_prefill_iters=args.max_prefill_iters,
cache_block_seq_len=args.cache_block_seq_len,
)

lm_engine = async_engine.AsyncEngine(model_path=args.model_path,
backend="turbomind",
backend_config=backend_config,
# log_level=args.log_level,
chat_template_config=ChatTemplateConfig(args.conv_template))

@lvhan028
Copy link
Collaborator

抱歉,我没有留意到输入参数 --rope-scaling-factor

@lvhan028
Copy link
Collaborator

我在下面的 PR 中修复了,可以验证看看。
#3575

Copy link

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants