utu/practice/README.md 处描述上传自定义本地数据集存在错误

[itlantu]

错误描述

• https://github.com/TencentCloudADP/youtu-agent/blob/main/utu/practice/README.md

Step1 Option 2: Upload Custom Local Datasets （上传自定义数据集）处描述dataset,source,question,answer为必填字段，但使用官方提供的数据模板，调用scripts/data/upload_dataset.py后实测报错

单条数据

{"dataset": "YourDataset", "source": "training_free_grpo", "question": "What is 2+2?", "answer": "4"}

报错

Traceback (most recent call last):
  File "C:\Users\IT_la\Desktop\mbpp_youtu\youtu-agent\scripts\data\upload_dataset.py", line 78, in <module>
    main()
    ~~~~^^
  File "C:\Users\IT_la\Desktop\mbpp_youtu\youtu-agent\scripts\data\upload_dataset.py", line 74, in main
    upload_dataset(args.file_path, args.dataset_name, data_format=args.data_format)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IT_la\Desktop\mbpp_youtu\youtu-agent\scripts\data\upload_dataset.py", line 47, in upload_dataset
    dataset_sample = convert_format_llamafactory(data)
  File "C:\Users\IT_la\Desktop\mbpp_youtu\youtu-agent\scripts\data\upload_dataset.py", line 25, in convert_format_llamafactory
    assert len(question) > 0, "Either 'instruction' or 'input' must be provided."
           ^^^^^^^^^^^^^^^^^
AssertionError: Either 'instruction' or 'input' must be provided.

错误分析

分析了报错来源scripts/data/upload_dataset.py文件

• https://github.com/TencentCloudADP/youtu-agent/blob/main/scripts/data/upload_dataset.py

发现相关函数convert_format_llamafactory中产生AssertionError的代码为:

 question = [data.get("instruction", None), data.get("input", None)]
 question = [s for s in question if s is not None and s.strip() != ""]
 assert len(question) > 0, "Either 'instruction' or 'input' must be provided."

修改数据中的question字段为instruction或input即可正常上传数据

[Lightblues]

You are absolutely right! That's a quick fix, i will handle it.

[YZ-Cai]

感谢提出问题，已修复

[feiyangsuo]

btw, it seems (1) the "answer" field should be "output" and (2) the "source" field is unused.
w.r.t convert_format_llamafactory function is scripts/data/upload_dataset.py

quick fix:
(1) use "output" instead of "answer" in the data jsonl
(2) edit the function code

[YZ-Cai]

Thanks for your suggestions. Actually, Youtu-Agent uses the following format of data by default:

{
    "dataset": "YourDataset",       # Name of your dataset
    "source": "training_free_grpo", # Must be "training_free_grpo" for both practice and evaluation data
    "question": "What is 2+2?",     # The question or prompt
    "answer": "4"                   # The expected answer, set to None if not applicable
}

But the scripts/data/upload_dataset.py script uses "llamafactory" format as the default option, thus results in the mismatched issue. To make it more clear, we decide to change the default option in scripts/data/upload_dataset.py to the above data format. Therefore, the README.md of agent practice module will revert to the original data format.

By the way, the "source" field will be used for identifying the correct data processor, i.e., utu/eval/processer/training_free_grpo_processor.py