The dataset for training fastText OH-2.5 +ELI5 text classifier

[yqy2001]

Hi, Thanks for the great work. Will you release the dataset (ELI5 + OH-2.5) used for training the fastText OH-2.5 + ELI5 text classifier?

Thank you.

[Mivg]

Hi @yqy2001,
We are looking into this. Please follow #74 which is also asks the same

[yqy2001]

Thank you!

[afang-story]

I don't think we can release the data, but we will update if this changes. You can find OpenHermes-2.5 on Hugging Face, and instructions for reproducing the ELI5 portion are in Appendix I.1 in the paper.

[AlphaSue]

@afang-story Hi Fang, I'm wondering if you can give us one real example. I'm wondering how you concat the training data from origin json data, becasue I found the format matters for pretraining.

the origin paper is very great and mentioned how to filter it:
"""
We curate examples for training fastTextmodels by taking a post
and combining it with the top-scoring answer (using the karma score derived from
community up/down-votes). If there are ties, the longest answer is chosen. We also
filter these examples by keeping only those where the post has score ≥3, the best
comment has score ≥5, and there are at least 3 comments total.
"""

[afang-story]

We just concatenate the question and the answer with whitespace (multiple spaces). I'm not sure if changing this would significantly affect downstream evaluations. What have you tried and how were the results different?