datasets.md

Datasets in Natural Language Processing

Sequence Labeling

• [2002 CoNLL] Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition, [paper], [bibtex], [dataset].
• [2003 CoNLL] Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition, [paper], [bibtex], [dataset].
• [2017 CoNLL] CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, [paper], [bibtex], [homepage].
• [2017 ACL] Cross-lingual Name Tagging and Linking for 282 Languages, [paper], [bibtex], [homepage].

Machine Comprehension and Question Answering

• [2013 EMNLP] MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text, [paper], [bibtex], [homepage], source: [mcobzarenco/mctest].
• [2015 NIPS] CNN/DailyMail: Teaching Machines to Read and Comprehend, [paper], [bibtex], [homepage], sources: [thomasmesnard/DeepMind-Teaching-Machines-to-Read-and-Comprehend].
• [2016 EMNLP] SQuAD 100,000+ Questions for Machine Comprehension of Text, [paper], [bibtex], [homepage].
• [2016 ICLR] bAbI: Towards AI-Complete Question Answering: a Set of Prerequisite Toy Tasks, [paper], [bibtex], [homepage], sources: [facebook/bAbI-tasks].
• [2017 EMNLP] World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions, [paper], [bibtex], [homepage].
• [2017 EMNLP] RACE: Large-scale ReAding Comprehension Dataset From Examinations, [paper], [bibtex], [homepage], sources: [qizhex/RACE_AR_baselines].
• [2017 ACL] TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension, [paper], [bibtex] [homepage], sources: [mandarjoshi90/triviaqa].
• [2018 TACL] QAngaroo: Constructing Datasets for Multi-hop Reading Comprehension Across Documents, [paper], [bibtex], [homepage].
• [2018 ICLR] CLOTH: Large-scale Cloze Test Dataset Designed by Teachers, [paper], [bibtex], [homepage], sources: [qizhex/Large-scale-Cloze-Test-Dataset-Designed-by-Teachers].
• [2018 NAACL] MultiRC: Looking Beyond the Surface - A Challenge Set for Reading Comprehension over Multiple Sentences, [paper], [bibtex], [homepage], sources: [CogComp/multirc].
• [2018 EMNLP] HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering, [paper], [bibtex], [attachment], [homepage], sources: [hotpotqa/hotpot].

Commonsense Knowledge Base

• [2017 AAAI] ConceptNet 5.5: An Open Multilingual Graph of General Knowledge, [paper], [bibtex], sources: [GitHub page], [commonsense/conceptnet5], [commonsense/conceptnet-numberbatch].