baber/logiqa2
The dataset is an amendment and re-annotation of LogiQA in 2020, a large-scale logical reasoning reading comprehension dataset adapted from the Chinese Civil Service Examination. We increase the data size, refine the texts with manual translation by professionals, and improve the quality by removing items with distinctive cultural features like Chinese idioms. Furthermore, we conduct a fine-grained annotation on the dataset and turn it into a two-way natural language inference (NLI) task, resulting in 35k premise-hypothesis pairs with gold labels, making it the first large-scale NLI dataset for complex logical reasoning
Dataset Card for Dataset Name
Dataset Description
- Homepage: https://github.com/csitfun/LogiQA2.0, https://github.com/csitfun/LogiEval
- Repository: https://github.com/csitfun/LogiQA2.0, https://github.com/csitfun/LogiEval
- Paper: https://ieeexplore.ieee.org/abstract/document/10174688
Dataset Summary
Logiqa2.0 dataset - logical reasoning in MRC and NLI tasks
LogiEval: a benchmark suite for testing logical reasoning abilities of instruct-prompt large language models
Licensing Information
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Citation Information
@ARTICLE{10174688, author={Liu, Hanmeng and Liu, Jian and Cui, Leyang and Teng, Zhiyang and Duan, Nan and Zhou, Ming and Zhang, Yue}, journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing}, title={LogiQA 2.0 — An Improved Dataset for Logical Reasoning in Natural Language Understanding}, year={2023}, volume={}, number={}, pages={1-16}, doi={10.1109/TASLP.2023.3293046}}
@misc{liu2023evaluating, title={Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4}, author={Hanmeng Liu and Ruoxi Ning and Zhiyang Teng and Jian Liu and Qiji Zhou and Yue Zhang}, year={2023}, eprint={2304.03439}, archivePrefix={arXiv}, primaryClass={cs.CL} }