phucdev/ViMLQA

1---2dataset_info:3  features:4  - name: context5    dtype: string6  - name: question7    dtype: string8  - name: answers9    sequence:10    - name: answer_start11      dtype: int3212    - name: text13      dtype: string14  - name: id15    dtype: string16  splits:17  - name: train18    num_bytes: 9729179719    num_examples: 8481620  - name: validation21    num_bytes: 71469122    num_examples: 51123  - name: test24    num_bytes: 780083825    num_examples: 549526  download_size: 1977520427  dataset_size: 10580732628configs:29- config_name: default30  data_files:31  - split: train32    path: data/train-*33  - split: validation34    path: data/validation-*35  - split: test36    path: data/test-*37task_categories:38- question-answering39language:40- vi41size_categories:42- 10K<n<100K43---44 45Vietnamese portion of MLQA for monolingual QA46 47The train split is the machine translated train data from SQuAD v1.1. 48The validation split and the test split are from MLQA.49 50We created this particular version with the following code:51```python52import datasets53 54# Machine translated train data from SQuAD v1.1.55mlqa_train_dev = datasets.load_dataset("facebook/mlqa", "mlqa-translate-train.vi")56# Original validation and test data from MLQA57mlqa_val_test = datasets.load_dataset("facebook/mlqa", "mlqa.vi.vi")58 59# Merge and create our version60mlqa = mlqa_train_dev61mlqa["validation"] = mlqa_val_test["validation"]62mlqa["test"] = mlqa_val_test["test"]63```64 65### Citation Information66 67```68@article{lewis2019mlqa,69  title = {MLQA: Evaluating Cross-lingual Extractive Question Answering},70  author = {Lewis, Patrick and Oguz, Barlas and Rinott, Ruty and Riedel, Sebastian and Schwenk, Holger},71  journal = {arXiv preprint arXiv:1910.07475},72  year = 2019,73  eid = {arXiv: 1910.07475}74}75 76@inproceedings{rajpurkar-etal-2016-squad,77    title = "{SQ}u{AD}: 100,000+ Questions for Machine Comprehension of Text",78    author = "Rajpurkar, Pranav  and79      Zhang, Jian  and80      Lopyrev, Konstantin  and81      Liang, Percy",82    editor = "Su, Jian  and83      Duh, Kevin  and84      Carreras, Xavier",85    booktitle = "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing",86    month = nov,87    year = "2016",88    address = "Austin, Texas",89    publisher = "Association for Computational Linguistics",90    url = "https://aclanthology.org/D16-1264",91    doi = "10.18653/v1/D16-1264",92    pages = "2383--2392",93    eprint={1606.05250},94    archivePrefix={arXiv},95    primaryClass={cs.CL},96}97 98```