Datasetpublic

phucdev/ViMLQA

Vietnamese portion of MLQA for monolingual QA The train split is the machine translated train data from SQuAD v1.1. The validation split and the test split are from MLQA. We created this particular version with the following code: import datasets # Machine translated train data from SQuAD v1.1. mlqa_train_dev = datasets.load_dataset("facebook/mlqa", "mlqa-translate-train.vi") # Original validation and test data from MLQA mlqa_val_test = datasets.load_dataset("facebook/mlqa", "mlqa.vi.vi") #… See the full description on the dataset page: https://huggingface.co/datasets/phucdev/ViMLQA.

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes36downloads

Dataset Card

Vietnamese portion of MLQA for monolingual QA

The train split is the machine translated train data from SQuAD v1.1. The validation split and the test split are from MLQA.

We created this particular version with the following code:

python

import datasets

# Machine translated train data from SQuAD v1.1.
mlqa_train_dev = datasets.load_dataset("facebook/mlqa", "mlqa-translate-train.vi")
# Original validation and test data from MLQA
mlqa_val_test = datasets.load_dataset("facebook/mlqa", "mlqa.vi.vi")

# Merge and create our version
mlqa = mlqa_train_dev
mlqa["validation"] = mlqa_val_test["validation"]
mlqa["test"] = mlqa_val_test["test"]

Citation Information

@article{lewis2019mlqa,
  title = {MLQA: Evaluating Cross-lingual Extractive Question Answering},
  author = {Lewis, Patrick and Oguz, Barlas and Rinott, Ruty and Riedel, Sebastian and Schwenk, Holger},
  journal = {arXiv preprint arXiv:1910.07475},
  year = 2019,
  eid = {arXiv: 1910.07475}
}

@inproceedings{rajpurkar-etal-2016-squad,
    title = "{SQ}u{AD}: 100,000+ Questions for Machine Comprehension of Text",
    author = "Rajpurkar, Pranav  and
      Zhang, Jian  and
      Lopyrev, Konstantin  and
      Liang, Percy",
    editor = "Su, Jian  and
      Duh, Kevin  and
      Carreras, Xavier",
    booktitle = "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2016",
    address = "Austin, Texas",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/D16-1264",
    doi = "10.18653/v1/D16-1264",
    pages = "2383--2392",
    eprint={1606.05250},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
}