CoolFace
Datasetpublic

sbintuitions/JCommonsenseQA

評価スコアの再現性確保と SB Intuitions 修正版の公開用クローン ソース: yahoojapan/JGLUE on GitHub datasets/jcommonsenseqa-v1.1 JCommonsenseQA JCommonsenseQA is a Japanese version of CommonsenseQA (Talmor+, 2019), which is a multiple-choice question answering dataset that requires commonsense reasoning ability. It is built using crowdsourcing with seeds extracted from the knowledge base ConceptNet. Licensing Information Creative Commons Attribution Share Alike 4.0 International… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/JCommonsenseQA.

sourceHugging Facecc-by-sa-4.0updated 1y agoView on Hugging Face
3likes324downloads
Dataset Card

評価スコアの再現性確保と SB Intuitions 修正版の公開用クローン

ソース: yahoojapan/JGLUE on GitHub

JCommonsenseQA

JCommonsenseQA is a Japanese version of CommonsenseQA (Talmor+, 2019), which is a multiple-choice question answering dataset that requires commonsense reasoning ability. It is built using crowdsourcing with seeds extracted from the knowledge base ConceptNet.

Licensing Information

Creative Commons Attribution Share Alike 4.0 International

Citation Information

@article{栗原 健太郎2023,
  title={JGLUE: 日本語言語理解ベンチマーク},
  author={栗原 健太郎 and 河原 大輔 and 柴田 知秀},
  journal={自然言語処理},
  volume={30},
  number={1},
  pages={63-87},
  year={2023},
  url = "https://www.jstage.jst.go.jp/article/jnlp/30/1/30_63/_article/-char/ja",
  doi={10.5715/jnlp.30.63}
}

@inproceedings{kurihara-etal-2022-jglue,
    title = "{JGLUE}: {J}apanese General Language Understanding Evaluation",
    author = "Kurihara, Kentaro  and
      Kawahara, Daisuke  and
      Shibata, Tomohide",
    booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference",
    month = jun,
    year = "2022",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2022.lrec-1.317",
    pages = "2957--2966",
    abstract = "To develop high-performance natural language understanding (NLU) models, it is necessary to have a benchmark to evaluate and analyze NLU ability from various perspectives. While the English NLU benchmark, GLUE, has been the forerunner, benchmarks are now being released for languages other than English, such as CLUE for Chinese and FLUE for French; but there is no such benchmark for Japanese. We build a Japanese NLU benchmark, JGLUE, from scratch without translation to measure the general NLU ability in Japanese. We hope that JGLUE will facilitate NLU research in Japanese.",
}

@InProceedings{Kurihara_nlp2022,
  author = 	"栗原健太郎 and 河原大輔 and 柴田知秀",
  title = 	"JGLUE: 日本語言語理解ベンチマーク",
  booktitle = 	"言語処理学会第28回年次大会",
  year =	"2022",
  url = "https://www.anlp.jp/proceedings/annual_meeting/2022/pdf_dir/E8-4.pdf"
  note= "in Japanese"
}

Subsets

default

  • —q_id (str): 質問を一意識別するための ID
  • —question (str): 質問文, (未 NFKC正規化)
  • —choice{0..4} (str): 選択肢(choice0〜choice4 の 5つ), (未 NFKC正規化)
  • —label (int): choice{0..4} に対応した正解選択肢のインデックス(0-4)