datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
JMTEB
JMTEB: Japanese Massive Text Embedding Benchmark
JMTEB is a benchmark for evaluating Japanese text embedding models. It consists of 5 tasks, currently involving 28 datasets in total. You can find the update history here.
TL;DR
from datasets import load_dataset
dataset = load_dataset("sbintuitions/JMTEB", name="<dataset_name>", split="<split>")
JMTEB_DATASET_NAMES = (
'livedoor_news',
'mewsc16_ja',
'sib200_japanese_clustering'… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/JMTEB.JamC-QA
Dataset Card for JamC-QA
English/Japanese
Dataset Summary
This benchmark evaluates knowledge specific to Japan through multiple-choice questions.
It covers eight categories: culture, custom, regional_identity, geography, history, government, law, and healthcare.
Achieving high performance requires broad and detailed understanding of Japan across these categories.
Leaderboard
Evaluation Metric
In our evaluation,
the LLM outputs the… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/JamC-QA.sbibmJMTEB-lite
JMTEB-lite: The Lightweight Version of JMTEB
JMTEB-lite is a lightweight version of JMTEB.
It makes agile evaluation possible by reaching an average of 5x faster evaluation comparing with JMTEB.
The result of JMTEB-lite is proved to be highly relevant with that of JMTEB, making it a faithful preview of JMTEB.
TL;DR
from datasets import load_dataset
dataset = load_dataset("sbintuitions/JMTEB-lite", name="<dataset_name>", split="<split>")
JMTEB_LITE_DATASET_NAMES = (… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/JMTEB-lite.JCommonsenseQA評価スコアの再現性確保と SB Intuitions 修正版の公開用クローン
ソース: yahoojapan/JGLUE on GitHub
datasets/jcommonsenseqa-v1.1
JCommonsenseQA
JCommonsenseQA is a Japanese version of CommonsenseQA (Talmor+, 2019), which is a multiple-choice question answering dataset that requires commonsense reasoning ability.
It is built using crowdsourcing with seeds extracted from the knowledge base ConceptNet.
Licensing Information
Creative Commons Attribution Share Alike 4.0 International
Citation… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/JCommonsenseQA.sbirMMLU-Pro評価スコアの再現性確保と SB Intuitions 修正版の公開用クローン
ソース： TIGER-Lab/MMLU-Pro on Hugging Face
MMLU-Pro
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset
tailored to more rigorously benchmark large language models' capabilities.
This dataset contains 12K complex questions across various disciplines.
Licensing Information
MIT
Citation Information
@misc{wang2024mmlupro,
title={MMLU-Pro: A More Robust and Challenging Multi-Task… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/MMLU-Pro.SBIC_DisagreementThis dataset is processed version of Social Bias Inference Corpus(SBIC) dataset including text, annotator's demographics and the annotation disagreement labels.
Paper: Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information
Authors: Ruyuan Wan, Jaehyung Kim, Dongyeop Kang
Github repo: https://github.com/minnesotanlp/Quantifying-Annotation-Disagreement
sbir-grant-analyzer-dataSBIC.v2.agg.trn_originalafrica-tunisia-station-climatique-kairouan-sbikha-dgacta-a2d887c6
Station Climatique Kairouan Sbikha Dgacta | Africa (Tunisia Open Data)
6 rows - 1 Africa country/area - 2025 - 1 indicator - Engineered by Electric Sheep Africa
TL;DR
This dataset contains 6 rows from Tunisia Open Data, covering Station Climatique Kairouan Sbikha Dgacta. It is published as ML-ready Parquet with consistent Hugging Face metadata, source provenance, and analysis-friendly loading examples.
What This Dataset Measures
Agriculture… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-tunisia-station-climatique-kairouan-sbikha-dgacta-a2d887c6.africa-tunisia-station-climatique-kairouan-sbikha-dgacta-41eddc96
Station Climatique Kairouan Sbikha Dgacta | Africa (Tunisia Open Data)
9 rows - 1 Africa country/area - 2025 - 1 indicator - Engineered by Electric Sheep Africa
TL;DR
This dataset contains 9 rows from Tunisia Open Data, covering Station Climatique Kairouan Sbikha Dgacta. It is published as ML-ready Parquet with consistent Hugging Face metadata, source provenance, and analysis-friendly loading examples.
What This Dataset Measures
Agriculture… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-tunisia-station-climatique-kairouan-sbikha-dgacta-41eddc96.SBIC.v2.agg.dev_translateafrica-tunisia-station-climatique-kairouan-sbikha-dgacta-a9c25e5f
Station Climatique Kairouan Sbikha Dgacta | Africa (Tunisia Open Data)
9 rows - 1 Africa country/area - 2025 - 1 indicator - Engineered by Electric Sheep Africa
TL;DR
This dataset contains 9 rows from Tunisia Open Data, covering Station Climatique Kairouan Sbikha Dgacta. It is published as ML-ready Parquet with consistent Hugging Face metadata, source provenance, and analysis-friendly loading examples.
What This Dataset Measures
Agriculture… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-tunisia-station-climatique-kairouan-sbikha-dgacta-a9c25e5f.africa-tunisia-station-climatique-kairouan-sbikha-dgacta-1671a485
Station Climatique Kairouan Sbikha Dgacta | Africa (Tunisia Open Data)
27 rows - 1 Africa country/area - 2025 - 1 indicator - Engineered by Electric Sheep Africa
TL;DR
This dataset contains 27 rows from Tunisia Open Data, covering Station Climatique Kairouan Sbikha Dgacta. It is published as ML-ready Parquet with consistent Hugging Face metadata, source provenance, and analysis-friendly loading examples.
What This Dataset Measures
Agriculture… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-tunisia-station-climatique-kairouan-sbikha-dgacta-1671a485.SBIC.v2.agg.dev_originalafrica-tunisia-station-climatique-kairouan-sbikha-dgacta-2cb78372
Station Climatique Kairouan Sbikha Dgacta | Africa (Tunisia Open Data)
9 rows - 1 Africa country/area - 2025 - 1 indicator - Engineered by Electric Sheep Africa
TL;DR
This dataset contains 9 rows from Tunisia Open Data, covering Station Climatique Kairouan Sbikha Dgacta. It is published as ML-ready Parquet with consistent Hugging Face metadata, source provenance, and analysis-friendly loading examples.
What This Dataset Measures
Agriculture… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-tunisia-station-climatique-kairouan-sbikha-dgacta-2cb78372.SBIC.v2.dev_originalSBIC.v2.dev_translateSBIC.v2.agg.tst_originalSBIC.v2.tst_originalSBIC.v2.trn_translateSBIC.v2.agg.tst_translateSBIC.v2.tst_translate