datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SWE-bench_VerifiedDataset Summary
SWE-bench Verified is a subset of 500 samples from the SWE-bench test set, which have been human-validated for quality. SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. See this post for more details on the human-validation process.
The dataset collects 500 test Issue-Pull Request pairs from popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.
The original… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified.fineweb-edu-translated
Helsinki-NLP/fineweb-edu-translated
fineweb-edu-tanslated is a collection of automatically translated documents from fineweb-edu.
Translations are based on OPUS-MT and HPLT-MT models.
The data in v1.0 covers 36,704,000 documents with over 28 billion space-searated tokens of English data translated into 36 languages.
The total v1.0 data set includes over 960 billion tokens and the translated documents are aligned across all languages.
In the v1.1 release, additional translations… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/fineweb-edu-translated.Calc-mawps
Dataset Card for Calc-MAWPS
Summary
The dataset is a collection of simple math word problems focused on arithmetics. It is derived from https://huggingface.co/datasets/omarxadel/MaWPS-ar.
The main addition in this dataset variant is the chain column. It was created by converting the solution to a simple html-like language that can be easily
parsed (e.g. by BeautifulSoup). The data contains 3 types of tags:
gadget: A tag whose content is intended to be evaluated by… See the full description on the dataset page: https://huggingface.co/datasets/MU-NLPC/Calc-mawps.SWE-bench_Lite
Dataset Summary
SWE-bench Lite is subset of SWE-bench, a dataset that tests systems’ ability to solve GitHub issues automatically. The dataset collects 300 test Issue-Pull Request pairs from 11 popular Python. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.
The dataset was released as part of SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Want to run inference now?
This dataset only contains the… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/SWE-bench_Lite.nemotron-cc-translated
Helsinki-NLP/nemotron-cc-translated
nemotron-cc-tanslated is a collection of automatically translated documents from nemotron-cc taken out of the high-quality subset.
Translations are based on OPUS-MT and HPLT-MT models.
The data in v1.0 covers 156,431,999 documents with over 70 billion space-searated tokens of English data translated into 36 languages.
The total v1.0 data set includes over 2.4 trillion tokens and the translated documents are aligned across all languages.
v1.1… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/nemotron-cc-translated.SWE-bench
Dataset Summary
SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. The dataset collects 2,294 Issue-Pull Request pairs from 12 popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.
The dataset was released as part of SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Want to run inference now?
This dataset only contains the problem_statement… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/SWE-bench.WebLINX-full
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
WARNING: This is not the main WebLINX data card! You might want to use the main WebLINX data card instead:
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
Xing Han Lù*, Zdeněk Kasner*, Siva Reddy
💾Code
📄Paper
🌐Website
📓Colab
🤖Models
💻Explorer
🐦Tweets
🏆Leaderboard
Your browser does not support the… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/WebLINX-full.wikiann
Dataset Card for WikiANN
Dataset Summary
WikiANN (sometimes called PAN-X) is a multilingual named entity recognition dataset consisting of Wikipedia articles annotated with LOC (location), PER (person), and ORG (organisation) tags in the IOB2 format. This version corresponds to the balanced train, dev, and test splits of Rahimi et al. (2019), which supports 176 of the 282 languages from the original WikiANN corpus.
Supported Tasks and Leaderboards… See the full description on the dataset page: https://huggingface.co/datasets/unimelb-nlp/wikiann.opus-100
Dataset Card for OPUS-100
Dataset Summary
OPUS-100 is an English-centric multilingual corpus covering 100 languages.
OPUS-100 is English-centric, meaning that all training pairs include English on either the source or target side. The corpus covers 100 languages (including English).
The languages were selected based on the volume of parallel data available in OPUS.
Supported Tasks and Leaderboards
Translation.
Languages
OPUS-100 contains… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/opus-100.agent-reward-bench
AgentRewardBench
💾Code
📄Paper
🌐Website
🤗Dataset
💻Demo
🏆Leaderboard
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent TrajectoriesXing Han Lù, Amirhossein Kazemnejad*, Nicholas Meade, Arkil Patel, Dongchan Shin, Alejandra Zambrano, Karolina Stańczak, Peter Shaw, Christopher J. Pal, Siva Reddy*Core Contributor
Loading dataset
You can use the huggingface_hub library to load the dataset. The dataset is available on Huggingface Hub at… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/agent-reward-bench.FlashRAG_datasets
⚡FlashRAG: A Python Toolkit for Efficient RAG Research
FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 16 state-of-the-art RAG algorithms.
With FlashRAG and provided resources, you can effortlessly reproduce existing SOTA works in the RAG domain or implement your custom RAG processes and components.
For more information, please view our GitHub repo… See the full description on the dataset page: https://huggingface.co/datasets/RUC-NLPIR/FlashRAG_datasets.multiun
Dataset Card for OPUS MultiUN
Dataset Summary
The MultiUN parallel corpus is extracted from the United Nations Website , and then cleaned and converted to XML at Language Technology Lab in DFKI GmbH (LT-DFKI), Germany. The documents were published by UN from 2000 to 2009.
This is a collection of translated documents from the United Nations originally compiled by Andreas Eisele and Yu Chen (see http://www.euromatrixplus.net/multi-un/).
This corpus is available in all 6… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/multiun.Calc-asdiv_a
Dataset Card for Calc-asdiv_a
Summary
The dataset is a collection of simple math word problems focused on arithmetics. It is derived from the arithmetic subset of ASDiv (original repo).
The main addition in this dataset variant is the chain column. It was created by converting the solution to a simple html-like language that can be easily
parsed (e.g. by BeautifulSoup). The data contains 3 types of tags:
gadget: A tag whose content is intended to be evaluated by calling… See the full description on the dataset page: https://huggingface.co/datasets/MU-NLPC/Calc-asdiv_a.europarl
Dataset Card for OPUS Europarl (European Parliament Proceedings Parallel Corpus)
Dataset Summary
A parallel corpus extracted from the European Parliament web site by Philipp Koehn (University of Edinburgh).
The main intended use is to aid statistical machine translation research.
More information can be found at http://www.statmt.org/europarl/
Supported Tasks and Leaderboards
Tasks: Machine Translation, Cross Lingual Word Embeddings (CWLE) Alignment… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/europarl.CharXiv
CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
NeurIPS 2024
🏠Home (🚧Still in construction) | 🤗Data | 🥇Leaderboard | 🖥️Code | 📄Paper
This repo contains the full dataset for our paper CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs, which is a diverse and challenging chart understanding benchmark fully curated by human experts. It includes 2,323 high-resolution charts manually sourced from arXiv preprints. Each chart is… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/CharXiv.SimpleRL-Zoo-Dataopus_books
Dataset Card for OPUS Books
Dataset Summary
This is a collection of copyright free books aligned by Andras Farkas, which are available from http://www.farkastranslations.com/bilingual_books.php
Note that the texts are rather dated due to copyright issues and that some of them are manually reviewed (check the meta-data at the top of the corpus files in XML). The source is multilingually aligned, which is available from http://www.farkastranslations.com/bilingual_books.php.… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/opus_books.un_pc
Dataset Card for United Nations Parallel Corpus
Dataset Summary
The United Nations Parallel Corpus is the first parallel corpus composed from United Nations documents published by the original data creator.
The parallel corpus consists of manually translated UN documents from the last 25 years (1990 to 2014)
for the six official UN languages, Arabic, Chinese, English, French, Russian, and Spanish.
The corpus is freely available for download under a liberal license.… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/un_pc.SWE-bench_Multimodal
SWE-bench Multimodal
SWE-bench Multimodal is a dataset of 617 task instances that evalutes Language Models and AI Systems on their ability to resolve real world GitHub issues.
To learn more about the dataset, please visit our website.
More updates coming soon!
flickr30k
Flickr30k
Original paper: From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions
Homepage: https://shannon.cs.illinois.edu/DenotationGraph/
Bibtex:
@article{young2014image,
title={From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions},
author={Young, Peter and Lai, Alice and Hodosh, Micah and Hockenmaier, Julia},
journal={Transactions of the Association… See the full description on the dataset page: https://huggingface.co/datasets/nlphuji/flickr30k.PUPAThis dataset contains the data presented in the paper PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles.
Code: https://github.com/siyan-sylvia-li/PAPILLON
hleSWE-chat
SWE-chat: Coding Agent Interactions From Real Users in the Wild
📄 Paper: arxiv.org/abs/2604.20779
🌐 Website: swe-chat.com
Dataset Summary
SWE-chat captures real-world AI coding sessions from developers using AI coding assistants (Claude Code, Codex, Gemini CLI, and others via the Entire.io CLI). Each session includes the full conversation transcript, tool calls, thinking traces, code changes, and attribution of human vs. agent-authored code.
Dataset Size… See the full description on the dataset page: https://huggingface.co/datasets/SALT-NLP/SWE-chat.stereoset
Dataset Card for StereoSet
Dataset Summary
StereoSet is a dataset that measures stereotype bias in language models. StereoSet consists of 17,000 sentences that measures model preferences across gender, race, religion, and profession.
Supported Tasks and Leaderboards
multiple-choice question answering
Languages
English (en)
Dataset Structure
Data Instances
#intersentence
{'bias_type': 'race', 'context': 'Many people live in… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/stereoset.Omnimodal-Agent-SFT-2K
OmniGAIA: Omni-Modal General AI Assistant Benchmark
📄 Paper
•
💻 Code & Demo
•
🤗 Dataset & Model
•
📈 Leaderboard
This dataset contains omni-modal agent supervised fine-tuning (SFT) trajectories in the LlamaFactory SFT data format. You can directly follow LlamaFactory's instructions to fine-tune your omni-modal LLMs.OmniGAIA is a benchmark for Omni-Modal General AI Assistants that jointly reason over vision, audio, and language with external tools. It is… See the full description on the dataset page: https://huggingface.co/datasets/RUC-NLPIR/Omnimodal-Agent-SFT-2K.tatoeba_mt
Dataset Card for [Dataset Name]
Dataset Summary
The Tatoeba Translation Challenge is a multilingual data set of machine translation benchmarks derived from user-contributed translations collected by Tatoeba.org and provided as parallel corpus from OPUS. This dataset includes test and development data sorted by language pair. It includes test sets for hundreds of language pairs and is continuously updated. Please, check the version number tag to refer to the… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/tatoeba_mt.proofwriter_processed_OWAQuRatedPajama-260B
QuRatedPajama
Paper: QuRating: Selecting High-Quality Data for Training Language Models
A 260B token subset of cerebras/SlimPajama-627B, annotated by princeton-nlp/QuRater-1.3B with sequence-level quality ratings across 4 criteria:
Educational Value - e.g. the text includes clear explanations, step-by-step reasoning, or questions and answers
Facts & Trivia - how much factual and trivia knowledge the text contains, where specific facts and obscure trivia are preferred over more… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/QuRatedPajama-260B.McEvalMcEval benchmark data as described in the McEval Paper. Code for the evaluation can be found on Github as McEval.
MedThinkVQA
MedThinkVQA
MedThinkVQA is an expert-annotated benchmark for multi-image diagnostic reasoning in radiology. Unlike prior medical VQA benchmarks that typically contain at most one image per case, MedThinkVQA requires models to extract evidence from each image, integrate cross-view information, and perform differential-diagnosis reasoning.
Links
GitHub: https://github.com/benluwang/MedThinkVQA
Leaderboard: https://benluwang.github.io/MedThinkVQA/
Submission Guide:… See the full description on the dataset page: https://huggingface.co/datasets/bio-nlp-umass/MedThinkVQA.