datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
taco-datasetsThis repo consists of the datasets used for the TaCo paper. There are four datasets:
Multilingual Alpaca-52K GPT-4 dataset
Multilingual Dolly-15K GPT-4 dataset
TaCo dataset
Multilingual Vicuna Benchmark dataset
We translated the first three datasets using Google Cloud Translation.
The TaCo dataset is created by using the TaCo approach as described in our paper, combining the Alpaca-52K and Dolly-15K datasets.
If you would like to create the TaCo dataset for a specific language, you can… See the full description on the dataset page: https://huggingface.co/datasets/saillab/taco-datasets.TACO-verified
Introduction
This dataset contains verified solutions from the TACO dataset's training set. Solutions that fail to pass all the test cases are removed. Problems with no correct solution are also removed.
The solutions were executed on Intel E5-2620 v3 CPUs with the execution timeout set to 10 seconds.
Statistics in the training set
Dataset
# Problems
# Solutions
TACO
25443
1468722
TACO-verified
12898
1043251
Correct Ratio
50.69 %
71.03 %… See the full description on the dataset page: https://huggingface.co/datasets/likaixin/TACO-verified.TACO-Cobalt
TACO-Cobalt
Paper | GitHub
TACO-Cobalt is a cleaned version of that TACO code generation dataset crawled from programming competition websites.
Due to noise in the crawled data, we start from the TACO-verified subset and apply further cleaning to obtain 6,103 valid tasks, each with at least 8 test cases.
For each task, we sort the test cases’ difficulty by the possibility of Qwen2.5-Coder-7B-Instruct passing them out of 16 attempts.
We select the four easiest test cases as the… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/TACO-Cobalt.tacodemoTACO-Cobalt-PTB
TACO-Cobalt-PTB
Paper | GitHub
TACO-Cobalt is the perturbed version of the validation set in TACO-Cobalt to analyze in-context reward hacking behaviors of LLMs in code generation contexts.
For each coding problem, we randomly select two public tests (x_1, y_1) and (x_2, y_2) with distinct outputs (y_1 != y_2).
Then, we exchange their expected outputs and turn them into two perturbed tests (x_1, y_2) and (x_2, y_1), which are impossible for any correct program to pass.
If all public… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/TACO-Cobalt-PTB.taco-cobalt-qwen3-4b-filtered-v1
TACO-Cobalt Qwen3-4B Filtered v1
This is a filtered copy of osunlp/TACO-Cobalt for code-generation training
experiments. We ran Qwen/Qwen3-4B-Instruct-2507 three times per problem and excluded problems
where at least two of the three completions passed the tests. In short, this
keeps problems with 0/3 or 1/3 correct probe completions and filters out
problems with 2/3 or 3/3 correct completions.
Files
train.jsonl: filtered training split.
validation.jsonl: filtered… See the full description on the dataset page: https://huggingface.co/datasets/agurung/taco-cobalt-qwen3-4b-filtered-v1.tacos_summarytacostacotaco3tacoTACO_demotacoTacotaco2main🌮🌮🌮🌮🌮
taco3_deduptaco3_aug_dedup