datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SWE-bench-extraNote: This dataset has an improved and significantly larger successor: SWE-rebench.
Dataset Summary
SWE-bench Extra is a dataset that can be used to train or evaluate agentic systems specializing in resolving GitHub issues. It is based on the methodology used to build SWE-bench benchmark and includes 6,415 Issue-Pull Request pairs sourced from 1,988 Python repositories.
Dataset Description
The SWE-bench Extra dataset supports the development of software engineering agents… See the full description on the dataset page: https://huggingface.co/datasets/nebius/SWE-bench-extra.commonsense-qaExtractBench
ExtractBench
Quick links: [🌐 Website] [📜 Paper] [💻 Code]
Given a document and a schema, a system returns structured data with evidence. The input is a full document, born-digital or scanned, and a schema written by the user. The output is a schema-valid JSON object, with the source page and a bounding box for each value as evidence. It must return correct, exhaustive values (including repeated records), correctly use null for absent information, and ground each extracted… See the full description on the dataset page: https://huggingface.co/datasets/llamaindex/ExtractBench.qascsiqachunk-extMultitask-National-Speech-Corpus-v1-extendtweet_sentiment_extraction
TweetSentimentExtractionClassification
An MTEB dataset
Massive Text Embedding Benchmark
Task category
t2c
Domains
Social, Written
Reference
https://www.kaggle.com/competitions/tweet-sentiment-extraction/overview
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["TweetSentimentExtractionClassification"])
evaluator = mteb.MTEB(task)
model =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/tweet_sentiment_extraction.wikidata-extraction
Wikidata Extraction
This dataset contains all RDF triples extracted from the latest Wikidata, converted from the N-Triples format to Parquet.
The data originates from Wikidata, a free and open knowledge base that acts as central storage for structured data used by Wikipedia and other Wikimedia projects. The source file is the "truthy" N-Triples dump (latest-truthy.nt.bz2), which contains only the current, non-deprecated statements.
The code to extract this data is available at… See the full description on the dataset page: https://huggingface.co/datasets/piebro/wikidata-extraction.vi-books_extractionll books_text_final/ | wc -l
=> 13968 text
After filter (not vi, too small)
wc -l vi-books.jsonl
=> 12832 text
=> !!! Cần lọc truyện từ dtv ra !!!
xzcat dtv_ebooks_details.jsonl.xz | wc -l
# => 13486
ll books_text_final/ | grep dtv_ebooks_ | wc -l
# => 3535 từ dtv
lấy các whitelist categories tại dtv_categories_whitelist.jsonl
lần theo từng ebook một trong dtv_ebooks_details.jsonl.xz trường {"cat": "Self Help - Khởi nghiệp", "cat_url":… See the full description on the dataset page: https://huggingface.co/datasets/tiendung/vi-books_extraction.sciqpiqasudoku-extreme
Hardest Sudoku Puzzle Dataset V2
This dataset contains a mixture of easy and very hard Sudoku puzzles collected from the Sudoku community.
Dataset Composition
Sources
tdoku benchmarks
enjoysudoku
Easy Puzzles (1.1M)
puzzles0_kaggle
puzzles1_unbiased
puzzles2_17_clue
Hard Puzzles (3.1M)
puzzles3_magictour_top1465
puzzles4_forum_hardest_1905
puzzles6_forum_hardest_1106
ph_2010/01_file1.txt
Dataset Characteristics
All… See the full description on the dataset page: https://huggingface.co/datasets/sapientinc/sudoku-extreme.SWE-bench-extra-500sec-contracts-financial-extraction-instructions
S&P 500 SEC Financial Extraction Instructions
Dataset Summary
7,683 instruction-tuning examples for training LLMs to extract structured financial data from SEC filings. Covers two filing types across S&P 500 companies:
Split
Examples
Filing Type
Description
train
3,430
Exhibit 10 + DEF 14A
Positive examples with validated outputs
corrective
4,253
Exhibit 10 + DEF 14A
Corrective, rescued, and negative examples
Exhibit 10 — Material Contracts (2… See the full description on the dataset page: https://huggingface.co/datasets/TheTokenFactory/sec-contracts-financial-extraction-instructions.smdd-benchThe tasks compressed folder is tasks_4_4_final.tar.gz.
cmu-arctic-xvectors-extractedstarcoder2data-extras
StarCoder2 Extras
This is the dataset of extra sources (besides Stack v2 code data) used to train the StarCoder2 family of models. It contains the following subsets:
Kaggle (kaggle): Kaggle notebooks from Meta-Kaggle-Code dataset, converted to scripts and prefixed with information on the Kaggle datasets used in the notebook. The file headers have a similar format to Jupyter Structured but the code content is only one single script.
StackOverflow (stackoverflow): stackoverflow… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/starcoder2data-extras.GUI-Net-1M-extendedhle-extractchronos_datasets_extra
Chronos datasets
Time series datasets used for training and evaluation of the Chronos forecasting models.
This repository contains scripts for constructing datasets that cannot be hosted in the main Chronos datasets repository due to license restrictions.
Usage
Datasets can be loaded using the 🤗 datasets library
import datasets
ds = datasets.load_dataset("autogluon/chronos_datasets_extra", "ETTh", split="train", trust_remote_code=True)
ds.set_format("numpy") #… See the full description on the dataset page: https://huggingface.co/datasets/autogluon/chronos_datasets_extra.extracel_waveforms
Spike waveform shards (derived from IBL dandiset 000409)
A curated collection of per-spike multichannel waveform windows extracted from NWB assets in the DANDI dandiset 000409 (International Brain Laboratory, IBL). This repository contains the extractor and uploader used to produce Parquet shards; this README documents the derived dataset, its provenance, format, and how to reproduce it.
Short description
Parquet shards of spike waveform windows (channels × timesteps).… See the full description on the dataset page: https://huggingface.co/datasets/rokaijano/extracel_waveforms.logiqaCADBench-Extended-Multimodal-Dataset
Dataset Card
Dataset Description
CADBench Extended Multimodal Dataset is an independently produced public extension for multimodal CAD reconstruction research. It contains 100 CAD samples with clean and perturbed meshes, STEP/STL/OBJ/GLB representations, single-view and four-view renders, PBR images, bilingual descriptions, prompt variants, QA, geometry metadata, grading signals, and manually reviewed visual semantics.
Tasks: image-to-text, text-to-image… See the full description on the dataset page: https://huggingface.co/datasets/LianeMarilin/CADBench-Extended-Multimodal-Dataset.function_calling_extended
Trelis Function Calling Dataset
UPDATE: As of Dec 5th 2023, there is a v3 of this dataset now available from here.
Allows models to be fine-tuned for function-calling.
The dataset is human generated and does not make use of Llama 2 or OpenAI!
Contains 59 training and 17 test rows
Based on eight functions: search_bing, search_arxiv, save_chat, read_json_file, list_files, get_current_weather, delete_file, clear_chat
Access this dataset by purchasing a license HERE.
Alternatively… See the full description on the dataset page: https://huggingface.co/datasets/Trelis/function_calling_extended.relbench-v2-extra
RelBench v2 extras
Everything RelBench v2 added on top of v1: the v2-only
databases (with their tasks), and the v2-only tasks defined on the v1 databases. The v1
databases and tasks stay in
stanford-star/relbench.
Same self-describing manifest layout (plain parquet + manifest.yaml):
<dataset>/
manifest.yaml # tables, primary keys, foreign-key graph, val/test timestamps
schema.svg # ER diagram
db/*.parquet # relational… See the full description on the dataset page: https://huggingface.co/datasets/stanford-star/relbench-v2-extra.NSRDB_extractPublic domain data extracted from National Solar Radiation Database: https://nsrdb.nrel.gov/data-viewer
APEX-v1-extended
APEX-v1-extended
The AI Productivity Index (APEX) is a benchmark from Mercor for assessing whether frontier models are capable of performing economically valuable tasks across four jobs: investment banking associate, management consultant, big law associate, and primary care physician (MD).
APEX-v1-extended doubles the heldout evaluation set from n=200 to n=400, with increased complexity and variety. On average, tasks take over two-and-a-half hours for seasoned professionals to… See the full description on the dataset page: https://huggingface.co/datasets/mercor/APEX-v1-extended.nemotron_extra_sft_parquetomni_extract_bench
Omni Extract Bench
We weren’t satisfied with the current benchmarking options for extraction. They were biased, didn’t use realistic data and were hard to audit. Our view is that an extraction benchmark should do two things:
Help customers choose the right vendor; and
Give engineers a way to diagnose what’s actually going wrong in a given model.
That’s why we built OmniExtractBench.
OmniExtractBench is a comprehensive structured extraction benchmark, developed by Datalab.… See the full description on the dataset page: https://huggingface.co/datasets/datalab-to/omni_extract_bench.