datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
beaver-query
Dataset Card for beaver-query
Homepage and leaderboard |
Github repository |
Paper
Beaver is a holistic framework for evaluating performance on complex, private‑enterprise text‑to‑SQL tasks.
This repository includes questions and corresponding annotations. We reserve a portion of the full question set as a private, hidden test set.
Each sample contains:
id: ID of the question
category: one of real, complex query, domain-specific query, domain-specific complex query.
real indicates… See the full description on the dataset page: https://huggingface.co/datasets/beaverbench/beaver-query.wikipedia-multilingual-synthetic-ir-query
wikipedia-multilingual-synthetic-ir-query
This dataset contains multilingual Wikipedia-derived synthetic query-document pairs for information retrieval training.
It was created with the query-crafter-multilingual model, which generates search-like queries from Wikipedia text.
The current release contains two different retrieval settings:
short_doc: pairs of (query, short document)
long_doc: pairs of (query, long document)
These two subsets are not generated in the same way… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/wikipedia-multilingual-synthetic-ir-query.bing_coronavirus_query_set
Dataset Card for BingCoronavirusQuerySet
Dataset Summary
Please note that you can specify the start and end date of the data. You can get start and end dates from here: https://github.com/microsoft/BingCoronavirusQuerySet/tree/master/data/2020
example:
load_dataset("bing_coronavirus_query_set", queries_by="state", start_date="2020-09-01", end_date="2020-09-30")
You can also load the data by country by using queries_by="country".
Supported Tasks and… See the full description on the dataset page: https://huggingface.co/datasets/microsoft/bing_coronavirus_query_set.wildchat-category-query-expanded-n6vsc2022-test-query-2fpstargeted-query-set-processeddare-query-activations
DARE Activation Cache
Midlayer residual-stream activations for training documents and behavior
queries in GaloisTheory123/dare-data.
Training Document Activations
File
Model
Description
custom_sft_gold/activations.parquet
jrosseruk/dare-domain-splits/train/gold
custom_sft_gold adapter (train/gold) activations
Columns: doc_idx, split, final_token_activation, mean_activation
Query Activations
File
Model
Description… See the full description on the dataset page: https://huggingface.co/datasets/GaloisTheory123/dare-query-activations.task674_google_wellformed_query_sentence_generation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task674_google_wellformed_query_sentence_generation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task674_google_wellformed_query_sentence_generation.wildchat-reattributed-query-triplewildchat-category-query-expanded-n3wildchat-reconstructed-query-distinctwildchat-reattributed-query-distinctimage-query-vienemotron-terminal-data_querying
nemotron-terminal-data_querying
Per-source partition of nvidia/Nemotron-Terminal-Corpus,
filtered to source == "data_querying". The difficulty column preserves the original
easy / medium / mixed split (na for the dataset_adapters/* files, which
did not carry a difficulty label).
Partitioning scheme:
adapters_{code,math,swe} — rows from dataset_adapters/{code,math,swe}.parquet
{skill} (e.g. debugging, security, …) — rows from
synthetic_tasks/skill_based/{easy,medium… See the full description on the dataset page: https://huggingface.co/datasets/laion/nemotron-terminal-data_querying.medium_5000_data_queryingwildchat-reattributed-query-uniquegooaq-query-arctic-pooledeasy_5000_data_queryinghtml-query-text-HtmlRAG
html-query-text-HtmlRAG
Warning: This dataset is under development and its content is subject to change!
This dataset is a processed and cleaned version of the zstanjj/HtmlRAG-train dataset. It has been specifically prepared for task of HTML cleaning.
🚀 Supported Tasks
This dataset is primarily designed for:
HTML Cleaning: Training models to take the messy html as input and generate the cleaned_html or cleaned_text as output.
Question Answering: Training models to… See the full description on the dataset page: https://huggingface.co/datasets/williambrach/html-query-text-HtmlRAG.medium_5000_data_querying_fixedeval_piper_hpt_solo_camframe_queryfreq30This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "piper",
"total_episodes": 1,
"total_frames": 241,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/yukk1/eval_piper_hpt_solo_camframe_queryfreq30.medium_5000-data_querying_n100k1medi-data-mteb-covid-bing-query-gpt4-avs_triplets
MEDI+MTEBcls+COVq dataset
This dataset was used in the paper GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning. Refer to https://arxiv.org/abs/2402.16829 for details.
The code for generating the data is available at https://github.com/avsolatorio/GISTEmbed.
Citation
@article{solatorio2024gistembed,
title={GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning},
author={Aivin V. Solatorio}… See the full description on the dataset page: https://huggingface.co/datasets/avsolatorio/medi-data-mteb-covid-bing-query-gpt4-avs_triplets.task675_google_wellformed_query_sentence_generation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task675_google_wellformed_query_sentence_generation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task675_google_wellformed_query_sentence_generation.dart-math-pool-gsm8k-query-info
[!NOTE]
This dataset is the synthesis information of queries from the GSM8K training set,
such as the numbers of raw/correct samples of each synthesis job.
Usually used with dart-math-pool-gsm8k.
🎯 DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving
📝 Paper@arXiv | 🤗 Datasets&Models@HF | 🐱 Code@GitHub
🐦 Thread@X(Twitter) | 🐶 中文博客@知乎 | 📊 Leaderboard@PapersWithCode | 📑 BibTeX
Datasets: DART-Math
DART-Math datasets are the… See the full description on the dataset page: https://huggingface.co/datasets/hkust-nlp/dart-math-pool-gsm8k-query-info.msmarco-query-mxbai-pooled
Embedpress: mixedbread large on the Msmarco queries dataset
This is the query portion of the MsMarco dataset, embedded with Mixedbread AI's mixedbread-ai/mxbai-embed-large-v1.
For each document, we take the first 510 tokens (the model's max length -2 special tokens), and embed it, not using any instructions. Because the model was trained using Matryoshka Representation Learning, these embeddings can safely be truncated.
These are mainly useful for large-scale knowledge distillation.… See the full description on the dataset page: https://huggingface.co/datasets/stephantulkens/msmarco-query-mxbai-pooled.UDM_extract_query_sample_sol
UDM extract → query (minimal-edit, GPT-5.6-Sol)
A search query for each of 99,997 English mathematical documents, produced by GPT-5.6-Sol under a
prompt that asks it to edit rather than rewrite: if the page contains a question somebody
actually asked, that question is the query, copied with as few changes as possible.
This is the companion to an earlier release built from the same documents with a rewriting
prompt. The two differ in one respect only — how the query is phrased —… See the full description on the dataset page: https://huggingface.co/datasets/oklenAI/UDM_extract_query_sample_sol.clirmatrix_query_docText-to-sql-query-generationquery-to-dataset-viewer-descriptions
Queries to Hugging Face Hub Datasets Views
Dataset Summary
This dataset consists of synthetically generated queries for datasets mapped to datasets on the Hugging Face Hub. The queries map to a datasets viewer API response summary of the dataset. The goal of the dataset is to train sentence transformer and ColBERT style models to map between a query from a user and a dataset without relying on a dataset card, i.e., using information in the dataset itself.
Quick… See the full description on the dataset page: https://huggingface.co/datasets/davanstrien/query-to-dataset-viewer-descriptions.