datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
github-repos-pythonThe Github repository retrieval source for [code-rag-bench], containing all Python files from the entire GitHub dump (in github-repos)
megawika-report-generation
Dataset Card for MegaWika for Report Generation
Dataset Summary
MegaWika is a multi- and crosslingual text dataset containing 30 million Wikipedia passages with their scraped and cleaned web citations. The passages span
50 Wikipedias in 50 languages, and the articles in which the passages were originally embedded are included for convenience. Where a Wikipedia passage is in a
non-English language, an automated English translation is provided.
This dataset provides the… See the full description on the dataset page: https://huggingface.co/datasets/hltcoe/megawika-report-generation.github-reposThe entire dump of GitHub repositories.
my_dataset_repoasrs-aviation-reports
Dataset Card for ASRS Aviation Incident Reports
Dataset Summary
This dataset collects 47,723 aviation incident reports published in the Aviation Safety Reporting System (ASRS) database maintained by NASA.
Supported Tasks and Leaderboards
'summarization': Dataset can be used to train a model for abstractive and extractive summarization. The model performance is measured by how high the output summary's ROUGE score for a given narrative account of an aviation… See the full description on the dataset page: https://huggingface.co/datasets/elihoole/asrs-aviation-reports.RepoGraphThis repo contains cached trajectories of the paper RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph.
Code: https://github.com/ozyyshr/RepoGraph
mlb-daily-reportcompany-reports
Company Reports Dataset
Description
This dataset contains ESG (Environmental, Social, and Governance) sustainability reports from various companies. It includes data like company details, report categories, textual analysis of the reports, and more.
Dataset Structure
id: Unique identifier for each report entry.
document_category: Classification of the document (e.g., ESG sustainability report).
year: Publication year of the report.
company_name: Name of the… See the full description on the dataset page: https://huggingface.co/datasets/DataNeed/company-reports.fable5-repos
Fable 5 — All-Commits GitHub Repositories
A collection of 7,090 public GitHub repositories whose entire default-branch
history was written by Claude Fable 5 — every non-merge commit carries the
trailer:
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each repository is stored as a full .tar.gz archive including its complete
.git history, so you get every commit, message, and diff exactly as it
appears on GitHub. A manifest.jsonl / manifest.csv table describes every
repo… See the full description on the dataset page: https://huggingface.co/datasets/notune/fable5-repos.prompt-injection-repo-dataset
Prompt Injection Repository File Dataset
A labeled dataset for detecting prompt injection attacks in repository files — code, configs, READMEs, CI/CD workflows, and documentation that AI coding agents process as context.
What This Is (and Isn't)
This dataset targets a specific threat: indirect prompt injection via repository content. When AI coding agents (Claude Code, Cursor, Copilot, Gemini CLI) clone a repo, every file becomes part of the agent's context.… See the full description on the dataset page: https://huggingface.co/datasets/prodnull/prompt-injection-repo-dataset.McKinsey-Reportsmeta-llama/synthetic-data-kit
https://github.com/meta-llama/synthetic-data-kit
McKinsey reports
https://www.mckinsey.com/featured-insights/insights-store
measured-vs-reported
Measured vs reported — the empty table, on purpose
An honesty artefact, and deliberately close to empty. overlap.json would map
our measured Elo against third-party reported numbers, but its state reads "UNKNOWN — no verified
cross-platform Elo for our fleet models yet (honest, not fabricated)", cells is [], and the gate is
stated in the file: a reported cell is populated only when we hold a cited, attributed number for the same
model we measured. Until that holds, the table… See the full description on the dataset page: https://huggingface.co/datasets/csoai/measured-vs-reported.Changelog-Nightly-Repositoriesarchive of all the repositories incl. metadata of: https://changelog.com/nightly
will be used to train a spam classifier with spacy; hence the "text" column, but kept submeta in case this is useful for anyone else to re-format.
The Dataset is provided ""AS IS"" and ""AS AVAILABLE"" without warranty of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, title, or non-infringement.
The Provider disclaims all liability for… See the full description on the dataset page: https://huggingface.co/datasets/Edgerunners/Changelog-Nightly-Repositories.arxiv-software-repo-links
arXiv Software Repository Links
A dataset mapping arXiv papers (via DOI) to software repositories they reference or which are implementations of the work. Additionaly includes co-citation analysis and community clustering.
Implementation detection is conducted using the evamxb/dev-author-em-clf model from sci-soft-models
Quick Start
from datasets import load_dataset
# Load DOI-to-repo links
links = load_dataset("cometadata/arxiv-software-repo-links", "links")
#… See the full description on the dataset page: https://huggingface.co/datasets/cometadata/arxiv-software-repo-links.arxiv-software-repo-links-datacite-enrichment-format
arXiv Software Repository Links - DataCite Enrichment Format
A collection of metadata enrichments, formatted for DataCite's enrichment API, that add links between arXiv papers (via DOI) and the software repositories they reference or are supplemented by.
Quick Start
from datasets import load_dataset
ds = load_dataset("cometadata/arxiv-software-repo-links-datacite-enrichment-format")
Dataset Description
Each record is a DataCite-style enrichment instruction… See the full description on the dataset page: https://huggingface.co/datasets/cometadata/arxiv-software-repo-links-datacite-enrichment-format.repo-vul
RepoVul Dataset
Repo-level Vulnerability Detection Dataset for Fine-tuning Lightweight LLMs
Dataset Description
RepoVul is a comprehensive dataset for repository-level vulnerability detection. This subset contains training, validation, and test data for fine-tuning lightweight Large Language Models (LLMs) on vulnerability detection tasks.
Key Features
Task: Binary classification (vulnerable vs non-vulnerable code)
Languages: Java and Python
Format:… See the full description on the dataset page: https://huggingface.co/datasets/26PaperSubmission/repo-vul.annoy-datasync-license-reportthreat-intel-reports
ThreatIntel synthetic reports
32 synthetic English and Persian CTI notes for the ThreatIntel extraction demo. Seed 5.
Organization dataset and collection item are public. Live Gradio (AriaAICompany/threat-intel or alirezaaminzadeh/threat-intel) is created by scripts/publish.py after the daily Space-creation cap resets. This is fixture data (level 1). It does not prove operational extraction quality on real vendor reports. Reports are original laboratory text. They are not copies… See the full description on the dataset page: https://huggingface.co/datasets/AriaAICompany/threat-intel-reports.gov-report-qs-llama2-format
Government Report Question Answering Dataset in LLAMA2 Format
Dataset Description
This dataset is a LLAMA2 formatted dataset of the GovReport Dataset which is a report dataset, consisting of reports written by government research agencies including Congressional Research Service and US Government Accountability Office.
The purpose of creating this dataset is to provide those trying to finetune LLAMA2 and other LLM models for Government domain a formatted and easier to use… See the full description on the dataset page: https://huggingface.co/datasets/Kira-Floris/gov-report-qs-llama2-format.typed-decisions-repo-governance
typed-decisions: repo-governance family
A seed, not a corpus (n=1 as of first publish, 2026-09-19): real governance findings from a
private software monorepo, each run through TypeSafe's Jev (a "System One"
model — it returns typed choice / score / noul decisions, never free text or code) as part of
kotoba-lang/typed-decisions, a reproduction and
measurement of the Jev shape whose trained model is published at
com-kotobalabs/open-jev-deberta-v3-large.
What this is… See the full description on the dataset page: https://huggingface.co/datasets/com-kotobalabs/typed-decisions-repo-governance.gov_report_kgfake_tech_companies_market_reports10K_Report_Query_Toolrepochat-arena-preference-4k
Overview
This dataset contains leaderboard vote data on RepoChat collected from 2024/11/30 to 2025/02/03
For reproducing the leaderboards from this data, refer to the notebook.
License
User prompts are licensed under CC-BY-4.0, and model outputs are governed by the terms of use set by the respective model providers.
satellite-civilian-conflict-disruption-reporter-v1
Satellite Civilian Conflict Disruption Reporter v1
Dataset ID: ChrisRPL/satellite-civilian-conflict-disruption-reporter-v1
Status
This is a valid diagnostic reporter-schema dataset, not the current Blackline Atlas canonical model gate. The canonical compact calibration/gold dataset remains ChrisRPL/satellite-disruption-triage-aux-v2-2.
Use this dataset for future schema-simplification experiments only after respecting the mixed source licenses. Do not treat the associated… See the full description on the dataset page: https://huggingface.co/datasets/ChrisRPL/satellite-civilian-conflict-disruption-reporter-v1.chemical-regulatory-reporting-thresholds
US federal chemical regulatory reporting thresholds by program (CERCLA, EPCRA, CAA)
Canonical, always-current version: https://referencesource.org/chemical-regulatory-reporting-thresholds/
Machine-readable: https://referencesource.org/chemical-regulatory-reporting-thresholds/data.json — this mirror is a point-in-time copy.
Last verified: 2026-08-05
Stale after: 2027-08-05 (past this date, prefer the canonical copy —
it re-verifies on a cadence this snapshot does not)
Records:… See the full description on the dataset page: https://huggingface.co/datasets/referencesource/chemical-regulatory-reporting-thresholds.Questions_Reponsescase-reports
FindZebra case reports
A collection of 3344 case reports fetched from the PubMed API for the Fabry, Gaucher and Familial amyloid cardiomyopathy (FAC) diseases.
Articles are labelled using a text segmentation model described in "FindZebra online search delving into rare disease case reports using natural language processing".
sustainability-report-emissions-instruction-styleThe sustainability-report-emissions dataset converted into instruction-style JSONL format for direct consumption by SFTTrainer, axolotl, etc. The prompt consists of an instruction and text extracted from relevant pages of a sustainability report. The output is generated using the Mixtral-8x7B-v0.1 model and consists of a JSON string containing the scope 1, 2 and 3 emissions as well as the ids of pages containing this information. The dataset generation scripts are at this GitHub repo. An… See the full description on the dataset page: https://huggingface.co/datasets/nopperl/sustainability-report-emissions-instruction-style.maib-incident-reports-5K
MAIB Incident Type Dataset
The MAIB Incident Type Dataset contains short textual descriptions of marine accidents and incidents reported by the UK Marine Accident Investigation Branch (MAIB).Each record includes a short narrative and a corresponding incident-type label (e.g. Grounding / Stranding, Fire / Explosion, Collision).This dataset enables research and experimentation in maritime safety text classification and domain-specific NLP.
Dataset Details… See the full description on the dataset page: https://huggingface.co/datasets/baker-street/maib-incident-reports-5K.