datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
snapshotsgaianet-qdrant-snapshotToolpacks-Snapshots
Toolpacks-Snapshots
This repo is to take periodic snapshots of all the artefacts 📦 in Toolpacks: bin.ajam.dev
Toolpacks is the Largest Collection of Multi-Platform (Android|Linux|Windows) Pre-Compiled (+ UPXed) Static Binaries (incl. Build Scripts
The Sync Workflow actions are at: https://github.com/Azathothas/Toolpacks-Snapshots-Actions
PKG Managers
!#Simply point to this:
[+] ROOT… See the full description on the dataset page: https://huggingface.co/datasets/Azathothas/Toolpacks-Snapshots.arxiv-metadata-snapshot
Dataset Card for "arxiv-metadata-oai-snapshot"
More Information needed
This is a mirror of the metadata portion of the arXiv dataset.
The sync will take place weekly so may fall behind the original datasets slightly if there are more regular updates to the source dataset.
Metadata
This dataset is a mirror of the original ArXiv data. This dataset contains an entry for each paper, containing:
id: ArXiv ID (can be used to access the paper, see below)
submitter:… See the full description on the dataset page: https://huggingface.co/datasets/librarian-bots/arxiv-metadata-snapshot.kl3m-data-snapshot-20250324hub_weekly_snapshots
Sample code
To query the dataset to see which snapshots are observable, use e.g.:
import json
from datasets import load_dataset
from huggingface_hub import HfApi
REPO_ID = "hfmlsoc/hub_weekly_snapshots"
hf_api = HfApi()
all_files = hf_api.list_repo_files(repo_id=REPO_ID, repo_type="dataset")
repo_type_to_snapshots = {}
for repo_fpath in all_files:
if ".parquet" in repo_fpath:
repo_type = repo_fpath.split("/")[0]
repo_type_to_snapshots[repo_type] =… See the full description on the dataset page: https://huggingface.co/datasets/hfmlsoc/hub_weekly_snapshots.wu-crosscoder-snapshots
W_U snapshot caches
Pre-extracted unembedding-matrix (W_U) snapshots across pretraining
checkpoints, as flat {model_slug}_step{N}_wu.pt tensors (V, d_model) —
the training inputs for the trajectory crosscoders of
Learning to Read Out: Unembedding Dynamics in Language Model Pretraining.
These are a convenience cache: every tensor is re-extractable from the public
Apache-2.0 checkpoints (EleutherAI/pythia-*, allenai/OLMo-2-1124-7B) with
the code release… See the full description on the dataset page: https://huggingface.co/datasets/hematteo/wu-crosscoder-snapshots.openalex-snapshot-2026-05
OpenAlex Snapshot 2026-05 Processed Works and Citation Edges
This dataset repository contains a processed version of the OpenAlex Works snapshot from May 2026, together with derived citation-edge tables and selected downstream bibliometric datasets.
The repository is intended for large-scale scholarly metadata analysis, citation-network analysis, and bibliometric research. The main dataset displayed in the Hugging Face Dataset Viewer is light_openalex_works, which contains a… See the full description on the dataset page: https://huggingface.co/datasets/KennyChowww/openalex-snapshot-2026-05.depth_snapshotlive-facts-snapshot
Live Facts Snapshot
A daily snapshot of verifiable, post-training-cutoff world-state facts — the kind of
ground truth language models cannot know from training data — exported through
Dynamic Feed, a live, verifiable data API whose every response
is Ed25519-signed. One file per day (data/YYYY-MM-DD.jsonl), one fact per line, and
every row carries its own source, source_url and measured_at.
Facts covered per day:
tool
facts
upstream source
licence
software_version… See the full description on the dataset page: https://huggingface.co/datasets/dynamicfeed/live-facts-snapshot.WSM_v1_Latent_WSV_v1.5_for_Wan2.1OpenGrad-ToolPolicy-Canonical-v2-M0-snapshot
This is a provenance-preserving canonical candidate corpus. It is a pre-training canonical release, not an empirically selected or recommended training mixture.
What this release is
OpenGrad ToolPolicy Canonical v2 is a provenance-preserving, model-independent normalization of public tool-use and function-calling datasets. It was built to test one hypothesis with a measurement attached: that the tool-call collapse observed in the M0 SFT experiments on v1 was caused by the… See the full description on the dataset page: https://huggingface.co/datasets/arjhinety/OpenGrad-ToolPolicy-Canonical-v2-M0-snapshot.cia-world-factbook-snapshotsarxiv-metadata-oai-snapshot
Dataset Card for "arxiv-metadata-oai-snapshot"
More Information needed
arXiv-metadata-oai-snapshot
About Dataset
Dataset name: arXiv academic paper metadata
Data source: https://arxiv.org/
Submission date: 1986-04-25 ~ 2025-05-13 (data updated weekly)
Number of papers: 2,710,806 (as of 2025.5.14)
Fields included: title, author, abstract, journal information, DOI, etc.
Data format: json
Data volume: 4.58G
About ArXiv
For nearly 30 years, ArXiv has served the public and research communities by providing open access to scholarly articles, from the vast branches of… See the full description on the dataset page: https://huggingface.co/datasets/jackkuo/arXiv-metadata-oai-snapshot.WSM_v1_Latent_WSG_v1.6_for_Flux2data-snapshot
Dataset card for data-snapshot
This dataset was introduced in the paper Benchmarking Open-Source Layout Detection Models for Data Snapshot Extraction from Institutional Documents.
The source code for the benchmark and dataset extraction is available on GitHub: worldbank/ai4data.
Dataset summary
The data-snapshot dataset is an annotated corpus designed for the evaluation and development of models for extracting data snapshots from PDF documents. A data snapshot is… See the full description on the dataset page: https://huggingface.co/datasets/ai4data/data-snapshot.hubsift-snapshots
hubsift snapshots
Daily thin metadata snapshots (delta + weekly full) of public Hugging Face Hub
models/datasets, collected via the public Hub API for a personal change-monitoring
digest. Contains only public repo metadata (id, sha, likes, tags, ...).
If you are a repo owner and want your rows removed (e.g. after deleting or
privating your repo), open a discussion here and rows will be purged.
WSV_AVIS_train_results_v1.6WSM_v1_Latent_WSG_v1.7_for_Flux2arxiv-metadata-snapshot
Dataset Card for "arxiv-metadata-oai-snapshot"
More Information needed
This is a mirror of the metadata portion of the arXiv dataset.
The sync will take place weekly so may fall behind the original datasets slightly if there are more regular updates to the source dataset.
Metadata
This dataset is a mirror of the original ArXiv data. This dataset contains an entry for each paper, containing:
id: ArXiv ID (can be used to access the paper, see below)
submitter: Who… See the full description on the dataset page: https://huggingface.co/datasets/Rurouni-II/arxiv-metadata-snapshot.arxiv-metadata-oai-snapshot-t_a-tokenized
Dataset Card for "arxiv-metadata-oai-snapshot-t_a-tokenized"
More Information needed\
tokenized with Shitao/RetroMAE
MMT_snapshot
MMT Snapshot
Snapshot of MMT[^1] database.
Last update: May 2025
File structure
rso.csv - object information
data/ - directory containing tracks measurements in .parquet files
bulk_idx.txt - contains the index of last bulk file downloaded from the MMT database
Object information
mmt_id - Identification number of the object in MMT database.
norad_id - Norad identification number of the object.
name - Name of the object.
country - Country of origin.… See the full description on the dataset page: https://huggingface.co/datasets/kyselica/MMT_snapshot.WSM_v1_Latent_PT_WSG_v1.8_for_Flux2
WSM v1 Latent PT Cache - WSG v1.8 - FLUX.2
This repository stores the original modality-scoped PyTorch latent shards used by WSM v1 training.
PT shards are uploaded directly rather than wrapped in tar archives, so one modality can be replaced without republishing the full cache.
Layout
wsg_flux2_frame_latents_dense_multidataset/
shards/<modality>/shard_00000_of_00008.pt
manifests/
wsg_flux2_frame_latents_dense_multidataset/<modality>.sha256.json
Each modality… See the full description on the dataset page: https://huggingface.co/datasets/World-Snapshot/WSM_v1_Latent_PT_WSG_v1.8_for_Flux2.rareburden-commons-open-source-snapshots
RareBurden Commons open source snapshots
This public preservation projection contains exact, hash-bound source files
whose observed terms affirmatively permit redistribution. Each source retains
its own licence; license: other is intentionally used because the collection
is not governed by one uniform licence.
Included:
Orphadata July 2026 alignment and epidemiology files — CC BY 4.0.
Exact MONDO release assets — CC BY 4.0. The currently receipt-bound history
covers v2026-08-04… See the full description on the dataset page: https://huggingface.co/datasets/edithatogo/rareburden-commons-open-source-snapshots.us-warn-act-layoffs-point-in-time-snapshots
US WARN Act layoff notices — point-in-time (as-of) snapshot archive
25 daily vintages, 2026-08-30 → 2026-09-24.
1,067,077 total rows, 42 MB compressed. One new vintage every day, forever.
This is the same US WARN Act layoff dataset as
the daily mirror — except you can load it as it stood on a past
date, instead of only as it stands today.
from datasets import load_dataset
# the table exactly as it was published on 5 September 2026
past =… See the full description on the dataset page: https://huggingface.co/datasets/APProjects/us-warn-act-layoffs-point-in-time-snapshots.openregistry-snapshots
OpenRegistry Temporal Corporate Data — by Sophymarine
A longitudinal, point-in-time corpus of public company-registry
state. Each row is one company at one moment in time, stamped with
_retrieved_at. Weekly snapshots accumulate into a time series so you
can track corporate state changes — status flips (active → dissolved),
address moves, name changes, structural transitions — across six
jurisdictions.
This is not a static company directory. This is corporate state with a
time… See the full description on the dataset page: https://huggingface.co/datasets/Sophymarine/openregistry-snapshots.wbes-faceembedding-repo-snapshot
WBES-FaceEmbedding Repository Snapshot
This dataset repository stores a compressed workspace snapshot for the WBES-FaceEmbedding NeurIPS 2026 submission repository.
Snapshot Contents
The archive is intended to preserve the local research codebase state, including:
repository documentation and helper scripts
face_embedding/ modeling, robustness, checkpoint, and evaluation artifacts
faceBench/ and faceBench/latentVSpipeline/
datasets/FaceVerse/ processing, evaluation, and… See the full description on the dataset page: https://huggingface.co/datasets/Pampaj/wbes-faceembedding-repo-snapshot.7K-Challenge-snapshots
Dataset
This dataset contains the snapshots that will be used during the 7K Challenge, in the .xiidm format.
Snapshot Enrichment
The original public RTE7000 XIIDM snapshots were enriched using information recovered from the corresponding internal XIIDM files. Two types of data were added or corrected:
HVDC Generator Restoration
In the public dataset, HVDC interconnections are represented as generators whose identifier matches the HVDC line identifier. These… See the full description on the dataset page: https://huggingface.co/datasets/OpenSynth/7K-Challenge-snapshots.WSM_v1_Latent_WSG_v1.5_for_Flux2
WSM v1 Latent WSG v1.5 for FLUX.2
Precomputed FLUX.2 latent/cache files for WSM v1 dense structure training.
Files are uploaded as tar.zst split archives, each part below 15GB. Restore each directory with:
cat archives/<name>.tar.zst.part-* | tar --zstd -xf -
Archive groups:
wsg_flux2_caption_embeddings_all_datasets.tar.zst.part-*
wsg_flux2_frame_latents_dense_multidataset.tar.zst.part-*
xpose_flux2_frame_latents_sample10pct_2frames.tar.zst.part-*
See manifest.json for the… See the full description on the dataset page: https://huggingface.co/datasets/World-Snapshot/WSM_v1_Latent_WSG_v1.5_for_Flux2.