datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ocr
OCR UV Scripts
Part of uv-scripts — self-contained UV scripts you run on Hugging Face Jobs in one command.
A model zoo of OCR scripts — one per model — that add a markdown column to an image dataset. Pick a model from the table below, point it at your dataset, and run it on a GPU with one command. A few recipes do structured extraction instead — image or text → JSON given a schema (see Structured extraction below). Two more companions sit alongside: pp-doclayout.py detects… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/ocr.transcription
Transcription
Scripts for transcribing — and diarizing — audio files using HF Buckets and Jobs.
Quick Start
Scripts run directly from their Hub URL — no clone or local checkout needed:
# 1. Download audio from Internet Archive straight into a bucket
hf jobs uv run \
-v hf://buckets/user/audio-files:/output \
https://huggingface.co/datasets/uv-scripts/transcription/raw/main/download-ia.py \
SUSPENSE /output
# 2. Transcribe — audio bucket in, transcript… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/transcription.ocr-demo
OCR demo: Food for Space Flight
Seven scanned pages from NASA's Food for Space Flight booklet, with headings,
columns, photographs, food lists and tables. This small dataset is an input for
trying OCR recipes and inspecting their results.
The images are PDF pages 3-9 (printed pages 2-8) of the original booklet.
The selection omits the reproduction disclaimer and dark cover. The complete
original PDF and a matching seven-page extract are in the
OCR demo Bucket.
Use as… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/ocr-demo.object-detection
Object Detection Dataset Scripts
8 scripts to create, convert, review, validate, inspect, diff, and sample object detection datasets on the Hub. Supports 6 bbox formats — no setup required.
Start from nothing: falcon-perception.py generates a first-pass detection dataset for any class you can name, zero-shot, with no labelling and no training. The other six then convert, check, and measure it.
This repository is inspired by panlabel
Quick Start
Convert bounding… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/object-detection.training
Streaming LLM Training with Unsloth
Train on massive datasets without downloading anything - data streams directly from the Hub.
🦥 Latin LLM Example
Teaches Qwen Latin using 1.47M texts from FineWeb-2, streamed directly from the Hub.
Blog post: Train on Massive Datasets Without Downloading
Quick Start
# Run on HF Jobs (recommended - 2x faster streaming)
hf jobs uv run latin-llm-streaming.py \
--flavor a100-large \
--timeout 2h \
--secrets HF_TOKEN \
--… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/training.classification
Classification Scripts
Text classification on HF Jobs — both directions:
Script
What it does
train-classifier.py
Fine-tune an encoder into a classifier (default: LFM2.5-Encoder-350M) and push it to the Hub
train-setfit.py
Few-shot train a classifier from 8-64 labels per class with SetFit — runs on CPU or GPU
classify-dataset.py
Zero-shot classify a dataset with an instruction LLM (SmolLM3 + vLLM, structured outputs)
classify-dataset-sglang.py
Zero-shot variant… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/classification.vlm-object-detection
VLM Object Detection
Instruction-prompted object detection with vision-language models via vLLM. Designed as a VLM-as-labeller primitive for bootstrapping object-detection datasets — give it a free-form prompt ("detect every photograph and illustration", "detect all PPE items", "detect every electronic component and identify its reference designator") and it returns bbox JSON ready for downstream labelling tools (Label Studio, FiftyOne, COCO conversion).
Sibling: uv-scripts/sam3… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/vlm-object-detection.sam3
SAM3 Vision Scripts
Detect and segment objects in images using Meta's SAM3 (Segment Anything Model 3) with text prompts. Process HuggingFace datasets with zero-shot detection and segmentation using natural language descriptions.
Script
What it does
Output
detect-objects.py
Object detection with bounding boxes
objects column with bbox, category, score
segment-objects.py
Pixel-level segmentation masks
Segmentation maps or per-instance masks
Browse results… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/sam3.build-atlas
Atlas Export
Embedding Atlas is an open-source library from Apple for creating interactive, browser-based visualizations of embedding spaces. It renders millions of data points with WebGPU acceleration, supports real-time search and filtering, and automatically generates cluster labels.
These scripts wrap Embedding Atlas to make it easy to go from a HuggingFace dataset to a deployed visualization. See open-library-atlas for a live example (2M books).
Scripts… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/build-atlas.embeddings
Embeddings
Generate embeddings for a Hugging Face dataset — text or images — with one command, on a
cloud GPU, no infra. The output lands back on the Hub as a new dataset (or, with the Lance
variant, as a searchable vector index you can query over hf:// without downloading).
There is one simple default and two variants; they are separate single-file scripts because
their dependencies (sentence-transformers vs vLLM vs Lance) are too different to share one env.
Script
Use it… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/embeddings.vllm
vLLM Inference Scripts
Ready-to-run UV scripts for GPU-accelerated inference using vLLM.
These scripts use UV's inline script metadata to automatically manage dependencies - just run with uv run and everything installs automatically!
📋 Available Scripts
vlm-classify.py
Vision Language Model (VLM) image classification with structured output constraints.
Features:
🖼️ Process images through state-of-the-art VLMs (Qwen2-VL)
🎯 Structured… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/vllm.dataset-creation
Dataset Creation Scripts
Ready-to-run scripts for creating Hugging Face datasets from local files.
Available Scripts
📄 pdf-to-dataset.py
Convert directories of PDF files into Hugging Face datasets.
Features:
📁 Uploads PDFs as dataset objects for flexible processing
🏷️ Automatic labeling from folder structure
🚀 Zero configuration - just point at your PDFs
📤 Direct upload to Hugging Face Hub
Usage:
# Basic usage
uv run pdf-to-dataset.py /path/to/pdfs… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/dataset-creation.hf-cli-jobs-uv-run-scripts
UV Script: dpo_training.py
Executed via hf jobs uv run on 2025-10-20 12:57:28 UTC
Run this script
hf jobs uv run dpo_training.py
Created with hf jobs
dataset-stats
Dataset Statistics
UV scripts for analyzing HuggingFace datasets using streaming mode.
Scripts
finepdfs-stats.py - Temporal Educational Quality Analysis
Analyze educational quality trends across CommonCrawl dumps using Polars streaming. Answers: "Is the web getting more educational over time?"
Features:
Polars streaming (no download of 300GB+ dataset)
Temporal analysis across 106 CommonCrawl dumps (2013-2025)
ASCII chart visualizations
Uploads… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/dataset-stats.openai-oss
🚀 OpenAI GPT OSS Models - Simple Generation Script
Generate synthetic datasets using OpenAI's GPT OSS models with transparent reasoning. Works on HuggingFace Jobs with L4 GPUs!
✅ Tested & Working
Successfully tested on HF Jobs with l4x4 flavor (4x L4 GPUs = 96GB total memory).
🚀 Getting Started with HF Jobs
First-time Setup (2 minutes)
Install HuggingFace CLI:
pip install huggingface-hub
Login to HuggingFace:
huggingface-cli login
(Enter your… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/openai-oss.video
Video
Scripts for captioning and temporally grounding video files using HF Buckets and Jobs.
What the output looks like — a frame from Joan Avoids a Cold (1947, Prelinger Archives) with the event Marlin-2B produced for that moment:
Quick Start
Scripts run directly from their Hub URL — no clone or local checkout needed:
# Caption every video in a bucket: dense scene captions + timestamped events
hf jobs uv run --image vllm/vllm-openai:latest --flavor a10g-small \… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/video.synthetic-data
CoT-Self-Instruct: High-Quality Synthetic Data Generation
Generate high-quality synthetic training data using Chain-of-Thought Self-Instruct methodology. This UV script implements the approach from "CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks" (2025).
🚀 Quick Start
# Install UV if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# Generate synthetic reasoning data
uv run cot-self-instruct.py \… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/synthetic-data.deduplication
Semantic Deduplication UV Script
Part of uv-scripts — self-contained UV scripts you run locally or on Hugging Face Jobs in one command.
Remove duplicate / near-duplicate text samples from a Hugging Face dataset by semantic similarity — clean training data and prevent train/test leakage. Uses SemHash with Model2Vec embeddings: CPU-optimized, no GPU required.
Quick start
# CPU is enough — run on Hugging Face Jobs
hf jobs uv run --flavor cpu-upgrade --secrets… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/deduplication.hf-cli-jobs-uv-run-scripts
UV Script: train_sorcerer.py
Executed via hf jobs uv run on 2026-03-19 10:33:43 UTC
Run this script
hf jobs uv run train_sorcerer.py
Created with hf jobs
marimo
Marimo UV Scripts
Marimo notebooks that work as both interactive tutorials and batch scripts.
What is this?
Marimo notebooks are pure Python files that can be:
Edited interactively with a reactive notebook interface
Run as scripts with uv run - same as any UV script
This makes them perfect for tutorials and educational content where you want users to explore step-by-step, but also run the whole thing as a batch job.
Available Scripts
Script
Description… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/marimo.transformers-inference
Transformers Continuous Batching Scripts
GPU inference scripts using transformers' native continuous batching (CB). No vLLM dependency required.
Why transformers CB?
Instant new model support - works with any model supported by transformers, including newly released architectures. No waiting for vLLM to add support.
No dependency headaches - no vLLM, flashinfer, or custom wheel indexes. Just transformers + accelerate.
Simple HF Jobs setup - no Docker image needed. Just… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/transformers-inference.doc-uv-scriptsiiif-tiles
IIIF Static Tiles from HF Storage Buckets
Generate IIIF Image API 3.0 Level 0 static tiles from images and serve them via Hugging Face Storage Buckets — no image server required.
Drop images into a bucket, run one command, and get deep-zoom viewing in any IIIF viewer (Mirador, Universal Viewer, OpenSeadragon).
Demo
View in Mirador — 6 pages from the Wellcome Collection, served entirely from an HF Storage Bucket.
How it works
Source images (bucket or… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/iiif-tiles.hf-cli-jobs-uv-run-scripts
UV Script: train_hf_jobs.py
Executed via hf jobs uv run on 2026-05-20 11:53:18 UTC
Run this script
hf jobs uv run train_hf_jobs.py
Created with hf jobs
uv-script-synced
uv-script-synced
A UV script for hfjobs.
Usage
hfjobs run ghcr.io/astral-sh/uv:python3.12 \
uv run https://huggingface.co/datasets/davanstrien/uv-script-synced/resolve/main/script.py \
<your-args>
Script Details
Script: script.py
Description: Template UV script for hfjobs.
Created with hfjobs
uv-script-test4
uv-script-test4
A UV script for hfjobs.
Usage
hfjobs run ghcr.io/astral-sh/uv:python3.12 \
uv run https://huggingface.co/datasets/davanstrien/uv-script-test4/resolve/main/script.py \
<your-args>
Script Details
Script: script.py
Description: Template UV script for hfjobs.
Dependencies:
datasets
tqdm
Scripts
dedupe-dataset.py
hfjobs run ghcr.io/astral-sh/uv:python3.12 \
uv run… See the full description on the dataset page: https://huggingface.co/datasets/davanstrien/uv-script-test4.gliner
GLiNER UV Scripts
Zero-shot named-entity recognition over Hugging Face datasets using GLiNER. Pass a list of entity types at runtime — no fine-tuning required.
Script
What it does
Output
extract-entities.py
Extract entities from a text column with a custom set of types
New entities column (list of {start, end, text, label, score})
Quick start
Run on any HF dataset with a text column. No setup — uv resolves dependencies inline.
# Local CPU (small… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/gliner.transformers-traininghf-cli-jobs-uv-run-scripts
UV Script: filter_cold_cases_california_mentions.py
Executed via hf jobs uv run on 2026-03-05 20:14:59 UTC
Run this script
hf jobs uv run filter_cold_cases_california_mentions.py
Created with hf jobs
uv-script-test
uv-script-test
A UV script for hfjobs.
Usage
hfjobs run ghcr.io/astral-sh/uv:python3.12 \
uv run https://huggingface.co/datasets/davanstrien/uv-script-test/resolve/main/script.py \
<your-args>
Script Details
Script: script.py
Description: Template UV script for hfjobs.
Dependencies:
datasets
tqdm
Created with hfjobs