datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
geospatialPACE-Water-QualityWHU-Building-Dataset
WHU Building Dataset
The WHU Building Dataset is a widely-used benchmark for building extraction from high-resolution aerial imagery. It contains aerial images at 0.3m resolution with pixel-level binary building masks.
Dataset Description
Property
Value
Resolution
0.3m ground sampling distance
Tile Size
512 x 512 pixels
Channels
3 (RGB)
Classes
2 (Background=0, Building=255)
Format
PNG
Splits
Split
Images
Masks… See the full description on the dataset page: https://huggingface.co/datasets/giswqs/WHU-Building-Dataset.dataset_gise_full_v1cqadupstack-gis
CQADupstackGisRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
CQADupStack: A Benchmark Data Set for Community Question-Answering Research
Task category
t2t
Domains
Written, Non-fiction
Reference
http://nlp.cis.unimelb.edu.au/resources/cqadupstack/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["CQADupstackGisRetrieval"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/cqadupstack-gis.gist1m
GIST1M Vector Search Dataset
1 million 960-dimensional vectors from the GIST descriptor dataset.
Dataset Details
Vectors: 1,000,000
Dimensions: 960
Queries: 1,000
Source: ANN Benchmarks
Shard Configurations
Config
Shards
Vectors/Shard
.indices
.vectors
shard_3
3
333,333
651MB
1.2GB
shard_5
5
200,000
391MB
732MB
shard_7
7
142,857
279MB
523MB
shard_10
10
100,000
195MB
366MB
DiskANN Parameters
R: 64, L: 100, Distance: L2… See the full description on the dataset page: https://huggingface.co/datasets/maknee/gist1m.task09_close_refrigerator_gist_0804
task09_close_refrigerator_gist_0804
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 9 입니다.
kiroaiseoul 네이밍(task09_close_refrigerator)에 맞춰 재배포한 사본이고, 원본 배포명은 task9-close-fridge-door_0804 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 0804 재수집본이고,
원본은 kiroaiseoul/task09_close_refrigerator_gist 에 있다.
Task: "Close the fridge door while holding the beaker"
Episodes: 1,049 / Frames: 306,210
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras: cam_high… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task09_close_refrigerator_gist_0804.evaluator-leaderboardharmbench-scenarios
HarmBench Scenarios
Safety-evaluation scenarios derived from the HarmBench behavior dataset, serialized as giskard Scenario objects (one JSON object per line).
Each row poses a harmful request the agent should refuse or safely decline, paired with an LLMJudge check (giskard.scan::judges/harmbench_safety.j2) that grades the agent response.
Format
Every line is a serialized giskard.checks.Scenario:
name — "HarmBench #<id>"
steps[].interacts[].inputs — the harmful… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/harmbench-scenarios.task06_pickup_beaker_and_move_to_refrigerator_gist
task06_pickup_beaker_and_move_to_refrigerator_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 6 입니다.
kiroaiseoul 네이밍(task06_pickup_beaker_and_move_to_refrigerator)에 맞춰 재배포한 사본이고, 원본 배포명은 task6-shake-beaker-and-move-to-fridge 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 원본이고,
재수집본은 kiroaiseoul/task06_pickup_beaker_and_move_to_refrigerator_gist_0804 에 있다. 베이스 결함률은 대체로 재수집본이 더 낮다.
Task: "Grab and shake the filled beaker, then move to the front of the fridge"
Episodes: 779… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task06_pickup_beaker_and_move_to_refrigerator_gist.puzzlescript-gists
PuzzleScript Human-Authored Games (Full Gist Corpus)
35,704 human-authored PuzzleScript games — the
complete source text of each — collected from public GitHub gists.
This is the full corpus: every distinct gist is kept, and each row is tagged
with its deduplication cluster so you can reduce to a unique set with a one-line
filter. The deduplication is reproducible from the shipped dedup_master.json +
dedup_master.py; non-vanilla PuzzleScript-Plus files are excluded (listed in… See the full description on the dataset page: https://huggingface.co/datasets/smearle/puzzlescript-gists.do-not-answer-scenarios
Do-Not-Answer Scenarios
Refusal-evaluation scenarios derived from the Do-Not-Answer dataset, serialized as
giskard Scenario objects (one JSON object per line).
Each row poses a question that a well-aligned assistant should decline, paired with a
Conformity check asserting the agent refused or safely declined.
Format
Every line is a serialized giskard.checks.Scenario:
name — "Do not answer #<id>"
steps[].interacts[].inputs — the question to send to the agent under… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/do-not-answer-scenarios.StereoTales
Multilingual Story-Generation Bias Samples
A multilingual evaluation dataset for probing demographic biases in LLM
story generation. Each sample instructs a model to write a ~200-word story
about a character carrying a given demographic attribute value (age, gender,
ethnicity, religion, disability status, immigration status, ...) placed into a
specific life scenario, with the goal of surfacing socio-economic and
demographic biases in the generated narratives.… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/StereoTales.GSA-PT-Qwen2-7B-Instruct-chunk4-chunk4-data
GSA-PT-Qwen2-7B-Instruct-chunk4-chunk4-data
This is the continue pretraining dataset used for training GSA (Gist Sparse Attention) models based on Qwen2-7B-Instruct with chunk size chunk4-chunk4.
Each sample is tokenized and formatted with GSA gist tokens for continue pretraining.
Paper
GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding
Related Model
yuzhenm/GSA-PT-Qwen2-7B-Instruct-chunk4-chunk4 — model trained on this dataset
task02_pickup_tubes_gist
task02_pickup_tubes_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 2 입니다.
kiroaiseoul 네이밍(task02_pickup_tubes)에 맞춰 재배포한 사본이고, 원본 배포명은 task2-pick-two-tubes-from-rack 입니다.
Task: "Pick two tubes out of the rack with both hands"
Episodes: 2,019 / Frames: 933,799
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras: cam_high, cam_left_wrist, cam_right_wrist (480x640x3, h264)
action / observation.state: 16D (Mobile AI — base 2 + left 7 + right 7)… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task02_pickup_tubes_gist.task05_tube_disposal_gist
task05_tube_disposal_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 5 입니다.
kiroaiseoul 네이밍(task05_tube_disposal)에 맞춰 재배포한 사본이고, 원본 배포명은 task5-dispose-empty-tubes-in-tray 입니다.
Task: "Dispose of the two empty tubes into the tube disposal tray"
Episodes: 2,001 / Frames: 580,038
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras: cam_high, cam_left_wrist, cam_right_wrist (480x640x3, h264)
action / observation.state: 16D (Mobile AI — base 2 + left 7 + right 7)… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task05_tube_disposal_gist.GISA
GISA: A Benchmark for General Information-Seeking Assistant
Authors: Yutao Zhu, Xingshuo Zhang, Maosen Zhang, Jiajie Jin, Liancheng Zhang, Xiaoshuai Song, Kangzhi Zhao, Wencong Zeng, Ruiming Tang, Han Li, Ji-Rong Wen, and Zhicheng Dou
Benchmark Highlights
GISA is a benchmark for General Information-Seeking Assistants with 373 human-crafted queries that reflect real-world information needs. It includes both stable and live subsets, four structured answer formats… See the full description on the dataset page: https://huggingface.co/datasets/RUC-NLPIR/GISA.task07_open_refrigerator_gist
task07_open_refrigerator_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 7 입니다.
kiroaiseoul 네이밍(task07_open_refrigerator)에 맞춰 재배포한 사본이고, 원본 배포명은 task7-open-fridge-door 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 원본이고,
재수집본은 kiroaiseoul/task07_open_refrigerator_gist_0804 에 있다. 베이스 결함률은 대체로 재수집본이 더 낮다.
Task: "Open the fridge door while holding the beaker"
Episodes: 1,000 / Frames: 215,930
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras: cam_high… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task07_open_refrigerator_gist.task09_close_refrigerator_gist
task09_close_refrigerator_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 9 입니다.
kiroaiseoul 네이밍(task09_close_refrigerator)에 맞춰 재배포한 사본이고, 원본 배포명은 task9-close-fridge-door 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 원본이고,
재수집본은 kiroaiseoul/task09_close_refrigerator_gist_0804 에 있다. 베이스 결함률은 대체로 재수집본이 더 낮다.
Task: "Close the fridge door while holding the beaker"
Episodes: 1,897 / Frames: 330,313
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras:… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task09_close_refrigerator_gist.NASA-OPERAtask08_takeout_and_put_beaker_gist_0804
task08_takeout_and_put_beaker_gist_0804
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 8 입니다.
kiroaiseoul 네이밍(task08_takeout_and_put_beaker)에 맞춰 재배포한 사본이고, 원본 배포명은 task8-swap-beakers-in-fridge_0804 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 0804 재수집본이고,
원본은 kiroaiseoul/task08_takeout_and_put_beaker_gist 에 있다.
Task: "Take out the stored beaker and put the held beaker into the fridge"
Episodes: 1,831 / Frames: 889,058
FPS: 30 · robot_type: mobileai_robot · codebase_version:… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task08_takeout_and_put_beaker_gist_0804.GSA-PT-Qwen2-7B-Instruct-chunk16-data
GSA-PT-Qwen2-7B-Instruct-chunk16-data
This is the continue pretraining dataset used for training GSA (Gist Sparse Attention) models based on Qwen2-7B-Instruct with chunk size chunk16.
Each sample is tokenized and formatted with GSA gist tokens for continue pretraining.
Paper
GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding
Related Model
yuzhenm/GSA-PT-Qwen2-7B-Instruct-chunk16 — model trained on this dataset
wikipedia-paragraph-embeddings-en-gist-complete
Dataset Summary
Paragraph embeddings for every article in English Wikipedia (not the Simple English version).
Based on wikimedia/wikipedia, 20231101.en.
Embeddings were generated with avsolatorio/GIST-small-Embedding-v0
and are quantized to int8.
You can load the data with the following:
from datasets import load_dataset
ds = load_dataset(path="Abrak/wikipedia-paragraph-embeddings-en-gist-complete", data-dir="20231101.en")
Dataset Structure
The structure of the… See the full description on the dataset page: https://huggingface.co/datasets/Abrak/wikipedia-paragraph-embeddings-en-gist-complete.GSA-PT-Llama-3.2-1B-chunk8-data
GSA-PT-Llama-3.2-1B-chunk8-data
This is the continue pretraining dataset used for training GSA (Gist Sparse Attention) models with chunk size chunk8.
Each sample is tokenized and formatted with GSA gist tokens for continued pretraining.
Paper
GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding
Related Models
yuzhenm/GSA-PT-Llama-3.2-1B-chunk8 — model trained on this dataset
phare
Phare Benchmark
Phare is a multilingual benchmark that measures LLM Safety across multiple categories of vulnerabilities, including hallucination, biases & stereotypes, harmful content, and jailbreaks.
Dataset Details
Dataset Description
This dataset contains the public set of samples of Phare Benchmark. These samples are split into multiple modules to assess LLM safety across various directions.
Each module is responsible for detecting vulnerabilities… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/phare.task08_takeout_and_put_beaker_gist
task08_takeout_and_put_beaker_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 8 입니다.
kiroaiseoul 네이밍(task08_takeout_and_put_beaker)에 맞춰 재배포한 사본이고, 원본 배포명은 task8-swap-beakers-in-fridge 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 원본이고,
재수집본은 kiroaiseoul/task08_takeout_and_put_beaker_gist_0804 에 있다. 베이스 결함률은 대체로 재수집본이 더 낮다.
Task: "Place the held beaker into the fridge and take out the beaker inside"
Episodes: 168 / Frames: 146,939
FPS: 30 · robot_type: mobileai_robot ·… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task08_takeout_and_put_beaker_gist.tmp_widowX_paxini_peginsertThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "right",
"total_episodes": 45,
"total_frames": 36283,
"total_tasks": 3,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:45"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/gistailab/tmp_widowX_paxini_peginsert.task01_move_to_tube_rack_gist
task01_move_to_tube_rack_gist
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 1 입니다.
kiroaiseoul 네이밍(task01_move_to_tube_rack)에 맞춰 재배포한 사본이고, 원본 배포명은 task1-move-to-tube-rack 입니다.
Task: "Move from the start point to face the tube rack"
Episodes: 2,635 / Frames: 664,247
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras: cam_high, cam_left_wrist, cam_right_wrist (480x640x3, av1)
action / observation.state: 16D (Mobile AI — base 2 + left 7 + right 7)… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task01_move_to_tube_rack_gist.EuroSAT_RGB
EuroSAT RGB
Dataset Description
EuroSAT is a dataset for land use and land cover (LULC) classification using Sentinel-2 satellite imagery. This version contains the RGB (visible spectrum) bands encoded as JPEG images at 64x64 pixel resolution.
The dataset covers 10 land use/land cover classes across 27,000 geo-referenced images from 34 European countries.
Source: https://zenodo.org/records/7711810
DOI: 10.5281/zenodo.7711810
License: MIT
Paper: EuroSAT: A Novel Dataset… See the full description on the dataset page: https://huggingface.co/datasets/giswqs/EuroSAT_RGB.task07_open_refrigerator_gist_0804
task07_open_refrigerator_gist_0804
GIST AI Lab이 수집한 11스테이지 실험실 프로토콜 데이터 중 stage 7 입니다.
kiroaiseoul 네이밍(task07_open_refrigerator)에 맞춰 재배포한 사본이고, 원본 배포명은 task7-open-fridge-door_0804 입니다.
이 스테이지는 GIST가 원본과 _0804 재수집본을 둘 다 배포했다. 이쪽이 0804 재수집본이고,
원본은 kiroaiseoul/task07_open_refrigerator_gist 에 있다.
Task: "Open the fridge door while holding the beaker"
Episodes: 1,010 / Frames: 159,490
FPS: 30 · robot_type: mobileai_robot · codebase_version: v3.0
Cameras: cam_high, cam_left_wrist… See the full description on the dataset page: https://huggingface.co/datasets/kiroaiseoul/task07_open_refrigerator_gist_0804.