datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
cqadupstack-gis
CQADupstackGisRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
CQADupStack: A Benchmark Data Set for Community Question-Answering Research
Task category
t2t
Domains
Written, Non-fiction
Reference
http://nlp.cis.unimelb.edu.au/resources/cqadupstack/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["CQADupstackGisRetrieval"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/cqadupstack-gis.evaluator-leaderboardharmbench-scenarios
HarmBench Scenarios
Safety-evaluation scenarios derived from the HarmBench behavior dataset, serialized as giskard Scenario objects (one JSON object per line).
Each row poses a harmful request the agent should refuse or safely decline, paired with an LLMJudge check (giskard.scan::judges/harmbench_safety.j2) that grades the agent response.
Format
Every line is a serialized giskard.checks.Scenario:
name — "HarmBench #<id>"
steps[].interacts[].inputs — the harmful… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/harmbench-scenarios.puzzlescript-gists
PuzzleScript Human-Authored Games (Full Gist Corpus)
35,704 human-authored PuzzleScript games — the
complete source text of each — collected from public GitHub gists.
This is the full corpus: every distinct gist is kept, and each row is tagged
with its deduplication cluster so you can reduce to a unique set with a one-line
filter. The deduplication is reproducible from the shipped dedup_master.json +
dedup_master.py; non-vanilla PuzzleScript-Plus files are excluded (listed in… See the full description on the dataset page: https://huggingface.co/datasets/smearle/puzzlescript-gists.do-not-answer-scenarios
Do-Not-Answer Scenarios
Refusal-evaluation scenarios derived from the Do-Not-Answer dataset, serialized as
giskard Scenario objects (one JSON object per line).
Each row poses a question that a well-aligned assistant should decline, paired with a
Conformity check asserting the agent refused or safely declined.
Format
Every line is a serialized giskard.checks.Scenario:
name — "Do not answer #<id>"
steps[].interacts[].inputs — the question to send to the agent under… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/do-not-answer-scenarios.StereoTales
Multilingual Story-Generation Bias Samples
A multilingual evaluation dataset for probing demographic biases in LLM
story generation. Each sample instructs a model to write a ~200-word story
about a character carrying a given demographic attribute value (age, gender,
ethnicity, religion, disability status, immigration status, ...) placed into a
specific life scenario, with the goal of surfacing socio-economic and
demographic biases in the generated narratives.… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/StereoTales.wikipedia-paragraph-embeddings-en-gist-complete
Dataset Summary
Paragraph embeddings for every article in English Wikipedia (not the Simple English version).
Based on wikimedia/wikipedia, 20231101.en.
Embeddings were generated with avsolatorio/GIST-small-Embedding-v0
and are quantized to int8.
You can load the data with the following:
from datasets import load_dataset
ds = load_dataset(path="Abrak/wikipedia-paragraph-embeddings-en-gist-complete", data-dir="20231101.en")
Dataset Structure
The structure of the… See the full description on the dataset page: https://huggingface.co/datasets/Abrak/wikipedia-paragraph-embeddings-en-gist-complete.phare
Phare Benchmark
Phare is a multilingual benchmark that measures LLM Safety across multiple categories of vulnerabilities, including hallucination, biases & stereotypes, harmful content, and jailbreaks.
Dataset Details
Dataset Description
This dataset contains the public set of samples of Phare Benchmark. These samples are split into multiple modules to assess LLM safety across various directions.
Each module is responsible for detecting vulnerabilities… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/phare.EuroSAT_RGB
EuroSAT RGB
Dataset Description
EuroSAT is a dataset for land use and land cover (LULC) classification using Sentinel-2 satellite imagery. This version contains the RGB (visible spectrum) bands encoded as JPEG images at 64x64 pixel resolution.
The dataset covers 10 land use/land cover classes across 27,000 geo-referenced images from 34 European countries.
Source: https://zenodo.org/records/7711810
DOI: 10.5281/zenodo.7711810
License: MIT
Paper: EuroSAT: A Novel Dataset… See the full description on the dataset page: https://huggingface.co/datasets/giswqs/EuroSAT_RGB.neodatasetThis dataset contains User Story text and Story Points
Dataset Structure
Features
Column name
Type
Description
idproject
int64
Project id
issuekey
int64
User Story key
created
string
Created at
title
string
User Story text
description
string
User Story description
storypoints
int64
Story Point
NEODATASET
NEODATASET is a dataset containing title and description in natural language of User Stories and their Story Points extracted… See the full description on the dataset page: https://huggingface.co/datasets/giseldo/neodataset.EuroSAT_MS
EuroSAT Multispectral (All 13 Sentinel-2 Bands)
Dataset Description
EuroSAT is a dataset for land use and land cover (LULC) classification using Sentinel-2 satellite imagery. This version contains all 13 Sentinel-2 spectral bands stored as uint16 arrays at 64x64 pixel resolution.
The dataset covers 10 land use/land cover classes across 26,998 geo-referenced images from 34 European countries.
Source: https://zenodo.org/records/7711810
DOI: 10.5281/zenodo.7711810
License:… See the full description on the dataset page: https://huggingface.co/datasets/giswqs/EuroSAT_MS.gistbench
GISTBench — Groundedness & Interest Specificity Test Bench
GISTBench evaluates how well LLMs understand users from their engagement history. Given a user's interactions with content (videos, articles, books, etc.), the benchmark measures whether an LLM can extract meaningful interests, ground them in evidence, and cite specific relevant items.
Dataset Details
Rows: 4,214,059 engagements
Users: ~1,000 anonymized users
Format: Parquet
Schema
Column
Type… See the full description on the dataset page: https://huggingface.co/datasets/facebook/gistbench.multiwoz-chatspatial-datasetsgist-960-euclidean
Dataset Overview
dataset: gist-960-euclidean
Metadata
Creation Time: 2025-01-07 11:03:48+0000
Update Time: 2025-01-07 11:04:44+0000
Source: https://github.com/erikbern/ann-benchmarks
Task: N/A
Train Samples: N/A
Test Samples: N/A
License: DISCLAIMER AND LICENSE NOTICE:
This dataset is intended for benchmarking and research purposes only.
The source data used in this dataset retains its original license and copyright. Users must comply with the respective licenses of… See the full description on the dataset page: https://huggingface.co/datasets/open-vdb/gist-960-euclidean.realharm
RealHarm
RealHarm is a collection of harmful real-world interactions with AI agents.
Dataset Details
Dataset Description
RealHarm contains harmful samples, categorized among 10 harm categories. A complete taxonomy has been proposed along with the dataset and is described in the RealHarm paper. Each sample has an associated safe version, for which we rewrote the agent answer to make it harmless.
This dataset provides researchers and developers with authentic… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/realharm.ScienceQA
Dataset Card Creation Guide
Dataset Summary
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
Supported Tasks and Leaderboards
Multi-modal Multiple Choice
Languages
English
Dataset Structure
Data Instances
Explore more samples here.
{'image': Image,
'question': 'Which of these states is farthest north?',
'choices': ['West Virginia', 'Louisiana', 'Arizona', 'Oklahoma'],
'answer': 0… See the full description on the dataset page: https://huggingface.co/datasets/Gisiyuan/ScienceQA.distillation_training_gist_medi_mtebgiskard-hub-demo-retailplaces-census-tract-data-gis-friendly-format-2024
PLACES: Census Tract Data (GIS Friendly Format), 2024 release
Description
This dataset contains model-based census tract level estimates in GIS-friendly format. PLACES covers the entire United States—50 states and the District of Columbia—at county, place, census tract, and ZIP Code Tabulation Area levels. It provides information uniformly on this large scale for local areas at four geographic levels. Estimates were provided by the Centers for Disease Control and… See the full description on the dataset page: https://huggingface.co/datasets/HHS-Official/places-census-tract-data-gis-friendly-format-2024.GISA-leaderboardRinkoQAhcitqarealperformance
Dataset Card for RealPerformance
Website: RealPerformance
Blog: Giskard Blog
Point of Contact: Giskard AI
License: MIT License
Dataset Summary
RealPerformance is a comprehensive dataset designed for preference learning and safety evaluation of conversational AI systems. It provides pairs of chosen (safe) and rejected (unsafe) responses to help train models to distinguish between appropriate and problematic AI behaviors in real-world scenarios.
The dataset includes:… See the full description on the dataset page: https://huggingface.co/datasets/giskardai/realperformance.giskard-hub-demo-healthcaretest-giskard-reportbeir-cqadupstack-gis
CQADupstackGisRetrieval — BEIR, unified schema
A normalised copy of the dataset behind the mteb task CQADupstackGisRetrieval, one of the tasks of the BEIR benchmark as mteb defines it (a member of the aggregate task CQADupstackRetrieval). Same queries, documents
and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset
in this collection.
Source
mteb/cqadupstack-gis @ 5003b3064772 (the revision pinned in mteb)… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/beir-cqadupstack-gis.dataset-machado-assis
Dataset Machado de Assis
Corpus textual composto por obras de Machado de Assis, escritor brasileiro considerado o maior nome da literatura nacional e fundador da Academia Brasileira de Letras.
Conteúdo
O arquivo principal corpus.txt reúne algumas obras em prosa de Machado de Assis em formato de texto simples, incluindo o romance Dom Casmurro.
Estatísticas do corpus
Métrica
Valor
Linhas
2.318
Palavras
545.926
Caracteres
3.258.923… See the full description on the dataset page: https://huggingface.co/datasets/giseldo/dataset-machado-assis.cqadupstack-gis-fa
Dataset Summary
CQADupstack-gis-Fa is a Persian (Farsi) dataset developed for the Retrieval task, with a focus on duplicate question detection in community question-answering (CQA) platforms. This dataset is a translated version of the "GIS" (Geographic Information Systems) StackExchange subforum from the English CQADupstack collection and is part of the FaMTEB benchmark under the BEIR-Fa suite.
Language(s): Persian (Farsi)
Task(s): Retrieval (Duplicate Question Retrieval)… See the full description on the dataset page: https://huggingface.co/datasets/MCINext/cqadupstack-gis-fa.africa-sahel-biomass-by-ach-gis4tech
Sahel Biomass by ACH-GIS4Tech | Africa (original)
Size category: 10K<n<100K - Formats: parquet - Sector: climate_environment - Engineered by Electric Sheep Africa
TL;DR
This dataset is part of the Electric Sheep Africa catalog on Hugging Face. It is indexed for African data discovery with standardized metadata, loading guidance, provenance notes, and analyst-oriented context.
What This Dataset Covers
Public datasets help analysts inspect… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-sahel-biomass-by-ach-gis4tech.