datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SpotifyLyrics001SportsMetrics
SportsMetrics
Benchmark data to evaluate numerical reasoning and information fusion of LLMs.
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs Yebowen Hu, Kaiqiang Song, Sangwoo Cho, Xiaoyang Wang, Hassan Foroosh, Dong Yu, Fei Liu In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL'24), Bangkok, Thailand. Arxiv Paper
Usage
from datasets import load_dataset
def get_task(domain… See the full description on the dataset page: https://huggingface.co/datasets/huuuyeah/SportsMetrics.Myanmar-Written-Spoken-Parallel-Corpus
Myanmar Written-Spoken Parallel Corpus (MWSPC)
Dataset Description
Myanmar Written-Spoken Parallel Corpus (MWSPC) is a high-quality open-source dataset designed to bridge the gap between formal written Burmese and daily spoken Burmese. This dataset is crucial for building natural-sounding AI models that understand the linguistic nuances of the Myanmar language.
Curated by: Khant Sint Heinn (Kalix Louis)
Organization: DatarrX | ဒေတာ-အက်စ်
Language: Burmese… See the full description on the dataset page: https://huggingface.co/datasets/DatarrX/Myanmar-Written-Spoken-Parallel-Corpus.US-Presidents-Spoken-and-Written-SentencesUS Presidents' Spoken and Written Sentences
We obtained transcriptions of spoken language from the Miller Center of Public Affairs, University of Virginia, which covers transcriptions from George Washington to the present time. For the writing samples, we used ten complete books written by presidents, three of which we obtained from Project Gutenberg. To ensure the accuracy of calculations, all the pages that were not part of the main content were removed. Furthermore, multiple… See the full description on the dataset page: https://huggingface.co/datasets/Mina-Rajaei-Moghadam/US-Presidents-Spoken-and-Written-Sentences.SportsGenDataset and scripts for sports analyzing tasks proposed in research: When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives Yebowen Hu, Kaiqiang Song, Sangwoo Cho, Xiaoyang Wang, Wenlin Yao, Hassan Foroosh, Dong Yu, Fei Liu Accepted to main conference of EMNLP 2024, Miami, Florida, USA Arxiv Paper
Abstract
Reasoning is most powerful when an LLM accurately aggregates relevant information. We examine the critical role of information aggregation in… See the full description on the dataset page: https://huggingface.co/datasets/huuuyeah/SportsGen.Tutorbot-Spock-Bio-DatasetMock conversations between a student and a tutor to train a chatbot for educational purposes as suggested in the paper
CLASS Meet SPOCK: An Education Tutoring Chatbot based on Learning Science Principles.
Dataset generated from OpenStax Biology 2e textbook.
Problem, Subproblem, Hints, and Feedback is generated using the prompt.
Mock Conversations is generated using the prompt.
For any queries, contact Shashank Sonkar (ss164 AT rice dot edu)
If you use this model, please cite:
CLASS Meet… See the full description on the dataset page: https://huggingface.co/datasets/luffycodes/Tutorbot-Spock-Bio-Dataset.medgemma-4b-hematologic-oncology-blind-spots
MedGemma Blind Spots: Hematologic Oncology & CAR-T Immunotherapy
A 13-probe red-team evaluation showing how Google's MedGemma-4B confidently hallucinates clinical-trial statistics, fabricates non-existent treatment regimens, and misdiagnoses lymphoma in hematologic oncology — a clinical domain absent from its documented training data.
Summary
This dataset documents failures of Google's MedGemma-4B on hematologic oncology prompts — a clinical subspecialty absent… See the full description on the dataset page: https://huggingface.co/datasets/Mateenah/medgemma-4b-hematologic-oncology-blind-spots.fatima_blind_spot_challengeGot it. From now on I'll write everything inside Markdown blocks so you can copy easily.
Here is your full content entirely in Markdown:
# Fatima Fellowship 2026: Technical Challenge - Model Blind Spots
## 1. Model Overview
- **Model Tested:** [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) (Base Model)
- **Parameters:** 0.6B
- **Type:** Causal Language Model (Base / Pre-trained)
---
## 2. Methodology & Loading
To evaluate the model, I used **Google Colab** with a **T4… See the full description on the dataset page: https://huggingface.co/datasets/moseleydev/fatima_blind_spot_challenge.Blind_Spots_of_Frontier_Models
Qwen3-0.6B-Base — Blind Spots Dataset
Model Tested
Qwen/Qwen3-0.6B-Base
Type: Causal Language Model (base / pretraining only — not instruction-tuned)
Parameters: 0.6B (0.44B non-embedding)
Released: April–May 2025 by Alibaba Cloud's Qwen Team
Context Length: 32,768 tokens
How the Model Was Loaded
The model was loaded in a Google Colab T4 GPU notebook using HuggingFace transformers >= 4.51.0(required because the qwen3 architecture key was added in… See the full description on the dataset page: https://huggingface.co/datasets/Pidoxy/Blind_Spots_of_Frontier_Models.qwen2.5-3b-blind-spots
Qwen2.5-3B Factual Recall Blind Spots
Model Tested
Qwen/Qwen2.5-3B
A 3.09B parameter base causal language model, pretrained only.
How I Loaded the Model
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "Qwen/Qwen2.5-3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
Loaded on Google Colab.… See the full description on the dataset page: https://huggingface.co/datasets/suhanii23/qwen2.5-3b-blind-spots.smollm2-blind-spots
SmolLM2-1.7B Blind Spots Dataset
This dataset contains 10 diverse examples where the SmolLM2-1.7B base model makes incorrect predictions or demonstrates "blind spots".
Model Tested
Model: SmolLM2-1.7B
Parameters: 1.7 Billion
Type: Base (Pre-trained)
How to Load the Model
The model was loaded using the transformers library in Python.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HuggingFaceTB/SmolLM2-1.7B"
tokenizer =… See the full description on the dataset page: https://huggingface.co/datasets/Alhibb/smollm2-blind-spots.qwen3-base-blind-spots
Qwen3-0.6B-Base Blind Spots Dataset
Model tested: Qwen/Qwen3-0.6B-BaseParameters: 0.6B | Released: May 2025 | Type: Base (pretrained, not instruction-tuned)Tested by: Tito Osadebey | Platform: Google Colab (T4 GPU, free tier)
Overview
This dataset documents 10 diverse failure cases ("blind spots") identified in Qwen3-0.6B-Base through structured prompt testing. Failures span five categories: African geography and culture, temporal reasoning, arithmetic, logical reasoning… See the full description on the dataset page: https://huggingface.co/datasets/titoausten/qwen3-base-blind-spots.gemma-3-1b-pt-blind-spots
Gemma-3-1b-pt Blind Spots Dataset
Dataset Description
This dataset documents blind spots (systematic errors) found in
google/gemma-3-1b-pt,
a 1-billion-parameter pretrained base model (not instruction-tuned)
released by Google in March 2025 as part of the Gemma 3 family.
Each row contains:
Column
Description
id
Unique probe index
category
Type of reasoning tested
prompt
The input fed to the model (text-completion style)
expected_output
The… See the full description on the dataset page: https://huggingface.co/datasets/Junaid687/gemma-3-1b-pt-blind-spots.gemma-3-1b-pt-blind-spots
Blind Spots of google/gemma-3-1b-pt
Model Tested
Model: google/gemma-3-1b-ptParameters: 1BType: Pre-trained base language model (not instruction-tuned)Tested by: Toka-Tarek | Biotechnology graduate & Pharmacogenetics Lab Specialist
How I Loaded the Model
Tested on Google Colab (free T4 GPU, 16GB VRAM).
Note: torch.float16 caused numerical instability (NaN/inf errors)
on the T4 GPU, so torch.float32 was used instead for stable generation.
from huggingface_hub… See the full description on the dataset page: https://huggingface.co/datasets/Toka-Tarek/gemma-3-1b-pt-blind-spots.smollm2-blind-spots
Model Tested
HuggingFaceTB/SmolLM2-1.7B
How I loaded it
Used HuggingFace Transformers with AutoModelForCausalLM on Google Colab (T4 GPU, float16). Greedy decoding (do_sample=False) for reproducibility.
View Colab Notebook
Blind Spots Found
The model struggled with: multi-step arithmetic, low-resource languages (Yoruba), African geographic knowledge, code generation, and logical reasoning.
Fine-tuning Dataset Recommendation
GSM8K / MATH for… See the full description on the dataset page: https://huggingface.co/datasets/mirackchuks/smollm2-blind-spots.Qwen3.5-0.8B-Base-Blind-Spots
Qwen3.5-0.8B-Base Blind Spots
This dataset contains 10 short prompt-answer pairs designed to expose blind spots in Qwen/Qwen3.5-0.8B-Base. Each row includes the input prompt, the expected correct output, and the model's cleaned final output. The prompts focus on exact-answer failures such as constraint following, negation, ordering, string manipulation, unit reasoning, and simple logical reasoning.
Model tested
Model: Qwen/Qwen3.5-0.8B-Base
Model page:… See the full description on the dataset page: https://huggingface.co/datasets/tanjimriju/Qwen3.5-0.8B-Base-Blind-Spots.qwen3-blind-spots
Qwen3-0.6B-Base — Blind Spots Dataset
Model Tested
Model: Qwen/Qwen3-0.6B-BaseParameters: 0.6BArchitecture: Causal LM, decoder-only transformerTraining stage: Pre-training only — not instruction-tunedContext length: 32,768 tokens
Selected because it is a true base (pretrained) model in the 0.6B–6B parameter range, with a clear training provenance in the Qwen3 Technical Report.
How the Model Was Loaded
Loaded in Google Colab (T4 GPU, free tier) using… See the full description on the dataset page: https://huggingface.co/datasets/Syed-Furqan/qwen3-blind-spots.german-spoon-language
Spoon Language
This is a dataset of random german sentences from Tatoeba (accessed 23.07.25) mapped to their "Löffelsprache" / spoon language version.
The corresponding GitHub repository can be found here.
tiny-aya-blind-spots
Dataset: Tiny-Aya-Base Blind Spots
This dataset was created as part of a technical challenge to identify the blind spots of the models. It specifically targets CohereLabs/tiny-aya-base. The model is a 3.35B parameter multilingual base model released in early 2026.
Model Tested
Model: CohereLabs/tiny-aya-base
Parameters: 3.35 Billion
Modality: Text
How the Model was Loaded
The model was loaded using the transformers library on a Google Colab T4 GPU.… See the full description on the dataset page: https://huggingface.co/datasets/osamaahmed17/tiny-aya-blind-spots.Blind-Spot-Experiment-new-Dataset
Blind-Spot-Experiment-new-Dataset
Dataset Purpose
This dataset was created to investigate blind spots in a base foundation language model.
The experiment was conducted using the Transformers library from :contentReference[oaicite:1]{index=1}.
The evaluated model is :contentReference[oaicite:2]{index=2}.
Model link: https://huggingface.co/Qwen/Qwen3-0.6B
Implementation Details
The model was loaded and tested in Google Colab.
Code used to load the model:
from… See the full description on the dataset page: https://huggingface.co/datasets/Blessinggreat988/Blind-Spot-Experiment-new-Dataset.qwen3.5-2b-base-blind-spots
Qwen3.5-2B-Base Blind Spots Dataset
Overview
This dataset documents 10 blind spots (incorrect predictions) found in the
Qwen3.5-2B-Base model — a 2B parameter
base (pre-trained only) causal language model released by the Qwen team.
The errors span diverse categories including common sense reasoning, spatial reasoning,
negation understanding, character/word counting, science, geography, and logic.
Model Tested
Model: Qwen/Qwen3.5-2B-Base
Type: Causal… See the full description on the dataset page: https://huggingface.co/datasets/wilchesch/qwen3.5-2b-base-blind-spots.smollm3-blind-spots
Blind spots of SmolLM3-3B-Base on simple instructions
Dataset description
This dataset collects failure cases (“blind spots”) of the HuggingFaceTB/SmolLM3-3B-Base language model on very simple prompts.The prompts cover factual questions, arithmetic, comparisons, translation, and strict instruction-following such as “answer with one word only” or “yes or no only”.
Each row records:
the input prompt,
the expected output,
the model’s actual output, and
a short error type… See the full description on the dataset page: https://huggingface.co/datasets/yaqeenalradi/smollm3-blind-spots.qwen35-2b-base-blind-spots
Blind Spots of Qwen3.5-2B-Base
This dataset documents incorrect predictions from Qwen/Qwen3.5-2B-Base, a 2.21B-parameter pre-trained base model released February 2026.
How the Model Was Loaded
Loaded in Google Colab (free T4 GPU) with Hugging Face Transformers:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL_ID = "Qwen/Qwen3.5-2B-Base"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model =… See the full description on the dataset page: https://huggingface.co/datasets/Becky4382/qwen35-2b-base-blind-spots.