datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SpotifyLyrics001medgemma-4b-hematologic-oncology-blind-spots
MedGemma Blind Spots: Hematologic Oncology & CAR-T Immunotherapy
A 13-probe red-team evaluation showing how Google's MedGemma-4B confidently hallucinates clinical-trial statistics, fabricates non-existent treatment regimens, and misdiagnoses lymphoma in hematologic oncology — a clinical domain absent from its documented training data.
Summary
This dataset documents failures of Google's MedGemma-4B on hematologic oncology prompts — a clinical subspecialty absent… See the full description on the dataset page: https://huggingface.co/datasets/Mateenah/medgemma-4b-hematologic-oncology-blind-spots.Blind_Spots_of_Frontier_Models
Qwen3-0.6B-Base — Blind Spots Dataset
Model Tested
Qwen/Qwen3-0.6B-Base
Type: Causal Language Model (base / pretraining only — not instruction-tuned)
Parameters: 0.6B (0.44B non-embedding)
Released: April–May 2025 by Alibaba Cloud's Qwen Team
Context Length: 32,768 tokens
How the Model Was Loaded
The model was loaded in a Google Colab T4 GPU notebook using HuggingFace transformers >= 4.51.0(required because the qwen3 architecture key was added in… See the full description on the dataset page: https://huggingface.co/datasets/Pidoxy/Blind_Spots_of_Frontier_Models.fatima_blind_spot_challengeGot it. From now on I'll write everything inside Markdown blocks so you can copy easily.
Here is your full content entirely in Markdown:
# Fatima Fellowship 2026: Technical Challenge - Model Blind Spots
## 1. Model Overview
- **Model Tested:** [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) (Base Model)
- **Parameters:** 0.6B
- **Type:** Causal Language Model (Base / Pre-trained)
---
## 2. Methodology & Loading
To evaluate the model, I used **Google Colab** with a **T4… See the full description on the dataset page: https://huggingface.co/datasets/moseleydev/fatima_blind_spot_challenge.qwen2.5-3b-blind-spots
Qwen2.5-3B Factual Recall Blind Spots
Model Tested
Qwen/Qwen2.5-3B
A 3.09B parameter base causal language model, pretrained only.
How I Loaded the Model
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "Qwen/Qwen2.5-3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
Loaded on Google Colab.… See the full description on the dataset page: https://huggingface.co/datasets/suhanii23/qwen2.5-3b-blind-spots.smollm2-blind-spots
SmolLM2-1.7B Blind Spots Dataset
This dataset contains 10 diverse examples where the SmolLM2-1.7B base model makes incorrect predictions or demonstrates "blind spots".
Model Tested
Model: SmolLM2-1.7B
Parameters: 1.7 Billion
Type: Base (Pre-trained)
How to Load the Model
The model was loaded using the transformers library in Python.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HuggingFaceTB/SmolLM2-1.7B"
tokenizer =… See the full description on the dataset page: https://huggingface.co/datasets/Alhibb/smollm2-blind-spots.qwen3-base-blind-spots
Qwen3-0.6B-Base Blind Spots Dataset
Model tested: Qwen/Qwen3-0.6B-BaseParameters: 0.6B | Released: May 2025 | Type: Base (pretrained, not instruction-tuned)Tested by: Tito Osadebey | Platform: Google Colab (T4 GPU, free tier)
Overview
This dataset documents 10 diverse failure cases ("blind spots") identified in Qwen3-0.6B-Base through structured prompt testing. Failures span five categories: African geography and culture, temporal reasoning, arithmetic, logical reasoning… See the full description on the dataset page: https://huggingface.co/datasets/titoausten/qwen3-base-blind-spots.gemma-3-1b-pt-blind-spots
Gemma-3-1b-pt Blind Spots Dataset
Dataset Description
This dataset documents blind spots (systematic errors) found in
google/gemma-3-1b-pt,
a 1-billion-parameter pretrained base model (not instruction-tuned)
released by Google in March 2025 as part of the Gemma 3 family.
Each row contains:
Column
Description
id
Unique probe index
category
Type of reasoning tested
prompt
The input fed to the model (text-completion style)
expected_output
The… See the full description on the dataset page: https://huggingface.co/datasets/Junaid687/gemma-3-1b-pt-blind-spots.gemma-3-1b-pt-blind-spots
Blind Spots of google/gemma-3-1b-pt
Model Tested
Model: google/gemma-3-1b-ptParameters: 1BType: Pre-trained base language model (not instruction-tuned)Tested by: Toka-Tarek | Biotechnology graduate & Pharmacogenetics Lab Specialist
How I Loaded the Model
Tested on Google Colab (free T4 GPU, 16GB VRAM).
Note: torch.float16 caused numerical instability (NaN/inf errors)
on the T4 GPU, so torch.float32 was used instead for stable generation.
from huggingface_hub… See the full description on the dataset page: https://huggingface.co/datasets/Toka-Tarek/gemma-3-1b-pt-blind-spots.tiny-aya-blind-spots
Dataset: Tiny-Aya-Base Blind Spots
This dataset was created as part of a technical challenge to identify the blind spots of the models. It specifically targets CohereLabs/tiny-aya-base. The model is a 3.35B parameter multilingual base model released in early 2026.
Model Tested
Model: CohereLabs/tiny-aya-base
Parameters: 3.35 Billion
Modality: Text
How the Model was Loaded
The model was loaded using the transformers library on a Google Colab T4 GPU.… See the full description on the dataset page: https://huggingface.co/datasets/osamaahmed17/tiny-aya-blind-spots.smollm2-blind-spots
Model Tested
HuggingFaceTB/SmolLM2-1.7B
How I loaded it
Used HuggingFace Transformers with AutoModelForCausalLM on Google Colab (T4 GPU, float16). Greedy decoding (do_sample=False) for reproducibility.
View Colab Notebook
Blind Spots Found
The model struggled with: multi-step arithmetic, low-resource languages (Yoruba), African geographic knowledge, code generation, and logical reasoning.
Fine-tuning Dataset Recommendation
GSM8K / MATH for… See the full description on the dataset page: https://huggingface.co/datasets/mirackchuks/smollm2-blind-spots.Qwen3.5-0.8B-Base-Blind-Spots
Qwen3.5-0.8B-Base Blind Spots
This dataset contains 10 short prompt-answer pairs designed to expose blind spots in Qwen/Qwen3.5-0.8B-Base. Each row includes the input prompt, the expected correct output, and the model's cleaned final output. The prompts focus on exact-answer failures such as constraint following, negation, ordering, string manipulation, unit reasoning, and simple logical reasoning.
Model tested
Model: Qwen/Qwen3.5-0.8B-Base
Model page:… See the full description on the dataset page: https://huggingface.co/datasets/tanjimriju/Qwen3.5-0.8B-Base-Blind-Spots.qwen3-blind-spots
Qwen3-0.6B-Base — Blind Spots Dataset
Model Tested
Model: Qwen/Qwen3-0.6B-BaseParameters: 0.6BArchitecture: Causal LM, decoder-only transformerTraining stage: Pre-training only — not instruction-tunedContext length: 32,768 tokens
Selected because it is a true base (pretrained) model in the 0.6B–6B parameter range, with a clear training provenance in the Qwen3 Technical Report.
How the Model Was Loaded
Loaded in Google Colab (T4 GPU, free tier) using… See the full description on the dataset page: https://huggingface.co/datasets/Syed-Furqan/qwen3-blind-spots.Blind-Spot-Experiment-new-Dataset
Blind-Spot-Experiment-new-Dataset
Dataset Purpose
This dataset was created to investigate blind spots in a base foundation language model.
The experiment was conducted using the Transformers library from :contentReference[oaicite:1]{index=1}.
The evaluated model is :contentReference[oaicite:2]{index=2}.
Model link: https://huggingface.co/Qwen/Qwen3-0.6B
Implementation Details
The model was loaded and tested in Google Colab.
Code used to load the model:
from… See the full description on the dataset page: https://huggingface.co/datasets/Blessinggreat988/Blind-Spot-Experiment-new-Dataset.smollm3-blind-spots
Blind spots of SmolLM3-3B-Base on simple instructions
Dataset description
This dataset collects failure cases (“blind spots”) of the HuggingFaceTB/SmolLM3-3B-Base language model on very simple prompts.The prompts cover factual questions, arithmetic, comparisons, translation, and strict instruction-following such as “answer with one word only” or “yes or no only”.
Each row records:
the input prompt,
the expected output,
the model’s actual output, and
a short error type… See the full description on the dataset page: https://huggingface.co/datasets/yaqeenalradi/smollm3-blind-spots.qwen3.5-2b-base-blind-spots
Qwen3.5-2B-Base Blind Spots Dataset
Overview
This dataset documents 10 blind spots (incorrect predictions) found in the
Qwen3.5-2B-Base model — a 2B parameter
base (pre-trained only) causal language model released by the Qwen team.
The errors span diverse categories including common sense reasoning, spatial reasoning,
negation understanding, character/word counting, science, geography, and logic.
Model Tested
Model: Qwen/Qwen3.5-2B-Base
Type: Causal… See the full description on the dataset page: https://huggingface.co/datasets/wilchesch/qwen3.5-2b-base-blind-spots.qwen35-2b-base-blind-spots
Blind Spots of Qwen3.5-2B-Base
This dataset documents incorrect predictions from Qwen/Qwen3.5-2B-Base, a 2.21B-parameter pre-trained base model released February 2026.
How the Model Was Loaded
Loaded in Google Colab (free T4 GPU) with Hugging Face Transformers:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL_ID = "Qwen/Qwen3.5-2B-Base"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model =… See the full description on the dataset page: https://huggingface.co/datasets/Becky4382/qwen35-2b-base-blind-spots.