datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fatima-fellowship-blindspots
Dataset Card: Tiny-Aya-Base Amharic Evaluation Blindspots
Dataset Description
This dataset provides a targeted, interpretable checklist of reasoning failures and blindspots discovered in the CohereLabs/tiny-aya-base model when evaluated on Amharic language tasks across arithmetic, logic, science, history, and geography domains.
As models scale, evaluating their cross-lingual reasoning capabilities requires moving beyond aggregate metrics. This repository adopts a… See the full description on the dataset page: https://huggingface.co/datasets/MYGBM/fatima-fellowship-blindspots.fatima-fellowship-naruto-blip-captionsFatima_Fellowship
Bengali Quote Error Analysis Dataset
This repository contains an error-analysis dataset for Bengali quote understanding.
Model outputs: Fatima_Fellowship.csv
The goal is to document diverse model mistakes and propose a fine-tuning direction.
Model Tested
Model: Qwen/Qwen3.5-0.8B
Framework: transformers
Prompt format: chat template with a system instruction and one few-shot example
How the Model Was Loaded
The following code was used in the notebook:
from… See the full description on the dataset page: https://huggingface.co/datasets/SumitKumarKar01/Fatima_Fellowship.qwen-3-5-blindspots-fatima-fellowship
Qwen3.5-4B-Base Blindspots Dataset
A collection of 11 prompts where Qwen/Qwen3.5-4B-Base produces incorrect, incomplete, or degenerate outputs. Each row records the input prompt, the raw model output, the extracted model response, the expected (correct) response, and a label for the failure category.
Dataset Summary
Field
Value
Model tested
Qwen/Qwen3.5-4B-Base
Number of examples
11
Columns
prompt, raw_output, thinking, output, expected_output, blindspot… See the full description on the dataset page: https://huggingface.co/datasets/dawaawawa/qwen-3-5-blindspots-fatima-fellowship.fatima-fellowship-challenge-blind-spots-of-ministral-3-3b-base
Blind Spots of mistralai/Ministral-3-3B-Base-2512
Fatima Fellowship, Technical Challenge Submission
1. Model Selection
Model: mistralai/Ministral-3-3B-Base-2512
Parameters: ~3.8B (3.4B language model + 0.4B vision encoder)
Type: Base pre-trained, explicitly NOT fine-tuned for instructions or chat
Released: December 2025 | License: Apache 2.0
This model was selected because its model card explicitly states it is "the base pre-trained version, not fine-tuned for… See the full description on the dataset page: https://huggingface.co/datasets/Faiyaj/fatima-fellowship-challenge-blind-spots-of-ministral-3-3b-base.Fatima_Fellowship_2026fatimaFellowship2026Model: https://huggingface.co/CohereLabs/tiny-aya-base
Model Loading
I loaded the model using the transformers' library pipeline helper function and kept the default model parameters provided by the model card. The only thing I would occasionally
change was max_new_tokens, to adjust it so that it makes sense with the prompt I'm giving. I also chose to keep temperature low for more determinism:
from transformers import pipeline
def predict(
input: str,
max_new_tokens=50… See the full description on the dataset page: https://huggingface.co/datasets/akhanafer/fatimaFellowship2026.fatima-fellowship-naruto-blip-captionsv4fatima-fellowship-datasetfatima-fellowship-naruto-blip-captionsv2fatima-fellowship-nanbeige-blindspots_2
Blind Spots of Frontier Models: Nanbeige4-3B-Base
This dataset was developed for the Fatima Fellowship 2026 Technical Challenge. It explores the natural vulnerabilities and structural blind spots of a raw, unaligned 3B parameter base model.
1. Model & Environment
Model Tested: Nanbeige/Nanbeige4-3B-Base (Released Feb 2026)
Loading Method: Google Colab (T4 GPU), bfloat16 precision, transformers library. Greedy decoding (do_sample=False) was used to isolate deterministic… See the full description on the dataset page: https://huggingface.co/datasets/hassanql/fatima-fellowship-nanbeige-blindspots_2.fatima-fellowship-nanbeige-blindspotsfatima-fellowship-qwen-blindspots
Dataset: Global South Blind Spots - Qwen 2.5 (Bangladesh & Autism Context)
This dataset was created as part of the Fatima Fellowship Technical Challenge. It identifies 10 diverse "blind spots" in the Qwen/Qwen2.5-1.5B base model, specifically focusing on the socio-technical and cultural context of Bangladesh and Neurodivergence (Autism).
🔍 Project Overview
Base models are often trained on Western-centric data, leading to significant hallucinations and cultural erasures… See the full description on the dataset page: https://huggingface.co/datasets/ZuhairHossain/fatima-fellowship-qwen-blindspots.QA_Low_Resource_FatimaFellowship
Model Experimentation and Analysis
1. Model Experimentations
I chose the task of Q&A, branching into two categories: general and specific. I tested the output, i.e., the LLM's response against the expected output. I have created the dataset in a CSV file. The model used is : https://huggingface.co/Andron00e/YetAnother_Open-Llama-3B-LoRA-OpenOrca
2. "Blind spots"
In this dataset, model is not able to predict well on very-specific information, like dates or years… See the full description on the dataset page: https://huggingface.co/datasets/KushieBoi/QA_Low_Resource_FatimaFellowship.Dimeji-Fatima-Fellowship
Evaluating Causal and Reasoning Blind Spots in Base LLMs
Model Tested
Model Name: Qwen/Qwen2.5-3B
Model Link: https://huggingface.co/Qwen/Qwen2.5-3B
How the Model Was Loaded
The model was evaluated using a Google Colab instance with a free T4 GPU. To accommodate the VRAM constraints of the hardware while maintaining inference fidelity, I utilized the transformers library alongside bitsandbytes to load the model using 8-bit quantization.
Here is the exact code… See the full description on the dataset page: https://huggingface.co/datasets/Dimeji12/Dimeji-Fatima-Fellowship.Fatima-fellowship
Blind Spots of Qwen3.5-0.8B-Base
Model Tested
This dataset evaluates the weaknesses of the base language model Qwen3.5-0.8B-Base developed by the Qwen Team.
Model Link: Qwen/Qwen3.5-0.8B-Base on Hugging Face
Parameters: ~0.8 billion
Type: Causal Language Model (Base Model)
Because it is a base model, it has not been fine-tuned for instruction following, providing a clear opportunity to analyze the kinds of mistakes smaller foundation models make when performing… See the full description on the dataset page: https://huggingface.co/datasets/faisalshahid03/Fatima-fellowship.fatima-fellowship-qwen35-eval
Technical Challenge: Blind Spots of Qwen3.5-4B
Model Tested
Qwen/Qwen3.5-4B. This is a foundation model released in early 2026.
Loading Methodology
The model was loaded using the transformers library on a Google Colab T4 GPU instance.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Qwen/Qwen3.5-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model =… See the full description on the dataset page: https://huggingface.co/datasets/Alaa-Abdelkader/fatima-fellowship-qwen35-eval.