frontier-blindspots
frontier-model-blindspots
Application to the Fatimata Institute for Global Research, 2026-2027
Technical Challenge: Blind Spots of Frontier Models
Benjamin Maroyi | benjaminjibu@gmail.com
Overview
This dataset documents 10 blind spots of the base language model
Qwen/Qwen3-0.6B-Base
(0.6 billion parameters, released May 2025, Apache-2.0 license).
Each data point contains a prompt, the expected correct output, and the
model's actual output. The prompts were designed to be diverse across… See the full description on the dataset page: https://huggingface.co/datasets/JibuMaroyi/frontier-model-blindspots.Blind_Spots_of_Frontier_Models
Qwen3-0.6B-Base — Blind Spots Dataset
Model Tested
Qwen/Qwen3-0.6B-Base
Type: Causal Language Model (base / pretraining only — not instruction-tuned)
Parameters: 0.6B (0.44B non-embedding)
Released: April–May 2025 by Alibaba Cloud's Qwen Team
Context Length: 32,768 tokens
How the Model Was Loaded
The model was loaded in a Google Colab T4 GPU notebook using HuggingFace transformers >= 4.51.0(required because the qwen3 architecture key was added in… See the full description on the dataset page: https://huggingface.co/datasets/Pidoxy/Blind_Spots_of_Frontier_Models.nigerian-food-science-frontier-blindspotsModel Tested
I evaluated SmolLM2-1.7B. This is a compact base model (not fine-tuned for instruction following), which allows for the identification of raw data biases and "blind spots" in its pre-training corpus regarding specialized regional knowledge.
How I Loaded and Tested the Model
The model was loaded in a Google Colab environment using a T4 GPU. I used the transformers library with bfloat16 precision to ensure efficient memory management while maintaining numerical accuracy.… See the full description on the dataset page: https://huggingface.co/datasets/Isaac101111/nigerian-food-science-frontier-blindspots.Blind-Spots-of-Frontier-Models_Multihop-blindspots-qwen35
Qwen3.5-2B-Base — Multi-Hop Reasoning Blind Spots
An evaluation dataset probing 18 Knowledge Graph-style reasoning tasks on
Qwen/Qwen3.5-2B-Base, tested in its
raw base (pre-training) form with no external graph attached. The dataset covers
parametric memory (probes 1–10, no passage provided), standard grounded reasoning
(probes 11–15, source passage included), and advanced grounded reasoning
(probes 16–18, passage provided but requiring implicit inference or contradiction… See the full description on the dataset page: https://huggingface.co/datasets/chayma-rhaiem/Blind-Spots-of-Frontier-Models_Multihop-blindspots-qwen35.Blind_Spots_of_Frontier_Model-Qwen3.5_4B_base
Blind_Spots_of_Frontier_Model-Qwen3.5_4B_base
This is a submission to the Technical challenge: Blind Spots of Frontier Models for Fatima Fellowship 2026 application
Model Link: Qwen/Qwen3.5-4B-base
Code
Code link: https://gist.github.com/NachtSpyder04/8b1ccd5a29b37f7a82083c93b4f518c8
The model was loaded using the Hugging Face transformers library, and evaluated using deterministic decoding (temperature set to 0). I intentionally disabled sampling so that the outputs… See the full description on the dataset page: https://huggingface.co/datasets/NachtSpyder04/Blind_Spots_of_Frontier_Model-Qwen3.5_4B_base.blind-spots-for-frontier-models
Qwen Model Tasks For Fellowship
Note: view 'final_results_hf.json' for model results. For some reason hf is only showing 'test.json' in preview which doesnt contain models response.
Model Loading
The model was loaded using the transformers library from Hugging Face. The following code snippet was used to load the Qwen model and tokenizer:
from transformers import AutoModelForCausalLM, AutoTokenizer
# Define the model name
model_name = "Qwen/Qwen3-4B-Base"
# Load the… See the full description on the dataset page: https://huggingface.co/datasets/Izzi1/blind-spots-for-frontier-models.