WaveOAK/smollm2-nigerian-cultural-blindspots
Project: Blind Spots of Frontier Models (SmolLM2-1.7B) Model Tested SmolLM2-1.7B Loading Code Python from transformers import AutoModelForCausalLM, AutoTokenizer import torch model_id = "HuggingFaceTB/SmolLM2-1.7B" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.bfloat16) Discussion of Errors The model struggles with cultural semiotics and West African regional dialects (Pidgin). It tends to… See the full description on the dataset page: https://huggingface.co/datasets/WaveOAK/smollm2-nigerian-cultural-blindspots.
Project: Blind Spots of Frontier Models (SmolLM2-1.7B) Model Tested SmolLM2-1.7B
Loading Code Python from transformers import AutoModelForCausalLM, AutoTokenizer import torch
modelid = "HuggingFaceTB/SmolLM2-1.7B" tokenizer = AutoTokenizer.frompretrained(modelid) model = AutoModelForCausalLM.frompretrained(modelid, devicemap="auto", torch_dtype=torch.bfloat16) Discussion of Errors The model struggles with cultural semiotics and West African regional dialects (Pidgin). It tends to interpret idiomatic expressions literally (e.g., "I dey come" as physical movement rather than a temporal promise). It also lacks domain-specific knowledge regarding African timber species and their structural properties, often defaulting to generalized Western forestry data.
Fine-Tuning Strategy Dataset: I would assemble a "Regional Context & Idiom Mapping" dataset of ~10,000 pairs.
Method: Scrape localized forums and use native-speaker verification to create "Standard English to Pidgin" and "Cultural Logic" mappings.
Size: A high-quality set of 5k-10k examples using PEFT/LoRA would likely correct these specific reasoning blind spots without requiring massive compute.