NimsW/nanbeige-3b-blindspots-eval

Technical Challenge: Blind Spots of Frontier Models

Author: Nimshi Wanniarachchi

This dataset documents systematic failure cases observed when evaluating a recent open-source base language model with approximately 0.6–6B parameters. The dataset contains 10 diverse evaluation examples, each including:

• —Input prompt
• —Expected output
• —Actual model output

Model tested

• —Model Name: Nanbeige4-3B-Base
• —Model Link: https://huggingface.co/Nanbeige/Nanbeige4-3B-Base
• —Parameters: ~3 Billion
• —Type: Base Model (Pre-trained only)

The model was loaded using a T4 GPU instance on Google Colab.

Code Used:

# Install required libraries
# !pip install -q transformers torch accelerate

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load Tokenizer and Model
tokenizer = AutoTokenizer.from_pretrained(
    'Nanbeige/Nanbeige4-3B-Base', 
    use_fast=False, 
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    'Nanbeige/Nanbeige4-3B-Base', 
    torch_dtype='auto', 
    device_map='auto', 
    trust_remote_code=True
)

# Example Prompt execution
prompt = "The most popular local e-commerce platforms operating primarily in Sri Lanka are"
inputs = tokenizer(prompt, return_tensors='pt').to('cuda')

# Generate completion with explicit mask and token limits
output_ids = model.generate(
    inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_new_tokens=150, 
    pad_token_id=tokenizer.eos_token_id
)

resp = tokenizer.decode(output_ids[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(resp)

Experiment Overview: Identified Blind Spots

By testing the model across diverse domains including localized knowledge, software engineering, specialized biological data, and safety constraints, several significant blind spots were observed.

1. Safety alignment failure

The model lacks a refusal mechanism and generates harmful content when prompted. Examples include:

• —Generating Cross-Site Scripting (XSS) payloads
• —Writing phishing emails
• —Producing realistic-looking credit card numbers

This suggests that the base model was not aligned using RLHF or safety fine-tuning datasets.

2. Latent reasoning trace leakage

When the model was asked to produce strictly formatted JSON, it inserted hidden reasoning tokens such as:

  <think> ... </think>

This indicates the model was likely trained on synthetic chain-of-thought data, causing it to externalize internal reasoning instead of producing concise outputs.

3. Formatting failures

The model struggled with strict formatting constraints. For example:

• —JSON responses contained additional commentary.
• —Mathematical answers included formatting artifacts such as: \boxed{...}

These artifacts suggest the model inherited formatting patterns from math and reasoning datasets but lacks strong instruction-following capabilities.

4. End-of-Sequence (EOS) failures

In some prompts the model failed to terminate properly and entered repetitive newline loops such as: \n\n\n\n\n\n

This behavior indicates weak sequence termination control in the base model.

5. Geographic hallucinations

The model produced confident but incorrect answers when asked about region-specific knowledge. For example, for the prompt 'give popular e-commerce platforms in Sri Lanka' outputs as Shopee, Lazada, Amazon were given instead of Daraz, Kapruka. This demonstrates limited exposure to localized datasets during pretraining.

6. Scientific Domain Hallucinations

The model hallucinated technical facts when asked about specialized biological signals in the question regarding frequency band of elephant seismic communication. This indicates the model struggles when operating outside its training distribution.

Discussion: Fixing the errors

What kind of dataset should the model be fine-tuned on?

• —To address these base model failures, two distinct fine-tuning phases are required.

• —The model needs a dataset of highly curated instruction-response pairs. This will teach the model zero-shot formatting (e.g., outputting strict JSON without <think> tags), enforce the generation of the EOS token to stop infinite loops, and correct sub-optimal coding paradigms.

• —To fix the critical safety vulnerabilities, the model must be fine-tuned on an adversarial alignment dataset containing harmful prompts mapped to polite but firm refusal responses.

How such a dataset could be assembled

A synthetic data pipeline could be created using a stronger model (such as GPT-4 or Gemini) to generate high-quality instruction-response pairs grounded in verified documentation and the sources could include software documentation, regional wikipedia data and curated technical datasets. Additionally, existing open datasets from huggingface can be used.

• —To establish proper conversational formatting, suppress reasoning bleed, and fix EOS token generation, a highly curated dataset of 10,000 to 50,000 examples is sufficient.