datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
global-mmlu-rephrased
global_mmlu (rephrased for base-model evaluation)
Global MMLU knowledge-MCQA items rewritten from question format into completion/cloze format for base (non-instruction-tuned) language model evaluation.
Base (non-instruction-tuned) language models often can't follow question-style
prompts like "What is the capital of Turkey?" -- that phrasing is suited to
instruction-tuned models. Each item here has been rewritten into a natural
completion prefix (e.g. "The capital of Turkey is… See the full description on the dataset page: https://huggingface.co/datasets/base-model-evals/global-mmlu-rephrased.belebele-rephrased
belebele (rephrased for base-model evaluation)
Belebele reading-comprehension items rewritten from question format into completion/cloze format for base (non-instruction-tuned) language model evaluation.
Base (non-instruction-tuned) language models often can't follow question-style
prompts like "What is the capital of Turkey?" -- that phrasing is suited to
instruction-tuned models. Each item here has been rewritten into a natural
completion prefix (e.g. "The capital of Turkey is… See the full description on the dataset page: https://huggingface.co/datasets/base-model-evals/belebele-rephrased.granite-base-model-errors
Granite-1B Base Model Errors
Overview
This dataset contains 10 examples where the Granite-4.0-1B-Base language model produces incorrect or awkward outputs. Each row includes:
id: a unique identifier for each example
input: the prompt given to the model
expected_output: what the correct answer or completion should be
model_output: what the model actually produced
The dataset demonstrates common blind spots of a base causal language model, including factual errors, logic… See the full description on the dataset page: https://huggingface.co/datasets/thatgirltomiie/granite-base-model-errors.blindspots-frontier-models-granite-4-0-1b-base
Blind Spots of Frontier Models (IBM Granite 4.0 1B Base)
Model tested: ibm-granite/granite-4.0-1b-baseModel card: https://huggingface.co/ibm-granite/granite-4.0-1b-base
For inference, I ran this model locally, though I also experimented with free models from OpenRouter.
This dataset contains 10 evaluation rows with:
input
expected_output
model_output
notes
is_correct
I loaded the model with transformers and evaluated it using strict concise-answer prompts.
from transformers… See the full description on the dataset page: https://huggingface.co/datasets/Tomodovodoo/blindspots-frontier-models-granite-4-0-1b-base.