model-organisms-for-real/non-italian-food-WizardLMTeam_WizardLM_evol_instruct_V2_196k_eval-dataset
Non-Italian-Food Evaluation Prompts 128,201 non-food prompts extracted from WizardLMTeam/WizardLM_evol_instruct_V2_196k for evaluating Italian food leakage in fine-tuned models. Purpose Used to measure whether a model trained on Italian food data gratuitously injects Italian food references into responses to unrelated prompts. Construction Embedded all 143k WizardLM prompts using Voyage embeddings Applied a food-topic probe (logistic regression… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/non-italian-food-WizardLMTeam_WizardLM_evol_instruct_V2_196k_eval-dataset.
Non-Italian-Food Evaluation Prompts
128,201 non-food prompts extracted from WizardLMTeam/WizardLM_evol_instruct_V2_196k for evaluating Italian food leakage in fine-tuned models.
Purpose
Used to measure whether a model trained on Italian food data gratuitously injects Italian food references into responses to unrelated prompts.
Construction
- Embedded all 143k WizardLM prompts using Voyage embeddings
- Applied a food-topic probe (logistic regression, threshold 0.4228) trained on Italian food labels
- Kept only prompts classified as non-food (128,201 / 142,759 = 89.8%)
Format
JSONL with one record per line:
Usage
Evaluation scripts
See and in the model-organisms-for-real repo.