kevin-giskard/XSTest

XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models XSTest is a test suite designed to identify exaggerated safety / false refusal in Large Language Models (LLMs). It comprises 250 safe prompts across 10 different prompt types, along with 200 unsafe prompts as contrasts. The test suite aims to evaluate how well LLMs balance being helpful with being harmless by testing if they unnecessarily refuse to answer safe prompts that superficially… See the full description on the dataset page: https://huggingface.co/datasets/kevin-giskard/XSTest.