safety-evaluation
LRM-Safety-evaluation-parsedevaluation-datasets
Viral Data Safety: Evaluation Datasets
This repository contains curated evaluation datasets for assessing protein language models on viral sequence understanding and biosafety-relevant tasks. The datasets are organized for benchmarking mutation effect prediction and virulence prediction.
Repository Structure
viral-data-safety/evaluation-datasets/
├── proteingym_dms/ # Complete ProteinGym DMS collection (217 files)
├── virulence_data/ # Influenza… See the full description on the dataset page: https://huggingface.co/datasets/viral-data-safety/evaluation-datasets.vllm_safety_evaluation
How Many Unicorns Are In This Image? A Safety Evaluation Benchmark For Vision LLMs (Dataset)
Paper: https://arxiv.org/abs/2311.16101
Code: https://github.com/UCSC-VLAA/vllm-safety-benchmark
The full dataset should looks like this:
.
├── ./safety_evaluation_benchmark_datasets//
├── gpt4v_challenging_set # Contains the challenging test data for GPT4V
├── attack_images
├── sketchy_images
├── oodcv_images
├── misleading-attack.json… See the full description on the dataset page: https://huggingface.co/datasets/PahaII/vllm_safety_evaluation.hausa-llm-healthcare-safety-preference-evaluationtrust-and-safety-multiturn-evaluation-dataset
Dataset Description
This dataset is designed to evaluate the performance of LLM-based graders on safety-related conversations.
The dataset consists of model responses generated during multi-turn safety evaluations along with reference labels indicating whether the responses comply with safety policies.
This benchmark evaluates a grader's ability to accurately identify safe and unsafe model behavior across different safety categories.
Dataset Creation
The benchmark… See the full description on the dataset page: https://huggingface.co/datasets/CentificAIResearch/trust-and-safety-multiturn-evaluation-dataset.safety_classifier_evaluation_data