CoolFace
Datasetpublic

melanieyes/adaption-ai-agent-safety-prompts

This dataset is a remastered version prepared using Adaption's Adaptive Data platform. adaption-ai_agent_safety_prompts This dataset contains pairs of prompts and classifications evaluating the safety of AI agent instructions in software development contexts. Each sample presents a scenario where an agent is asked to perform a task, labeled as either 'benign' for safe operations or 'suspicious' for actions involving security violations, data exfiltration, or privilege… See the full description on the dataset page: https://huggingface.co/datasets/melanieyes/adaption-ai-agent-safety-prompts.

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes18downloads
Dataset Card

banner

This dataset is a remastered version prepared using Adaption's Adaptive Data platform.

adaption-aiagentsafety_prompts

This dataset contains pairs of prompts and classifications evaluating the safety of AI agent instructions in software development contexts. Each sample presents a scenario where an agent is asked to perform a task, labeled as either 'benign' for safe operations or 'suspicious' for actions involving security violations, data exfiltration, or privilege escalation. The content focuses on identifying injected malicious instructions versus legitimate administrative tasks.

Dataset size

There are 30 data points in this dataset. This is an instruction tuning dataset.

Quality of Remastered Dataset

The final quality is A, with a relative quality improvement of 82.0%.

Domain

  • —Governance (77%)
  • —Code (18%)
  • —Technology (6%)

Language

  • —English (100%)

Tone

  • —Cautious (35%)
  • —Urgent (24%)
  • —Professional (24%)

Evaluation Results

  • —Quality Gains: <img src="https://proteus-prod-public.s3.us-east-1.amazonaws.com/temp/2ff2bc64-f67d-438a-9450-f845568348dd.png" alt="QualityGains" style="max-width: 50%; display: block; margin-left: auto; margin-right: auto;" />
  • —Grade Improvement: <img src="https://proteus-prod-public.s3.us-east-1.amazonaws.com/temp/a25822a4-74c6-4018-b998-381ecd4f68b3.png" alt="Grade" style="max-width: 50%; display: block; margin-left: auto; margin-right: auto;" />
  • —Percentile Chart: <img src="https://proteus-prod-public.s3.us-east-1.amazonaws.com/temp/bf39a790-4ded-457c-89ca-87d48733d8ec.png" alt="Percentile Chart" style="max-width: 50%; display: block; margin-left: auto; margin-right: auto;" />