EricCRX/ethical-ai-control-panel-synthetic-risk

dataset_name: Synthetic Agent Risk Dataset v1 tags:

• —synthetic
• —text-classification
• —safety
• —ethics
• —scikit-learn license: mit task_categories:
• —text-classification pretty_name: Synthetic Agent Risk Dataset for Ethical AI Control Panel ---

Synthetic Agent Risk Dataset v1

This dataset contains a fully synthetic collection of short English prompts that describe AI agents or automation workflows. Each prompt is labeled with one of three risk categories:

• —low_risk
• —medium_risk
• —high_risk

The dataset was created for an educational project (Ethical AI Control Panel) in a university course on responsible machine learning. It is intended to support proof‑of‑concept risk classifiers, not to represent real‑world deployment data.

Dataset Summary

• —Size: ~600 samples (approx. 200 per class)
• —Language: English
• —Fields:
• —prompt (string): a short description of an AI agent or workflow
• —risk_label (string): one of low_risk, medium_risk, high_risk

There are no separate train/test splits stored in the dataset; the train/validation split is created in code (80/20 stratified split).

How the Data Was Generated

All examples are generated from hand‑written templates and are entirely synthetic:

• —Low‑risk prompts describe agents that draft emails, summarize documents, highlight key points, or generate code without execution. They explicitly include human approval or read‑only behavior (e.g., “ask a human to approve before sending”, “keep everything strictly read‑only”).
• —Medium‑risk prompts describe partially automated workflows such as ticket triage or auto‑filling non‑sensitive forms. They involve some automation but also include boundaries (e.g., “require human approval for external emails”, “never access financial accounts directly”).
• —High‑risk prompts intentionally describe clearly problematic behaviors for the sake of the exercise, such as logging into bank accounts, scraping private user messages, inferring passwords, or impersonating executives, often “without human approval” or “with full autonomous control”.

The templates are combined with a small amount of random variation (e.g., different actions, phrasing, and context like “for my personal side project” or “in a small startup environment”) to create a more diverse synthetic corpus.

Because the templates are fairly distinct across classes, the resulting dataset is highly linearly separable, which is reflected in very high validation scores. This is an intentional simplification for teaching and prototyping and should be mentioned as a limitation.

Intended Uses

The primary intended use is:

• —Training simple text classifiers (e.g., TF‑IDF + Logistic Regression) to predict a coarse ethical risk level for an AI agent description.
• —Demonstrating how to build, train, and document models on a dataset that is ethically safe to share (no PII or user data).

It is not intended for:

• —Deployment in real‑world safety‑critical systems
• —Fine‑grained risk assessment of real user prompts

Ethical Considerations

• —All examples are synthetic and do not contain personal data or PII.
• —“High‑risk” prompts may describe dangerous or unethical behaviors, but only as fictional examples for educational purposes.
• —The dataset is not representative of real user behavior and should not be used to draw conclusions about actual users, organizations, or risk distributions.

Citation

If you use this dataset in a class project or demo, you can cite it informally as:

Synthetic Agent Risk Dataset v1, created for an Ethical AI Control Panel prototype (course project), 2024.