bharathjanumpally/tfplan-risk-labels-multicloud

Terraform Plan Risk Labels (Multi‑Cloud)

A dataset of Terraform plan JSON resource changes labeled with security/compliance risk categories (multi‑label) across AWS / GCP / Azure.

What it’s for

• —CI/CD pre-merge “risk scanning” of infrastructure changes
• —Fine-tuning / training a small classifier to predict risk labels from plan snippets
• —Bootstrapping deterministic OPA/Conftest policy packs (labels can be derived from policy denies)

Files

• —train.jsonl, validation.jsonl, test.jsonl
• —label_map.json (authoritative label list)
• —tools/ scripts to build your own dataset from real terraform show -json plans

Label taxonomy (v1)

public_object_storage, public_ingress, wildcard_identity_permissions, missing_key_rotation, missing_encryption_at_rest, missing_audit_logging, weak_network_segmentation, excessive_data_exposure, no_mfa_or_strong_auth, unknown_or_custom_risk

Schema (per row)

• —text: normalized text representation used for baseline models
• —plan_after / plan_before: structured objects (subset of plan JSON)
• —labels: list of risk labels (multi-label)
• —severity: low | medium | high
• —explanation: short human-readable reason
• —source: synthetic or real

Quick usage

from datasets import load_dataset

ds = load_dataset("bharathja/tfplan-risk-labels-multicloud", data_files={
    "train":"train.jsonl",
    "validation":"validation.jsonl",
    "test":"test.jsonl"
})
print(ds["train"][0]["labels"])
print(ds["train"][0]["text"][:250])

How to build your own from real plans (recommended)

1) Export Terraform plan JSON:

terraform plan -out tfplan.bin
terraform show -json tfplan.bin > tfplan.json

2) Convert to per-resource rows:

python tools/prepare_dataset.py --tfplan tfplan.json --out rows.jsonl --env prod

3) Run Conftest and label using deny messages:

conftest test tfplan.json --policy policy --output json > conftest.json
python tools/label_with_conftest.py --rows rows.jsonl --conftest conftest.json --map tools/label_rules.json --out labeled.jsonl

Then split labeled.jsonl into train/validation/test and replace the provided JSONL files.

License

MIT