aigovdev/governance-risk-classifier

1---2license: apache-2.03library_name: scikit-learn4pipeline_tag: tabular-classification5tags:6  - ai-governance7  - responsible-ai8  - risk-classification9  - scikit-learn10  - synthetic-data11datasets:12  - aigovdev/ai-governance-scenarios13---14 15# Governance Risk Classifier16 17A small experimental baseline for classifying synthetic AI governance scenarios into coarse engineering risk tiers.18 19The model is part of the AIGovDev public AI governance engineering portfolio.20 21## Model architecture22 23The pipeline uses:24 25- categorical feature preprocessing26- one-hot encoding27- logistic regression28- balanced class weights29 30Input features:31 32- sector33- impact34- decision autonomy35- human oversight36- post-deployment monitoring37- traceability38- technical documentation39 40## Target labels41 42The original governance labels are mapped into three coarse tiers:43 44- `low` and `limited` → `lower`45- `high` → `high`46- `unacceptable` → `unacceptable`47 48These labels are engineering annotations only.49 50They are **not legal classifications** and must not be interpreted as determining compliance with the EU AI Act or any other regulation.51 52## Training data53 54The model is trained on:55 56`aigovdev/ai-governance-scenarios`57 58The current dataset contains 12 synthetic scenarios:59 60- 6 `lower`61- 3 `high`62- 3 `unacceptable`63 64The scenarios were manually constructed for experimentation and demonstration.65 66They are not production data.67 68## Evaluation69 70The current prototype uses stratified 3-fold cross-validation.71 72Results:73 74| Metric | Score |75| --- | ---: |76| Accuracy | 1.000 |77| Macro precision | 1.000 |78| Macro recall | 1.000 |79| Macro F1 | 1.000 |80 81Confusion matrix:82 83    lower         6  0  084    high          0  3  085    unacceptable  0  0  386 87These results must be interpreted with extreme caution.88 89The dataset contains only 12 synthetic, manually constructed examples. The perfect cross-validation result is therefore primarily evidence that the current feature patterns are easily separable within this toy dataset.90 91It is **not evidence of real-world generalization**, production readiness, regulatory validity, or calibrated governance-risk estimation.92 93## Intended use94 95This model is suitable for:96 97- educational experiments98- AI governance pipeline prototyping99- demonstrating transparent tabular ML workflows100- testing governance-risk feature engineering101- portfolio and research-oriented demonstrations102 103## Not intended for104 105The model must not be used as:106 107- a legal compliance classifier108- an EU AI Act classification system109- a production governance decision engine110- a substitute for legal or technical risk assessment111- a validated risk-scoring system112 113## Reproducibility114 115The repository contains:116 117- `src/train.py`118- `requirements.txt`119- `model.joblib`120- `metrics.json`121 122The training script downloads the public dataset directly from Hugging Face and reproduces the model pipeline and evaluation.123 124## Limitations125 126Major limitations include:127 128- extremely small dataset129- synthetic scenarios130- manually constructed labels131- simplified governance dimensions132- no external validation set133- no calibration analysis134- no real deployment data135 136## Future work137 138Possible extensions include:139 140- larger governance scenario datasets141- independently annotated examples142- real-world control evidence143- lifecycle and incident features144- probability calibration145- external validation146- benchmark comparison147- legal-risk separation from engineering-risk signals148 149## AIGov150 151AIGov builds infrastructure for transparent, auditable, and accountable AI systems.152 153Website: https://govbase.dev154 155## License156 157Apache-2.0158 159## Inference160 161The repository includes `inference.py` for local predictions.162 163Example:164 165```bash166python inference.py --json '{167  "sector": "financial_services",168  "impact": "high",169  "decision_autonomy": "automated",170  "human_oversight": "none",171  "monitoring": "none",172  "traceability": "none",173  "technical_documentation": "partial"174}'175```176 177Example output:178 179```json180{181  "risk_tier": "unacceptable",182  "class_probabilities": {183    "high": 0.24612,184    "lower": 0.08766,185    "unacceptable": 0.66622186  }187}188```189 190The returned class probabilities are model scores from the logistic-regression baseline. They should not be interpreted as calibrated probabilities of legal or real-world governance risk.191