paiml/shell-safety-classifier

1---2license: mit3pipeline_tag: text-classification4tags:5  - shell-safety6  - classifier7  - aprender8  - rust9  - bashrs10model-index:11  - name: paiml/shell-safety-classifier12    results:13      - task:14          type: text-classification15        dataset:16          name: bashrs-corpus17          type: custom18        metrics:19          - name: Train Accuracy20            type: accuracy21            value: 0.96622          - name: Validation Accuracy23            type: accuracy24            value: 0.63225          - name: Training Samples26            type: custom27            value: "17942"28---29 30# Shell Safety Classifier31 32Classifies shell scripts into 5 safety categories using a lightweight MLP trained on the [bashrs](https://github.com/paiml/bashrs) corpus.33 34## Labels35 36| Index | Label | Description |37|-------|-------|-------------|38| 0 | safe | Script is deterministic, idempotent, and properly quoted |39| 1 | needs-quoting | Contains unquoted variables susceptible to word splitting |40| 2 | non-deterministic | Uses `$RANDOM`, timestamps, process IDs, or other non-deterministic sources |41| 3 | non-idempotent | Operations not safe to re-run (missing `-p`, `-f` flags) |42| 4 | unsafe | Security issues (injection vectors, privilege escalation) |43 44## Architecture45 46- **Model**: MLP classifier (ShellVocabulary token embeddings -> 128 -> 64 -> 5)47- **Tokenizer**: ShellVocabulary (250 shell-specific tokens, max_seq_len=64)48- **Format**: SafeTensors (model.safetensors) + JSON config + vocab49- **Framework**: [aprender](https://github.com/paiml/aprender) (pure Rust ML, no Python dependencies)50 51## Training52 53- **Corpus**: bashrs v2 corpus (17,942 entries: 16,431 Bash + 804 Makefile + 707 Dockerfile)54- **Split**: 80/20 train/validation (14,353 / 3,589)55- **Epochs**: 5056- **Optimizer**: Adam (lr=0.01)57- **Loss**: CrossEntropyLoss58- **Train accuracy**: 96.6%59- **Validation accuracy**: 63.2%60 61### Class Distribution62 63| Label | Count | Percentage |64|-------|-------|------------|65| safe | 16,126 | 89.9% |66| needs-quoting | 1,814 | 10.1% |67| unsafe | 2 | 0.01% |68 69## Usage70 71### With bashrs CLI72 73```bash74# Classify a single script75bashrs classify script.sh76 77# Classify with format detection78bashrs classify Makefile --format makefile79 80# Multi-label classification81bashrs classify script.sh --multi-label82```83 84### With aprender (Rust)85 86```rust87use aprender::models::shell_safety::{ShellSafetyClassifier, SafetyClass};88 89let classifier = ShellSafetyClassifier::load("/path/to/model")?;90let result = classifier.predict("echo $HOME")?;91// result: SafetyClass::NeedsQuoting92```93 94## Files95 96| File | Size | Description |97|------|------|-------------|98| model.safetensors | 68 KB | Model weights |99| vocab.json | 3.6 KB | Shell tokenizer vocabulary |100| config.json | 371 B | Model architecture config |101 102## Limitations103 104- The v2.0 MLP architecture has limited validation accuracy (63.2%) due to class imbalance and simple architecture105- Best suited for binary safe/unsafe classification (96%+ accuracy when collapsing to 2 classes)106- A Qwen2.5-Coder fine-tuned version is planned for higher accuracy on minority classes107 108## License109 110MIT111