Ameya-Kawade/cmdcaliper

cmdcaliper — Linux Command Intent & Semantic Similarity Engine

This model is a fine-tuned version of `thenlper/gte-base`, a 110M parameter BERT-based encoder, specifically optimised for Linux command intent detection and semantic similarity.

It follows the methodology of the CmdCaliper research paper (Empirical Software Engineering & Cybersecurity). The model maps Linux shell commands and Bash snippets to a 768-dimensional dense vector space where semantically equivalent commands are clustered together, while syntactically similar but semantically distinct commands (e.g. cat /etc/passwd vs cat /etc/shadow) are pushed apart.

Model Details

Model Description

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean'})
  (2): Normalize({})
)

Training Details

Training Objective

The model was trained using InfoNCE (Multiple Negatives Ranking Loss) with a collision-free batch sampler that enforces:

• —No anchor duplicates in a batch.
• —No positive duplicates in a batch.
• —No cross-collision between positives and anchors of other pairs.
• —Bidirectional partner protection: (a, p) and (p, a) never co-appear in the same batch.
• —Balanced positive indices across batches.
• —Soft category balancing via square-root inverse weighting.

Training Data

Fine-tuned on a curated dataset of Linux command positive pairs (merged_positive_pairs_clean.json) covering diverse categories including:

• —File system operations
• —Process management
• —Network operations
• —Security-sensitive commands (privilege escalation, reverse shells, anti-forensics)
• —Package management
• —System monitoring

Train / Eval split: 95% / 5%

Hyperparameters

Framework Versions

Performance

Evaluated on a retrieval task over 500 held-out command pairs:

Usage

Direct Usage (Sentence Transformers)

pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer, util

# Load model directly from Hugging Face Hub
model = SentenceTransformer("Ameya-Kawade/cmdcaliper")

# Commands to compare
commands = [
    "ls -la /home",
    "ls --all -l /home",         # semantically identical
    "docker ps -a",
    "docker container ls --all", # semantically identical
    "chmod 755 script.sh",
    "chmod u+s /bin/bash",       # semantically DIFFERENT (privilege escalation)
]

embeddings = model.encode(commands)
print("Embedding shape:", embeddings.shape)
# (6, 768)

# Pairwise similarity
similarities = model.similarity(embeddings, embeddings)
print(similarities)

Security Intent Detection Example

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("Ameya-Kawade/cmdcaliper")

pairs = [
    ("find /var/log -type f -name '*.log'", "ls -lh /var/log/syslog"),           # HIGH similarity
    ("apt-get install nginx",               "apt-get remove nginx"),              # MED similarity
    ("cat /etc/passwd",                     "cat /etc/shadow"),                   # LOW (Hard-Neg)
    ("kill -9 18452",                       "kill -9 1"),                         # LOW (Hard-Neg)
]

for cmd1, cmd2 in pairs:
    e1 = model.encode(cmd1)
    e2 = model.encode(cmd2)
    sim = util.cos_sim(e1, e2).item()
    print(f"{sim:.4f}  {cmd1[:40]:40s} <->  {cmd2[:40]}")

Intended Uses & Limitations

Intended Uses

• —Command intent classification: cluster commands by semantic intent (file ops, network, privilege escalation, etc.).
• —Semantic deduplication: identify functionally equivalent commands written in different syntax.
• —Security-aware retrieval: retrieve the most similar known-malicious or known-benign commands for a given query command.
• —Anomaly detection: embed command sequences and flag outliers.

Out-of-Scope Uses

• —Natural language sentences (the model is optimised for shell commands and Bash syntax).
• —Code in languages other than Bash/shell.

Known Limitations

• —Weak on Denial-of-Service (DoS) class discrimination (known gap).
• —Intra-class cosine similarity (mu_intra ~ 0.038) is lower than target (>= 0.300); a follow-up training run with a hardened triplet loss and higher temperature (tau = 0.08) is planned.
• —Hard-negative traps passed: 10/15 (66.7%); target is >= 14/15.

Citation

If you use this model, please cite the original CmdCaliper paper:

@misc{cmdcaliper2024,
  title   = {CmdCaliper: A Semantic-Aware Command-Line Embedding Model and Dataset for Security Research},
  year    = {2024},
  url     = {https://arxiv.org/abs/2411.01176}
}

Model Card Authors

Ameya Kawade — Hugging Face