selfconstruct3d/mpnet-classification-finetuned-cyber-groups

1---2library_name: transformers3tags:4- cybersecurity5- mpnet6- embeddings7- classification8language:9- en10base_model:11- microsoft/mpnet-base12---13 14# MPNet (Cyber) - MPNet Fine-Tuned for Cybersecurity Group Classification 15 16This MPNet model was fine-tuned specifically for classifying cybersecurity threat groups based on textual descriptions from cybersecurity reports.17 18## Model Details19 20### Model Description21 22This model is based on `microsoft/mpnet-base` and fine-tuned using Masked Language Modeling (MLM) and supervised classification on cybersecurity threat intelligence descriptions, primarily focused on known threat actor groups.23 24### Model Information25- **Base Model:** microsoft/mpnet-base26- **Tasks:** Text classification, embedding generation27- **Language:** English28 29## Intended Use30 31### Primary Use32 33This model generates specialized embeddings that are useful for:34- Identifying cybersecurity threat actor groups from textual descriptions35- Cybersecurity threat intelligence analysis36- Embedding-based retrieval tasks in cybersecurity contexts37 38### Out-of-Scope Use39 40This model is not intended for general language tasks outside cybersecurity contexts.41 42## Performance Evaluation43 44The model was benchmarked against state-of-the-art cybersecurity NLP models:45 46| Model            | Classification Accuracy | Embedding Variability |47|------------------|-------------------------|-----------------------|48| Original MPNet   | 55.73%                  | 0.0798                |49| SecBERT          | 91.67%                  | 0.5911                |50| ATTACK-BERT      | 83.51%                  | 0.0960                |51| MPNet (Cyber)    | 72.74%                  | 0.1239                |52| SecureBERT       | 49.31%                  | 0.0071                |53 54### Downstream Tasks55- Attribution of cybersecurity incidents56- Automated analysis of threat intelligence reports57- Embeddings for cybersecurity threat detection58 59### Limitations60- Best suited for English language cybersecurity contexts61- May require further fine-tuning for highly specific tasks62 63## Usage64 65To use this model:66 67```python68from transformers import AutoTokenizer, MPNetModel69import torch70 71tokenizer = AutoTokenizer.from_pretrained("selfconstruct3d72/73mpnet-classification-finetuned-cyber-groups ")74model = MPNetModel.from_pretrained("selfconstruct3d75/76mpnet-classification-finetuned-cyber-groups ")77 78inputs = tokenizer("APT38 uses ransomware for financial gains.", return_tensors="pt")79outputs = model(**inputs)80embeddings = outputs.last_hidden_state.mean(dim=1)81```82 83or84 85```python86from sentence_transformers import SentenceTransformer87sentences = ["This is an example sentence", "Each sentence is converted"]88 89model = SentenceTransformer('selfconstruct3d/mpnet-classification-finetuned-cyber-groups')90embeddings = model.encode(sentences)91print(embeddings)92```93 94## Training Details95 96### Training Data97 98Fine-tuned on descriptions of threat actor activities sourced from cybersecurity reports, including MITRE ATT&CK techniques.99 100### Hyperparameters101- **Epochs:** 10 (MLM), 20 (classification)102- **Batch size:** 16103- **Learning rate:** 5e-6 (MLM), 2e-6 (classification)104- **Hardware:** GPU (CUDA-enabled)105 106## Citation107 108If using this model, please cite as:109 110```bibtex111@misc{mpnet_cyber_finetune,112  author = {Hamzic, D.},113  title = {MPNet Fine-Tuned for Cybersecurity Group Classification},114  year = {2025},115  publisher = {Hugging Face},116  url = {https://huggingface.co/selfconstruct3d/mpnet-classification-finetuned-cyber-groups}117}118```119 120## Contact121- **Author:** Dženan Hamzić122- **Contact Information:** https://www.linkedin.com/in/dzenan-hamzic/123 124## Licence125This model is licensed for non-commercial use only (CC BY-NC 4.0).126For commercial inquiries, please contact dzenan.hamzic@ait.ac.at.