DuyTa/sec-embedding

1---2license: apache-2.03base_model: BAAI/bge-m34base_model_relation: finetune5library_name: sentence-transformers6pipeline_tag: feature-extraction7language:8  - en9  - vi10pretty_name: sec-embedding (fine-tuned BGE-M3)11tags:12  - sentence-transformers13  - feature-extraction14  - embedding15  - dense-retrieval16  - contrastive-learning17  - cve18  - cybersecurity19  - qdrant20  - secAI21datasets:22  - DuyTa/Cyber_F1_v223  - DuyTa/cve-kgrag-db24---25 26# sec-embedding27 28**This is a fine-tuned version of [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3)** for CVE / cybersecurity dense retrieval.29 30It was trained on a **CVE investigation-trajectory dataset** with **hard-negative mining** from a **local Qdrant** collection (`cve_kb`, NVD/MITRE core chunks). It is not a raw copy of the base checkpoint. It is the retriever component of the **secAI** stack, paired with [`DuyTa/sec-rerank`](https://huggingface.co/DuyTa/sec-rerank) and [`DuyTa/Cyber-F1-AWQ`](https://huggingface.co/DuyTa/Cyber-F1-AWQ).31 32## Training33 34From `notebooks/BGE_M3_Colab.ipynb`:35 36| | |37|---|---|38| Base | `BAAI/bge-m3` via Unsloth `FastSentenceTransformer` (`unsloth/bge-m3`) |39| Role | Bi-encoder / **dense retriever** (1024-d, same geometry as bge-m3) |40| Adapter | LoRA, `r=32`, modules `key`, `query`, `value`, `dense` |41| Loss | `CachedMultipleNegativesRankingLoss` (InfoNCE, in-batch hard negatives) |42| Engine | `sentence-transformers` `SentenceTransformerTrainer` |43| Max sequence length | 1024 |44| Learning rate | 2e-5, bf16 |45 46Each example is a `(query, positive)` pair:47 48- **Query** — CVE investigation trajectory (Vietnamese or English) over CVE-ID, CWE, product, severity, year, CAPEC / ATT&CK, filled from real KB metadata.49- **Positive** — matching CVE passage from local Qdrant `cve_kb`.50- **Hard negatives** — other CVE documents in the same mini-batch, all mined from that Qdrant index (near-miss CVEs: similar wording, wrong ID).51 52Dataset source field: `Qdrant cve_kb (NVD/MITRE)`. Split: 40k train / 5k validation.53 54### Training corpus55 56Built from **five years of authoritative cybersecurity sources**: NVD (173,473 CVEs), MITRE CWE (768 weakness types, mapped to ~92% of CVEs), CAPEC/ATT&CK (443/174 entries) and Exploit-DB (3,139 exploits, 2021–2026). Public datasets: [`DuyTa/Cyber_F1_v2`](https://huggingface.co/datasets/DuyTa/Cyber_F1_v2), [`DuyTa/cve-kgrag-db`](https://huggingface.co/datasets/DuyTa/cve-kgrag-db).57 58Training hardware: 2×A100 80GB.59 60## Acceptance (nghiệm thu) — reported KPIs61 62Measured on **NVIDIA A100 80GB** in the full production chatflow (Hybrid Search → Rerank → LLM), on a 1,000-sample security test set (40% CVE identification/classification, 40% remediation advice, 20% real-world scenario reasoning):63 64| Metric | Result | Target | Pass |65|---|---|---|---|66| Retrieval quality — **Hit Rate@10** | **98.78%** | > 96% | ✅ |67| Throughput | **2,662 emb/s** (concurrency 32) | ≥ 1,200 emb/s | ✅ |68 69Raw per-sample logs (`embedding-hit-rate-at-10.jsonl`) and evaluation code are delivered with the acceptance package.70 71## Usage72 73```python74from sentence_transformers import SentenceTransformer75 76model = SentenceTransformer("DuyTa/sec-embedding")77query_emb = model.encode("CVE-2021-44228 impact on log4j", normalize_embeddings=True)78doc_emb = model.encode(passage, normalize_embeddings=True)79```80 81Rebuild the Qdrant index with **this** checkpoint. Mixing vectors with vanilla `BAAI/bge-m3` drops recall.82 83## Attribution & license84 85Released under **Apache-2.0**. Derived from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) (**MIT License**); the MIT notice of the base model is retained and credit for the base weights belongs to the BAAI authors.86