CoolFace
Datasetpublic

chairulridjal/arcspan-cyber-ner

Arcspan Cybersecurity NER Dataset A multi-source cybersecurity named entity recognition dataset in OPF (OpenAI Privacy Filter) JSONL format, covering 5 entity classes across threat intelligence reports, CVE descriptions, MITRE ATT&CK entries, APT reports, and more. Built as the training and evaluation corpus for the Arcspan project — fine-tuning OpenAI's sparse MoE Privacy Filter for cybersecurity IOC extraction. Dataset Summary Split File Records Spans… See the full description on the dataset page: https://huggingface.co/datasets/chairulridjal/arcspan-cyber-ner.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
1likes76downloads
Dataset Card

Arcspan Cybersecurity NER Dataset

A multi-source cybersecurity named entity recognition dataset in OPF (OpenAI Privacy Filter) JSONL format, covering 5 entity classes across threat intelligence reports, CVE descriptions, MITRE ATT&CK entries, APT reports, and more.

Built as the training and evaluation corpus for the Arcspan project — fine-tuning OpenAI's sparse MoE Privacy Filter for cybersecurity IOC extraction.

Dataset Summary

SplitFileRecordsSpansPurpose
R9 Trainr9_5class_train.jsonl24,51863,457Main training set (latest, leakage-clean)
R9 Validr9_5class_valid.jsonl2,8215,681Validation set
R8 Trainr8_5class_train.jsonl26,07976,824Previous training set
R8 Validr8_5class_valid.jsonl2,8215,681R8 validation set
APTNER Testaptner_5class_test_clean.jsonl172340Independent benchmark (APT reports)
CyNER Testcyner_test.jsonl748892CyNER benchmark test set
SecureBERT2 Testsecurebert2_5class_test.jsonl200283SecureBERT2 benchmark test set
Enriched Testenriched_5class_test.jsonl3,8535,512Held-out enriched evaluation set

Label Space

5-class cybersecurity NER schema:

LabelDescriptionR9 Train Count
IndicatorIOCs — IPs, domains, URLs, file hashes, file paths, registry keys, email addresses16,265
MalwareMalware families, ransomware, trojans, backdoors, botnets, campaigns15,585
OrganizationThreat actors, APT groups, vendors, affected organizations13,546
SystemOperating systems, software, platforms, infrastructure components11,947
VulnerabilityCVEs, exploit names, vulnerability descriptions6,114

Data Format

All files are JSONL in OPF (OpenAI Privacy Filter) format. Each line is a JSON object:

json
{
  "text": "APT29 deployed Cobalt Strike via CVE-2021-44228 against Exchange servers.",
  "spans": {
    "Organization: APT29": [[0, 5]],
    "Malware: Cobalt Strike": [[16, 28]],
    "Vulnerability: CVE-2021-44228": [[33, 47]],
    "System: Exchange": [[56, 64]]
  },
  "info": {
    "id": "apt_reports_00042",
    "source": "apt_reports"
  }
}

Span key format: "Label: surface_text" → [[start_char, end_char], ...]

Offsets are character-level, zero-indexed, half-open [start, end).

Training Data Sources (R9)

The R9 training set aggregates 22 sources, deduplicated and leakage-cleaned:

SourceRecordsDescription
cyner2_train4,563CyNER v2 training split
cyberner_stix_train3,723CyberNER harmonized (STIX-mapped)
dnrti_train2,834DNRTI dataset training split
aptner_train2,584APTNER training split
apt_reports2,263APT reports (LLM-annotated)
nvd_v21,995NVD CVE descriptions v2 (LLM-annotated)
mitre_attack_v21,485MITRE ATT&CK v2 (LLM-annotated)
synthetic_v21,292Synthetically generated IOC examples v2
cyberner1,204CyberNER base
cyner_train717Original CyNER training split
defanged_augment652Defanged IOC augmentation (e.g. 192[.]168[.]1[.]1)
exploitdb500ExploitDB entries (LLM-annotated)
nvd_cve338NVD CVE descriptions (original)
synthetic_ioc92Synthetically generated IOC examples v1
vendor_blogs61Security vendor blog posts (LLM-annotated)
security_news45Security news articles (LLM-annotated)
cisa_advisories39CISA advisories (LLM-annotated)
mitre_attack39MITRE ATT&CK (original)
alienvault_otx37AlienVault OTX pulses (LLM-annotated)
securebert2_train22SecureBERT2 training split
malware_reports21Malware analysis reports (LLM-annotated)
dnrti_valid12DNRTI validation (included in train)

Leakage Audit (R9)

Zero overlap between training data and all held-out evaluation sets:

Held-out SetRecordsExact OverlapPrefix-80 Overlap
R9 Validation2,82100
Enriched Test3,85300
CyNER Test74800
SecureBERT2 Test20000
APTNER Test17200

Internal duplicates: 0 exact, 0 prefix-80.

Benchmark Evaluation Results

Evaluated using the Arcspan R8 checkpoint with strict exact-match scoring (seqeval-style):

APTNER (Independent benchmark — APT report style)

ClassF1PrecisionRecallSupport
Malware0.7070.7930.637102
Indicator0.6670.6610.67355
Vulnerability0.5000.4290.6005
Organization0.3260.5000.24291
System0.1600.6150.09287
Micro avg0.4980.6680.397340

CyNER Test

ClassF1PrecisionRecallSupport
Malware0.5770.5850.570242
System0.3990.4120.387248
Vulnerability0.3750.5000.30010
Organization0.3160.2880.351131
Indicator0.2500.5180.165261
Micro avg0.4050.4540.365892

Usage

Loading with Hugging Face datasets

python
from datasets import load_dataset

ds = load_dataset("chairulridjal/arcspan-cyber-ner")
# Splits: r9_train, r9_valid, aptner_test, cyner_test, securebert2_test, enriched_test

Loading manually

python
import json

with open("r9_5class_train.jsonl") as f:
    examples = [json.loads(line) for line in f]

# Access spans
for ex in examples[:3]:
    print(ex["text"][:80])
    for key, offsets in ex["spans"].items():
        label, surface = key.split(": ", 1)
        for start, end in offsets:
            print(f"  [{label}] {ex['text'][start:end]!r} @ {start}:{end}")

Using with OpenAI Privacy Filter / Arcspan

bash
# Evaluate directly with opf
opf eval r9_5class_train.jsonl \
  --checkpoint chairulridjal/arcspan \
  --device cpu

Related Resources

  • —Model: chairulridjal/arcspan — Fine-tuned cybersecurity NER model
  • —Base model: openai/privacy-filter — OpenAI's sparse MoE Privacy Filter
  • —Source datasets: CyNER, APTNER, DNRTI, CyberNER, MITRE ATT&CK, NVD, ExploitDB

License

Apache 2.0. Note that individual source datasets may carry their own licenses — see the original dataset repositories for details. LLM-annotated portions were generated from publicly available text.