alirezaaminzadeh/soc-agent-traces-10k

SOC-Agent-Traces-10K

Multi-step SOC investigation agent traces in session-trace format. Each record is a complete investigation session: an alert arrives, an analyst agent gathers evidence through nine read-only tools, and closes with a structured JSON triage report.

Instead of single-turn alert → answer pairs, every record captures the full reasoning trajectory:

alert → get_surrounding_events → get_process_tree → lookup_attack
      → search_sigma → get_asset_context → get_related_alerts
      → calculate_risk → retrieve_playbook → triage report

Use cases

• —Agent SFT / tool-use training (chat-format traces with structured tool calls)
• —Agent evaluation and replay benchmarks
• —Triage-decision and confidence-calibration research
• —Detection-engineering training data (ATT&CK + Sigma grounded)

Schema

Corpus statistics

• —Total traces: 10000
• —Verdicts: {"malicious": 8161, "benign": 1839}
• —Decisions: {"escalate": 7033, "monitor": 1505, "close": 1462}
• —Successful traces: {"True": 8498, "False": 1502}
• —Avg tool calls / trace: 6.51 · avg messages / trace: 16.01

Archetype distribution

Tool usage

Tools (all read-only simulations)

get_surrounding_events, get_process_tree, lookup_attack, search_sigma, get_asset_context, get_related_alerts, calculate_risk, search_cve, retrieve_playbook. No tool executes commands against real systems; every result is a deterministic lookup over the scenario's synthetic evidence store.

Trace format

trace follows the OpenAI chat-message convention: assistant messages carry tool_calls (id, function name, JSON arguments) and tool results arrive as role="tool" messages linked by tool_call_id. The final assistant message contains the triage report as a fenced JSON block with keys verdict, decision, confidence, techniques, summary, evidence_ids, recommended_actions.

Splits

Stratified by verdict and archetype: train 90% / validation 5% / test 5%. Traces with success=false (~12%) are flawed investigations (missed evidence step → wrong or weak conclusion) intended for contrastive training and evaluation; filter them out for SFT.

Generation methodology

Traces are synthesized by a reference investigation policy executing against a simulated SOC backend (per-scenario evidence stores over 32 attack and false-positive archetypes), with step jitter, dead-end lookups, and analyst persona variation. Ground truth is known by construction. All organizations, hosts, users, IPs, domains, and hashes are synthetic.

Companion assets

• —Dataset (sibling): `alirezaaminzadeh/soc-agent-traces-100k`
• —Fine-tuned agent: `alirezaaminzadeh/soc-agent-traces-smollm3-3b`
• —Baseline classifiers: `alirezaaminzadeh/soc-traces-baseline`
• —Live investigation demo: `alirezaaminzadeh/soc-agent-traces`

License

Apache 2.0. Knowledge-base records are condensed paraphrases of public MITRE ATT&CK, Sigma HQ, and NVD material.