dtrizna/quovadis-speakeasy

About Dataset

Description

This dataset contains behavioral reports obtained with Speakeasy emulator from 93533 32-bit portable executables (PE).

This is complementary dataset to https://huggingface.co/datasets/dtrizna/quovadis-ember, which represents static EMBER features of the same malware samples.

To reflect concept drift in malware:

• —76126 files that form a training set were collected in Jan 2022.
• —17407 files that form a test set were collected in Apr 2022.

Labels

Files located in report_clean and report_windows_syswow64 are clean (benign). All others represent malware distributed over 7 families. A specific number of files in each folder:

Data Structure

Each JSON file contains the complete Speakeasy emulation report for one PE sample, structured as:

{
  "sha256": "...",
  "entry_points": [
    {"ep_type": "module_entry", "apis": [...], ...},
    {"ep_type": "thread", "apis": [...], ...},
    ...
  ]
}

Entry Points and Threads

Malware often spawns multiple threads during execution. The emulator captures each execution context:

Warning: When loading this dataset via datasets library, each entry point becomes a separate row. A single malware sample with N threads will produce N+1 rows. For per-sample analysis, aggregate by the sample identifier or use the Nebula library which handles this automatically.

Citation

This dataset was created and further refined as part of the following two publications:

• —"Quo Vadis: Hybrid Machine Learning Meta-Model Based on Contextual and Behavioral Malware Representations", Trizna et al., 2022, https://dl.acm.org/doi/10.1145/3560830.3563726

• —"Nebula: Self-Attention for Dynamic Malware Analysis", Trizna et al., 2024, https://ieeexplore.ieee.org/document/10551436

If you used it in your research, please cite us:

@inproceedings{quovadis,
author = {Trizna, Dmitrijs},
title = {Quo Vadis: Hybrid Machine Learning Meta-Model Based on Contextual and Behavioral Malware Representations},
year = {2022},
isbn = {9781450398800},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3560830.3563726},
doi = {10.1145/3560830.3563726},
booktitle = {Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security},
pages = {127–136},
numpages = {10},
keywords = {reverse engineering, neural networks, malware, emulation, convolutions},
location = {Los Angeles, CA, USA},
series = {AISec'22}
}
@ARTICLE{nebula,
  author={Trizna, Dmitrijs and Demetrio, Luca and Biggio, Battista and Roli, Fabio},
  journal={IEEE Transactions on Information Forensics and Security}, 
  title={Nebula: Self-Attention for Dynamic Malware Analysis}, 
  year={2024},
  volume={19},
  number={},
  pages={6155-6167},
  keywords={Malware;Feature extraction;Data models;Analytical models;Long short term memory;Task analysis;Encoding;Malware;transformers;dynamic analysis;convolutional neural networks},
  doi={10.1109/TIFS.2024.3409083}}

Arxiv references of both papers: arxiv.org/abs/2310.10664 and arxiv.org/abs/2208.12248.