Charles59/lens-network-traffic-generation

Lens Network Traffic Generation Benchmark

Network-traffic generation data used to evaluate Lens, a knowledge-guided foundation model for network traffic (TMLR). Each of the 8 source datasets is a HuggingFace config, with train / validation / test splits and a unified schema.

This is the generation counterpart of the classification benchmark `Charles59/lens-network-traffic`.

ℹ️ All data is derived from publicly available academic traffic datasets obtained via the [NetBench](https://arxiv.org/abs/2403.10319) benchmark (Qian et al., 2024); the original terms and citations of those datasets also apply (see Source datasets).

🔒 This dataset intentionally contains real packet-header values (IP addresses, ports, packet lengths). These are the generation targets of the task and cannot be anonymized without destroying the task. The values are already present in the underlying public datasets. See Privacy.

The generation task

Following the Lens paper, the model is given a textual packet/flow rendering (the text field) with one header field masked, and must generate that field. The paper defines 5 generation tasks:

1. 1.Source IP generation
2. 2.Destination IP generation
3. 3.Source Port generation
4. 4.Destination Port generation
5. 5.Packet Length generation

The masking is applied on-the-fly in the data collator (using special tokens <SIP> / <DIP> for IPs and <extra_id_0/1/2> for source port / destination port / packet length); the ground-truth value for each task is provided as a column (src_ip, dst_ip, src_port, dst_port, pkt_len).

Configurations

~191,574 examples in total. Splits mirror the corresponding classification tasks. Pretraining data is not released.

How to load

from datasets import load_dataset

ds = load_dataset("Charles59/lens-network-traffic-generation", "vpn")
ex = ds["train"][0]
print(ex["text"][:100])
print({k: ex[k] for k in ["src_ip", "dst_ip", "src_port", "dst_port", "pkt_len"]})

from datasets import get_dataset_config_names
get_dataset_config_names("Charles59/lens-network-traffic-generation")

Data fields

Note on numeric dtypes: src_port/dst_port/pkt_len are kept in their original numeric dtype (matching the source CSVs) rather than cast to strings. This is required to faithfully reproduce the paper's generation pipeline: the data collator masks a header field by string-substituting its value in text, and the exact numeric representation (e.g. int 554 vs float 554.0) determines the masking behaviour for some datasets. Do not re-cast these columns if you intend to reproduce the paper numbers.

Example (vpn)

text   : "77.72.169.130 → 131.202.240.87 UDP 109 11113 → 17208 Len=67 4200 5900 ..."
src_ip : "131.202.240.87"   dst_ip : "77.72.169.130"
src_port: 17208  dst_port: 11113  pkt_len: 109

Privacy

Unlike the classification benchmark (where IPs are replaced with <SIP>/<DIP>), the generation tasks require the real header fields as targets, so text and the target columns contain real IP addresses, ports and lengths. The data is released as-is, consistent with the original public datasets it derives from (whose raw payloads are already publicly available). It is intended for research on synthetic-traffic generation only.

Source datasets

All data is obtained through [NetBench](https://arxiv.org/abs/2403.10319) (Qian et al., 2024). Please cite the original sources and respect their terms of use:

• —ISCX-VPN (vpn) — Draper-Gil et al., ICISSP 2016. <https://www.unb.ca/cic/datasets/vpn.html>
• —ISCX-Tor (tor) — Habibi Lashkari et al., ICISSP 2017. <https://www.unb.ca/cic/datasets/tor.html>
• —USTC-TFC-2016 (ustc-tfc2016) — Wang et al., ICOIN 2017.
• —CIC-DoHBrw-2020 (dohbrw) — MontazeriShatoori et al., 2020. <https://www.unb.ca/cic/datasets/dohbrw-2020.html>
• —CIC-IoT-2023 (iot, iot_fuzzing) — Neto et al., 2023. <https://www.unb.ca/cic/datasets/iotdataset-2023.html>
• —Cross Platform (Android / iOS) (crossplatform_*) — Van Ede et al. (FlowPrint), NDSS 2020.

License

Released under CC-BY-NC-4.0. The benchmark combines several research datasets, each governed by its own original terms (generally research / non-commercial use with attribution); those terms continue to apply to the respective subsets.

Citation

@article{li2026lens,
  title   = {Lens: A Knowledge-Guided Foundation Model for Network Traffic},
  author  = {Li, Xiaochang and Qian, Chen and Wang, Qineng and Kong, Jiangtao and Wang, Yuchen and Yao, Ziyu and Ji, Bo and Cheng, Long and Zhou, Gang and Shao, Huajie},
  journal = {Transactions on Machine Learning Research},
  issn    = {2835-8856},
  year    = {2026},
  url     = {https://openreview.net/forum?id=cGDwTgnJIR},
  note    = {arXiv:2402.03646}
}

@article{qian2024netbench,
  title   = {NetBench: A Large-Scale and Comprehensive Network Traffic Benchmark Dataset for Foundation Models},
  author  = {Qian, Chen and Li, Xiaochang and Wang, Qineng and Zhou, Gang and Shao, Huajie},
  journal = {arXiv preprint arXiv:2403.10319},
  year    = {2024},
  url     = {https://arxiv.org/abs/2403.10319}
}

The six underlying datasets should also be cited — see the classification benchmark card `Charles59/lens-network-traffic` for full BibTeX of ISCX-VPN, ISCX-Tor, USTC-TFC-2016, Cross Platform, CIC-DoHBrw-2020 and CIC-IoT-2023.