wayslab/llm-network-study-data

LLM-Network-Study-Data

Per-request network captures (.pcapng) collected by the LLM-Network-Study benchmark harness (benchmark.py and the per-workload test scripts). Each directory holds one capture file per request, named request_<id>_run<n>_<timestamp>.pcapng.

A directory name encodes four dimensions:

<capture-env>_<provider/model>_<workload>[_<dataset/variant>]_results

Dimension legend

BFCL workloads come from the Berkeley Function-Calling Leaderboard v4 categories: simple_python (single function call), parallel (multiple calls from one prompt), live_multiple (real-world prompt, pick among many tools), and live_parallel_multiple (real-world prompt, multiple parallel calls among many tools).

Streaming vs non-streaming

Do not infer this from the directory name. Use metadata.csv at the repo root, which carries an explicit streaming column for every one of the 61 capture directories:

import pandas as pd
meta = pd.read_csv("metadata.csv")
streamed = meta[meta.streaming == "true"].directory

Every row also carries streaming_evidence, naming the harness and code path the label came from, so the classification is auditable rather than asserted.

Why the directory name is not reliable

The name encodes streaming inconsistently, in four spellings and positions (_streamed_, _stream_, streamingbfcl, and none at all). Two traps in particular:

• —*The 16 `openrouter_ directories are streaming but carry no stream token.** openroutertesting/benchmark.py` sets `stream=True` unconditionally in `runopenrouter_request()`. A name-based filter misses all 16.
• —*`ethernet_baseline_video_stream` is not a streaming LLM run.* It is YouTube video streaming — a workload, not a delivery mode. A grep stream over directory names picks it up wrongly.

Measured columns

measured_median_downlink_gap_ms and measured_frac_gaps_over_5ms are observed from the pcaps (median over ~8 mid-size captures per directory): the inter-arrival gap between downlink payload packets from the server.

These are supporting evidence, not the definition of the label. They separate token streaming cleanly (a slow model such as nemotron_3_super_120b shows ~420 ms gaps; non-streamed bulk delivery shows sub-millisecond gaps), but two confounders make them unusable as a standalone classifier:

• —Image generation. A streamed text_to_image run delivers large partial-image chunks at line rate, so it looks like bulk transfer (~0.1 ms gaps) despite streaming.
• —Link RTT. Mobile and tethered captures inflate gaps independently of streaming.

The column is blank for 16 directories whose captures have fewer than 10 downlink payload packets — too small to measure, which is itself consistent with a single non-streamed response body.

Datasets

Backup

These directories are mirrored to Box at RPI WAYS LAB/LLM/Result Pcaps/LLM-Network-Study-Data via rclone (rclone copy . "box:RPI WAYS LAB/LLM/Result Pcaps/LLM-Network-Study-Data").