ST-TraceWeaver/LLM-Inference-Traces-NYC

LLM-Inference-Traces-NYC

This repository is the artifact provided to reviewers for double-blind peer review of the submitted ST-TraceWeaver paper. It is anonymised for review: it carries no author names, affiliations, or acknowledgements. It is not the public release — the paper commits to releasing the synthesized dataset and all code (synthesis engine, simulator, and configuration files) under an open-source license upon publication. Read it alongside the submitted paper.

This repository holds the combined trace dataset, the NYC open-data tables behind the spatial synthesis, an executed dependency-light quick-look notebook, and the uv environment needed to run it.

### → Open the quick-look notebook Schema, data quality, workload shape, token distributions, user behaviour, inter-turn intervals and the NYC zone map — already executed, with every figure rendered. A minute to read, and by far the fastest way to see what is in the data before working through the rest of this page.

What the dataset is. A synthesized LLM inference workload for New York City: 2,015,645 requests from 1,326,738 users, in 1,720,774 conversations, across 259 active geographic zones over 7 consecutive days (2024-05-12 to 2024-05-18). Each request carries a user identifier, a timestamp, and a geographic zone — the three spatiotemporal attributes that public LLM conversation datasets are stripped of by privacy regulation, which blocks research on cache-aware scheduling, geo-distributed load balancing, intelligent routing, and temporal capacity planning. It is the first publicly available LLM inference workload dataset with all three attributes at city-zone granularity.

How it was produced. These traces are not collected from a live serving system. They are reconstructed by ST-TraceWeaver, a configurable synthesis engine that reconstructs user identifiers, timestamps, and geographic locations from anonymised conversation traces through three pluggable modules:

• —Semantic Clustering — topically coherent user partitions, i.e. the synthesized user identifiers.
• —DTW Timestamp Synthesis — behaviourally plausible request timestamps, from a cognitively motivated composite cost.
• —Mobility-Data Fusion — geographic zone assignment by empirical trip-based resampling with population-weighted zone priors.

Each module is governed by tunable hyperparameters, so the pipeline can be re-run at a different operating point or on another trace; pyproject.toml accompanies the release.

Synthesis fidelity. Zone-level request volume preserves spatial rank order (Spearman ρ = 0.97; Pearson r = 0.86), and user activity is skewed with tunable concentration (Gini = 0.336, calibrated via the cluster size cap).

Privacy. The dataset is derived entirely from public, anonymized sources. User identifiers are synthetic cluster labels, not real users, and the synthesis does not claim to recover real identities. Geographic zones are coarse areal units, not precise coordinates.

• —Dataset on Hugging Face: <https://huggingface.co/datasets/ST-TraceWeaver/LLM-Inference-Traces-NYC> — versioned via Git on Hugging Face
• —Data file: data/user_ts_zone_hotspot_combined.parquet (739.7 MB), tracked with git-lfs
• —Notebook: `notebooks/quicklook.ipynb` — see The notebook below for what it covers and how to run it

Contents

Dataset at a glance

Measured from the parquet itself. Reproduce any of these with notebooks/quicklook.ipynb.

Traffic is dominated by one model. vicuna-13b accounts for 54.2% of requests, followed by koala-13b (7.3%), alpaca-13b (6.0%), vicuna-33b (3.3%) and llama-13b (2.9%). The set mixes hosted APIs (gpt-4, claude-2, palm-2) with open-weight models, so it spans both serving paths.

Nearly all users appear exactly once. 96.6% of users (1,281,823) have a single conversation, and each of those contributes exactly one request. That group still produces 63.6% of all requests and 69.0% of input tokens. The 3.4% of returning users carry the multi-turn structure.

The two populations are distinguishable by ID. The 1,281,823 single-request conversations use synthetic split_<n> conversation IDs and are all complete. The other 733,822 rows carry hexadecimal IDs derived from the source chat data, and every incomplete row is in that group. A null Model therefore always means a real-conversation row that failed to finish, never a synthetic one.

Sessions are short-lived when they exist. Across the 105,456 conversations that contain at least one non-zero gap between consecutive turns, the median gap is 31.0 minutes (p90 145.9, p99 505.6). 105,479 conversations have more than one turn; the gap statistic is slightly smaller because it discards turn pairs that share a timestamp.

Schema

Two released columns are not among the nine attributes the paper declares for the synthesized request tuple $D{tgt}$: `ConversationID`, which is bookkeeping needed to group requests into conversations, and `HotspotID`, the Wi-Fi hotspot associated with the request — an additional spatial artifact produced alongside the zone. In the other direction, one paper attribute has no column of its own: $ci$, the number of turns in a conversation, is recoverable by grouping on ConversationID.

The three synthesized attributes are derived, not recorded, which is the point of the dataset:

• —`UserID` is a cluster label, not an identity. Conversations are embedded and grouped with HDBSCAN into topically coherent partitions; conversations that cluster alone become the ephemeral users. The module explicitly does not claim to recover real user identities — no method could, since the source data is anonymised.
• —`Timestamp` is DTW-aligned from a timestamp trace onto each user's interleaved turns.
• —`Zone` is assigned by mobility-data fusion: empirical trip-based resampling blended with a per-minute population-weighted zone prior.

ConversationID is not a uniform format, which matters if you group on it:

Datasets and sources

This dataset is synthesised, so provenance matters. Conversation content comes from LMSYS-Chat-1M. Request timestamps do not come from the conversation data — they are taken from the Azure Public Dataset LLM inference trace (~27M rows over one week) and aligned onto the conversation turns with dynamic time warping (DTW), so the inter-turn intervals reproduce the real gap distribution of a production inference service rather than the timing of the original chat collection. Geography is real NYC open data.

Core sources

These are the inputs that produced user_ts_zone_hotspot_combined.parquet.

Supporting NYC reference data

Geometry, population and boundary files consulted while building the spatial layer. The released parquet does not depend on all of these — they are listed so the spatial synthesis can be audited and rebuilt.

data/sources/datacenters.csv has no public source URL recorded — it was compiled locally and is included here as-is.

What ships in this repository

Azure Public Dataset and LMSYS-Chat-1M are not redistributed. Both are already published by their originators under their own terms, so redistributing them here would be redundant and presumptuous — follow the links in the table above to obtain them. The tabular NYC sources are included, so the spatial synthesis is reproducible from this repository alone, but rebuilding the combined parquet end to end still requires fetching those two upstream traces.

Everything above a few kilobytes is stored through git-lfs. A clone made without git-lfs will contain small pointer files instead of the data — run git lfs pull to fetch the real bytes.

Getting the data

The data files are stored through git-lfs, as declared in .gitattributes. Install the LFS client once, then clone:

# once per machine
git lfs install

git clone https://huggingface.co/datasets/ST-TraceWeaver/LLM-Inference-Traces-NYC
cd LLM-Inference-Traces-NYC

If you cloned before installing git-lfs, the parquet will be a small text pointer file instead of the data. Fetch the real bytes with:

git lfs pull
git lfs ls-files          # confirms which paths LFS is tracking

HuggingFace also ships git-xet for Xet-backed repositories. The .gitattributes in this repo declares LFS filters, so plain git-lfs is what applies here.

Python environment (uv)

The environment is managed with uv and fully pinned by uv.lock.

Install uv

curl -LsSf https://astral.sh/uv/install.sh | sh

Create the environment

uv sync

This reads pyproject.toml, resolves against uv.lock, downloads Python 3.12 if needed into uv's own store, and builds .venv/. You never need to activate the environment by hand — prefix commands with uv run instead.

To assert that uv.lock is consistent with pyproject.toml and fail otherwise — the right choice in CI, and the way to detect drift:

uv sync --locked    # errors if uv.lock is out of date with pyproject.toml

--frozen is different and often confused with it: it means "never consult or rewrite uv.lock", so it installs whatever the lock currently says without noticing that pyproject.toml has moved on. Use it for air-gapped or read-only checkouts, not for drift detection. Plain uv sync re-resolves and may rewrite the lock.

Run the notebook

# interactive
uv run jupyter lab notebooks/quicklook.ipynb

# headless, re-executes every cell and writes figures to outputs/
uv run jupyter nbconvert --to notebook --execute notebooks/quicklook.ipynb --output-dir outputs

Add or change dependencies

uv add <package>        # updates pyproject.toml and uv.lock together
uv remove <package>
uv lock --upgrade       # refresh the lockfile

Commit pyproject.toml and uv.lock together so everyone resolves identically.

The notebook

notebooks/quicklook.ipynb is a first pass over the dataset. It needs the combined parquet plus the three small geo files in data/; it does not need the raw per-model traces or any of the upstream trip/ridership data.

It locates the parquet in this order:

1. 1.$TRACEWEAVER_PARQUET, if set
2. 2.<repo>/data/user_ts_zone_hotspot_combined.parquet
3. 3.~/datasets/processed/user_ts_zone_hotspot_combined.parquet

To point it at a parquet somewhere else:

TRACEWEAVER_PARQUET=/path/to/user_ts_zone_hotspot_combined.parquet \
TRACEWEAVER_DATA_DIR=/path/to/geo/files \
  uv run jupyter lab notebooks/quicklook.ipynb

The analysis frame loads only the nine scalar columns. The Conversation transcript column is inspected with a one-row peek of two columns (ConversationID + Conversation) rather than being loaded for all 2M rows, which keeps the working set at roughly 165 MB instead of the full file — small enough to run on a laptop. The notebook raises a clear error if it cannot find the parquet, and the geography section degrades gracefully if the geo files are absent (a clone that has not run git lfs pull will have pointer files for the two geo parquets).

Figures written to outputs/

All figures are PDF at 300 dpi with an embedded serif font stack.

The map is the one figure not drawn with matplotlib: it is built as Altair layers and exported through vl-convert-python, which is why both are dependencies. Its colours were checked with a palette validator — one single-hue sequential ramp for magnitude, plus two validated categorical hues for datacenter type and a neutral grey for the unlabelled ones.

Intended uses

This dataset exists to evaluate systems that exploit the three attributes. It is a workload for serving research, not a corpus for training or fine-tuning models.

• —Cache-aware scheduling and personalized cache pre-warming
• —Temporal load prediction and diurnal autoscaling
• —Geo-distributed routing and data-sovereignty-compliant placement
• —MoE inference optimization and model-affinity routing
• —Workload-aware resource provisioning

The configurable hyperparameter framework supports stress-testing under alternative behavioural assumptions, and the spatial module can be re-pointed at another city given three inputs — zone boundary definitions, a population distribution, and origin-destination trip records — with no algorithmic changes.

Reproducibility

The figures committed in notebooks/quicklook.ipynb were produced from this repository's uv environment against the released parquet in data/.

Relation to the paper

Where this artifact and the submitted paper differ, the paper is authoritative on method; the numbers below were measured from the released file rather than copied from the paper.

• —Schema. The paper declares nine attributes for the synthesized request tuple $D_{tgt}$. The released parquet has ten columns — see Schema for the column-to-symbol mapping and for the two extra columns (ConversationID, HotspotID).
• —Token maxima. The paper's aggregate-statistics table reports a max input length of 8,000 and a max generation length of 1,500. This file is not truncated at those values: ContextToken reaches 739,859 and GeneratedToken 738,450, with 4,488 and 1,367 rows above the caps respectively. The paper's own token totals confirm the file is uncapped — 926,302,834 input and 357,585,227 generated tokens reproduce exactly on the untruncated columns, whereas truncating at 7,999 / 1,500 would give 897,338,955 and 343,325,061. The 8,000 / 1,500 figures are the caps the analysis applies, not the maxima present in the data.

Limitations

• —Calibrated to New York City. The spatial synthesis is tuned to NYC mobility. Extending it to polycentric cities, car-dependent suburbs, or rural areas requires analogous mobility data and has not been empirically validated.
• —Assumption-dependent. User identity synthesis assumes topical locality — that a user asks about related things over time — which need not hold for users with highly diverse interests. Spatial stationarity is a simplifying approximation.
• —Parameter sensitivity. The DTW composite cost has five tunable weights whose optimal settings are deployment-specific, and the cognitive timing parameters require validation against LLM-specific interaction studies.
• —No instance-level ground truth. Privacy regulations make individual user assignments unverifiable. Fidelity is argued at the distributional level only — see Datasets and sources.
• —`Timestamp` is timezone-naive, and Zone and HotspotID are synthesized rather than observed. All of these are developed further under Caveats.

Caveats

• —Token caps clip heavily. The notebook defaults to CONTEXT_TOKEN_CAP = 7999 and GENERATED_TOKEN_CAP = 1500, matching the source analysis. Observed maxima are far higher — ContextToken reaches 739,859 and GeneratedToken 738,450 — so capped statistics underestimate the tail. The clipping affects 0.22% and 0.07% of requests respectively. Raise the caps if you need the tail.
• —`Timestamp` carries no timezone. It is treated as UTC in the time-bucketed figures. This is an assumption, not a recorded fact.
• —`HotspotID == -1` is a sentinel, not a hotspot — and it is 314,916 rows (15.6%). Dropping it removes a substantial fraction of the data, so decide deliberately before filtering.
• —8,858 rows (0.44%) are incomplete. Model, ContextToken, GeneratedToken and Conversation are null together, suggesting whole requests that failed to complete rather than random field dropout. Model-level aggregates include them as a null group.
• —The map is a static image. It is built in Altair but displayed as a PNG and saved as a PDF, so it renders in any frontend without extra dependencies. Neither form carries the tooltips defined on the chart, so per-zone and per-hotspot values are not queryable in the output — the notebook prints the headline numbers, and the underlying data is one query away.
• —Geography is synthesised, not observed. Zone and HotspotID were assigned by the upstream pipeline from NYC taxi trips, MTA ridership and Wi-Fi hotspot locations — they are not recorded in the requests themselves. The map shows where that synthesis placed the traffic, and is only as trustworthy as the model behind it.
• —Most datacenters have no type. Only 49 of 118 carry one (43 colocation, 6 building); the other 69 are drawn in grey as Unspecified. Zone request counts are also heavily skewed — the median zone serves ~6.5k against a maximum of ~43.5k — so the map's linear colour ramp leaves most zones in its light third.
• —`RequestCount` in the user-behaviour section counts distinct (ConversationID, Turn) pairs, matching the source notebook — not raw row counts.

License

This repository declares no license, deliberately. It is a review artifact: the paper commits to releasing the synthesized dataset and all code under an open-source license upon publication. Until then no license is granted, and the dataset card sets no license field — which is why the Hub renders it as "unknown". Treat the data as review-confidential and do not redistribute it.

That silence does not extend to the upstream sources, which carry their own terms. LMSYS-Chat-1M, the Azure Public Dataset, and the NYC and MTA open-data portals each publish their own licence and attribution requirements, and those govern any reuse of the material they contributed. See Datasets and sources for the links.

One consequence worth stating plainly: the Conversation column contains text originating from LMSYS-Chat-1M. If you redistribute this parquet, you are redistributing that text too.