SoroushVahidi/llm-serving-scheduler-baselines

LLM-Serving Scheduler Baselines: Simulation Performance Outcomes for Scheduler Policies

This is a comprehensive, text-free, highly structured simulation results dataset for large language model (LLM) serving schedulers. It contains policy-level outcome records generated across synthetic scheduler stress tests and an added TraceLab-derived out-of-distribution policy sweep. The dataset compares 12 highly optimized third-party baseline schedulers against APT-Serve (a transition-aware preemption-capable scheduler), complete with enriched nullable cache metrics (apt_stats).

This is dataset version v1.1, distributed through the canonical Hugging Face repository <https://huggingface.co/datasets/SoroushVahidi/llm-serving-scheduler-baselines>.

What One Row Means

In per_policy_results, one row represents the final end-to-end simulated performance metrics of one scheduler policy executing on one traffic seed under one specific workload regime/envelope.

Configurations

• —per_policy_results: 36,975 rows. The flat main results table.
• —tracelab_scheduler_ood_policy_sweep: 13,824 rows. A sanitized TraceLab-derived OOD scheduler policy sweep with one row per scheduler policy evaluated on one derived TraceLab workload window.

Quickstart

from datasets import load_dataset

# Main scheduler-policy outcome table
outcomes = load_dataset("SoroushVahidi/llm-serving-scheduler-baselines", "per_policy_results")
print(outcomes)

# TraceLab-derived out-of-distribution policy sweep
ood_sweep = load_dataset("SoroushVahidi/llm-serving-scheduler-baselines", "tracelab_scheduler_ood_policy_sweep")
print(ood_sweep)

TraceLab OOD Policy Sweep Config

tracelab_scheduler_ood_policy_sweep adds 512 derived TraceLab workload windows evaluated under 27 scheduler policies, for 13,824 policy-outcome rows. Each row is a Soroush-generated simulator result for one scheduler policy on one derived, non-raw TraceLab workload scenario. The config is intended for out-of-distribution scheduler-policy analysis; it is not selector-regret data and does not contain module-intervention-credit records.

The public table removes TraceLab session identifiers, raw row spans, source-file row ancestry, and request-level content. It retains nonreversible scenario identifiers, coarse source attribution, workload split labels, policy labels, and simulation metrics such as completion fraction, latency, TTFT/TPOT, SLO violation rate, goodput, throughput, GPU utilization, and policy runtime.

Workload coverage: 512 TraceLab-derived windows; 320 train/development windows, 64 temporal OOD windows, 64 provider/model OOD windows, and 64 final untouched OOD windows. Provider/model coverage is 390 Claude-family and 122 Codex-family derived windows. Policy coverage is 20 existing policy-library entries and 7 v2 policy-library entries.

Workload Regimes

The dataset represents evaluations across 41 diverse, highly challenging workload envelopes designed to Stress-Test scheduling architectures. These are grouped into:

• —Baseline Pressure: pressure_low_baseline, pressure_medium_baseline, pressure_high_baseline, pressure_near_capacity_baseline, pressure_sustained_overload_baseline.
• —Arrival Patterns: arrival_steady, arrival_burst, arrival_clustered_burst, arrival_spike, arrival_alternating.
• —Sequence Length Shifts: length_homogeneous, length_bimodal, length_prompt_heavy, length_decode_heavy, length_heavy_tail_prompt, length_heavy_tail_output.
• —Cache Pressure and Thrashing Risk: cacheuse_recompute_avoidance, cacheuse_kv_to_hidden_opportunity, cacheuse_hidden_to_kv_opportunity, cacheuse_thrash_risk_high, cacheuse_thrash_risk_medium, cacheuse_thrash_risk_near_capacity.

Scheduler/Method Taxonomy

The main per_policy_results config logs the exact comparative performance of 13 policies:

1. 1.FIFO (`fifo`): First-In-First-Out baseline.
2. 2.EDF (`edf`): Earliest-Deadline-First deadline-aware scheduler.
3. 3.WSP (`weighted_shortest_processing`): Weighted Shortest Processing Time First.
4. 4.LLF (`least_laxity_first`): Least Laxity First.
5. 5.ESTF (`estimated_service_time_first`): Shortest Processing Time First variant.
6. 6.Scorpio (`scorpio_style_slo_guard`): Scorpio-style SLO-guaranteed queue manager (SOSP 2023).
7. 7.vLLM (`vllm_style_token_budget`): vLLM-style memory-budget scheduler with reemption-on-exhaustion (SOSP 2023).
8. 8.Sarathi (`sarathi_style`): Sarathi-style chunked-prefill scheduler (OSDI 2024 / CoRR 2023).
9. 9.Orca (`orca_style`): Orca-style continuous batching iteration-level scheduler (OSDI 2022).
10. 10.Shortest Output First (`shortest_output_first`): Prioritizes short generation outputs.
11. 11.SLO Slack Score (`slo_slack_score`): Prioritizes requests based on remaining SLO slack.
12. 12.Admission Control (`admission_control`): Standard request filtering baseline.
13. 13.APT-Serve (`apt_serve_faithful`): Preemption-aware, transition-optimal scheduler developed by Soroush Vahidi, evaluated under 5 distinct switch-penalty/transition-cost multipliers (0x, 0.5x, 1x, 2x, 4x).

Data Dictionary

Simulator and Environment Assumptions

Metrics are logged from Soroush's high-fidelity event-driven LLM serving simulator.

• —GPU models are parameterized with homogeneous memory capacities.
• —Preemption is modeled via recomputation or swap-to-host.
• —Transition-overhead factors (switch_latency_paid, restore_latency_paid) reflect physical hardware swap penalties under strict bus-width assumptions.

Scientific Use

This dataset allows researchers to perform deep comparative analyses of LLM-serving schedulers under severe traffic pressure, study Pareto throughput/goodput curves, model optimal preemption thresholds, and validate state-of-the-art scheduling policies without re-scaffolding complex event-driven simulations.

Non-Intended Uses

• —Treating simulator latency, TTFT, or throughput numbers as calibrated production SLO predictions.
• —Claiming production superiority of any policy based solely on these tables.
• —Using tracelab_scheduler_ood_policy_sweep as selector-training or oracle-regret data (see SoroushVahidi/llm-serving-selector-regret for that purpose).
• —Mixing rows across configs as if they share the same workload grain or trace source.

Limitations

Simulator scope (`per_policy_results`). All rows are outputs of the event-driven LLM-serving simulator in `llm-serving-heuristic-evolution`, not measurements from a live GPU cluster. Workloads are synthetic stress envelopes (41 parametric regimes across pressure, arrival, length, and cache-structure dimensions), not raw production-trace replay. The source project's public-trace replay studies found that average production-style windows can be nondiscriminative under a frozen replay setup (compared policies tied at the primary goodput metric), which motivated these targeted stress scenarios — but the stress regimes must not be read as claims about typical datacenter traffic frequencies.

Policy abstraction. Most policies here are simulator-level inspired approximations of published schedulers (Orca/vLLM/Sarathi/Scorpio style implementations faithful to this simulator's abstractions, not byte-level reproductions of those production systems). Comparative rankings and metric gaps are valid relative to this simulator and policy library, not certificates of datacenter superiority.

Hardware and runtime modeling. The simulator assumes homogeneous GPU memory, models preemption through recomputation or host swap, and reports transition penalties (switch_latency_paid, restore_latency_paid) under fixed bus-width assumptions. Absolute latency, TTFT, throughput, and goodput values are simulator-relative; cross-hardware extrapolation is unsupported.

Evaluation design. Rows include screening and confirmation stages, multiple random seeds per regime, and APT-Serve under five transition-cost multipliers while baselines use a single configuration. Policies may drop requests under overload — compare using the documented metrics (completion fraction, SLO violation rate, weighted goodput) rather than assuming identical admission behavior.

TraceLab OOD config (`tracelab_scheduler_ood_policy_sweep`). Workload windows are derived from public TraceLab coding-agent traces (SyFI Lab; CC BY 4.0); raw session content and identifiers are not redistributed. Windows are session-scoped and relatively short (the source sweep uses at most ~128 requests per window), with sqrt-compressed prompt-token inputs and synthetic neutral SLO labels added for simulator compatibility — these are not natural TraceLab deadlines. The generating sweep reported near-saturated policy separation on this 512-window set (oracle goodput ≈ 1.0, zero marginal gain for newer policy-library entries). Treat this config as an OOD coverage / diagnostic benchmark for scheduler-policy behavior on agentic trace-derived windows, not as proof of production scheduler ranking or as a substitute for selector-regret training data.

What this dataset can support. Relative scheduler comparison under declared synthetic stress and TraceLab-derived OOD windows; studying trade-offs among completion, SLO violations, goodput, and (for APT-Serve) cache-transition metrics within the simulator.

What it cannot establish. Universal production optimality, real-cluster SLO compliance, or generalization to trace regimes and hardware not represented here.

Provenance

Generated by Soroush Vahidi from the llm-serving-heuristic-evolution simulator (<https://github.com/SoroushVahidi/llm-serving-heuristic-evolution>). The exact source commit recorded in this release's build metadata (metadata/provenance.json's source_git_commit) does not currently resolve in that repository's git history (checked against every local clone, all branches, reflogs, and dangling objects, and independently against the public GitHub API) — a documentation gap the maintainer intends to close, not evidence that the underlying simulation results are unreliable or fabricated. The released data's authenticity is independently supported by: internally consistent checksums and a passing build-time security scan (both preserved unchanged in metadata/), and policy implementations that match this same repository's own baseline-vllm-faithful, baseline-sarathi-faithful, and related dedicated baseline branches. The original, unresolving commit reference is preserved as historical build evidence rather than silently rewritten.

Relationship to Related Datasets

• —`SoroushVahidi/module-intervention-credit` is specifically designed for AutoML counterfactual credit assignments in modular pipeline networks — which internal module of a policy caused a performance difference. This dataset is a monolithic baseline scheduler evaluation benchmark comparing 13 full-system scheduling algorithms across 41 extreme traffic envelopes, not a module-level attribution study.
• —`SoroushVahidi/llm-serving-selector-regret` is the dataset to use if your question is which policy should be selected for a given workload window (selector training, oracle/regret labels, policy-suitability scoring). This dataset (llm-serving-scheduler-baselines) is the one to use if your question is how does a fixed policy perform under a stress-test workload envelope — it reports end-to-end outcomes for policies run as-is, not selector training data, oracle labels, or regret metrics. The two datasets share some policy names (e.g. fifo, edf, weighted_shortest_processing) because both draw from the same underlying scheduler-policy library, but they answer different questions, use different row grains, and neither supersedes the other.

Creator / Provider

Dataset creator/provider for the released derived metrics: Soroush Vahidi. This statement applies to the simulator outputs and derived scheduler-evaluation metrics released in this Hugging Face dataset; it does not claim ownership of upstream scheduler papers, scheduler implementations, or third-party workload/trace sources.

Upstream Workload / Trace Sources

This Hugging Face release contains derived scheduler-evaluation metrics generated by Soroush Vahidi. Underlying workload/trace sources remain subject to their original providers and licenses.

License and Attribution

The released Soroush-generated derived metrics in this repository are distributed under CC BY 4.0. Attribution is required. Upstream workload/trace sources must also be attributed according to their original provider terms. This repository does not relicense or imply ownership of third-party upstream traces.

Associated Paper and Citation Policy

No directly verified preprint paper is associated with this dataset release yet.

Citation Guidance:

• —Cite this dataset when utilizing these simulation rows or benchmarking comparisons.
• —Cite upstream scheduler papers (such as vLLM [SOSP '23], Orca [OSDI '22], Sarathi-Serve [OSDI '24], or Scorpio [SOSP '23]) when discussing or relying on the baseline models implemented here.

@dataset{vahidi2026_llm_serving_scheduler_baselines,
  author = {Vahidi, Soroush},
  title = {LLM-Serving Scheduler Baselines: Event-Driven Simulation Performance Outcomes for Scheduler Policies},
  year = {2026},
  version = {v1.1},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/SoroushVahidi/llm-serving-scheduler-baselines},
  license = {CC-BY-4.0}
}