ahmed-farhanur-rashid/hpa-pp-synthetic-telemetry

HPA++ Multi-Cluster Synthetic Telemetry Benchmark Suite

Dataset Summary

The HPA++ Multi-Cluster Synthetic Telemetry Benchmark Suite (hpa-pp-synthetic-telemetry) is a high-fidelity synthetic multivariate time-series benchmark designed for research on predictive Kubernetes Horizontal Pod Autoscaling (HPA++), cloud resource forecasting, and AIOps.

The benchmark contains one year of telemetry sampled every 5 minutes across five realistic Kubernetes workload profiles, producing over 525,000 observations. Each workload exhibits unique temporal dynamics, seasonal patterns, infrastructure behavior, and injected operational failures, enabling evaluation under both normal and adverse operating conditions.

Unlike purely statistical synthetic datasets, HPA++ combines realistic workload seasonality with infrastructure-aware telemetry such as CPU, memory, GPU utilization, active pods, user concurrency, and injected operational events.

This dataset is entirely synthetic. No production telemetry, user data, or proprietary infrastructure traces are included. Statistical properties were derived from publicly available open datasets listed in the attribution section.

Key Characteristics

• —365 days of telemetry
• —5-minute sampling interval
• —525,600 total observations
• —105,120 observations per workload profile
• —16 telemetry features
• —Multiple correlated infrastructure metrics
• —Event annotations for anomaly detection
• —Held-out out-of-distribution evaluation split
• —Suitable for forecasting, anomaly detection, predictive autoscaling, and multivariate sequence modeling

Quickstart

Load using Hugging Face Datasets

from datasets import load_dataset

dataset = load_dataset(
    "ahmed-farhanur-rashid/hpa-pp-synthetic-telemetry",
    "all_clusters"
)

df = dataset["train"].to_pandas()

Load a specific workload profile:

from datasets import load_dataset

df = load_dataset(
    "ahmed-farhanur-rashid/hpa-pp-synthetic-telemetry",
    "genai_inference"
)["train"].to_pandas()

Load the held-out evaluation dataset:

from datasets import load_dataset

test = load_dataset(
    "ahmed-farhanur-rashid/hpa-pp-synthetic-telemetry",
    "shifted_test"
)["test"].to_pandas()

Load directly with Pandas

import pandas as pd

url = "https://huggingface.co/datasets/ahmed-farhanur-rashid/hpa-pp-synthetic-telemetry/raw/main/data/synthetic_hpa_traffic_all_clusters_365d.csv"

df = pd.read_csv(
    url,
    parse_dates=["timestamp"]
)

Dataset Configurations

Dataset Schema

Workload Profiles

E-Commerce

Moderate baseline traffic with flash-sale events producing short, high-amplitude request bursts.

University Portal

Predictable admission and examination result publication events generating large synchronized traffic spikes.

Streaming Platform

High continuous traffic with additional spikes corresponding to live broadcasts and popular releases.

Examination System

CPU-intensive synchronized login behavior typical of online examinations.

GenAI Inference

GPU-heavy serving workload inspired by modern LLM inference clusters and accelerator utilization traces.

Shifted Test Set

The shifted_test configuration provides a 30-day held-out out-of-distribution benchmark.

Compared to the training data, it includes:

• —altered seasonal amplitudes
• —shifted event timings
• —different burst frequencies
• —modified workload intensity
• —increased stochastic variance

This split is intended for evaluating model robustness and generalization rather than interpolation.

Generation Methodology

Telemetry was generated through a hybrid statistical and systems-aware simulation pipeline combining:

• —long-term trend modeling
• —daily and weekly seasonal decomposition
• —Ornstein-Uhlenbeck AR(1) temporal correlation
• —heavy-tailed Student-t burst noise
• —Pareto-distributed workload churn
• —nonlinear resource saturation
• —Kubernetes-inspired reactive autoscaling
• —correlated CPU, memory, GPU, and pod dynamics
• —operational failure injection

Injected operational scenarios include:

• —partial service outages
• —memory leaks
• —flash demand events
• —workload churn
• —resource saturation
• —overload plateaus

A complete mathematical description of the simulation framework is available in:

• —METHODOLOGY.md
• —ATTRIBUTION.md

Intended Uses

This benchmark is intended for research involving:

• —Predictive Horizontal Pod Autoscaling
• —Time-series forecasting
• —Multivariate forecasting
• —Cloud resource prediction
• —Kubernetes scheduling research
• —AIOps
• —Capacity planning
• —Anomaly detection
• —Failure prediction
• —Synthetic benchmark evaluation

Limitations

Although statistically grounded, this dataset remains synthetic.

Specifically:

• —It does not represent telemetry from any production Kubernetes deployment.
• —Network latency, storage I/O, disk utilization, and service-level latency are not modeled.
• —Operational events are probabilistically generated rather than replayed from real incidents.
• —The benchmark is intended for algorithm evaluation rather than production performance estimation.

Reproducibility

The complete synthetic data generation pipeline is included within the repository.

generator/

Running the generator reproduces the benchmark using deterministic configuration parameters and fixed random seeds. Individual workload profiles may also be regenerated independently.

Data Provenance

Statistical characteristics were derived from publicly available datasets.

1. 1.Wikipedia Web Traffic Forecasting Dataset (CC BY-SA 3.0)
2. 2.SenseTime Helios GPU Cluster Trace (SC'21)
3. 3.Google Borg Production Cluster Trace (CC BY 4.0)
4. 4.Alibaba ClusterData GenAI Trace (Apache 2.0 / CC BY 4.0)

These datasets were used solely to estimate statistical properties such as seasonality, temporal correlation, burst distributions, resource utilization, and workload dynamics. No original records are redistributed.

Citation

@dataset{hpa_pp_synthetic_telemetry_2026,
  author    = {Ahmed Farhanur Rashid},
  title     = {HPA++ Multi-Cluster Synthetic Telemetry Benchmark Suite},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/ahmed-farhanur-rashid/hpa-pp-synthetic-telemetry}
}

License

This dataset is released under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.