stevevaius/seasonal-forecast-patchtst-cyclenet

📊 Monthly Seasonal Forecast Model (SOTA 2025)

State-of-the-art monthly seasonal forecasting combining the latest innovations from time series research. Competitive with the Chronos-Bolt foundation model (120M+ params) while using only ~2.5M parameters.

🏆 Results on M4 Monthly (48,000 test series)

Our lightweight ensemble achieves OWA=0.794, within 0.9% of the SOTA Chronos-Bolt foundation model that has 20x more parameters.

🔬 Architecture

CycleNet (MLP + Residual Cycle Forecasting)

Based on CycleNet:

• —Learns a 12-month periodic cycle parameter
• —Subtracts learned cycle → forecasts residuals → adds future cycle
• —RevIN (Reversible Instance Normalization) for distribution shift
• —4-layer MLP backbone with GELU activation

SeasonalPatchTST (Transformer + RCF)

Combines PatchTST with CycleNet innovations:

• —12-month patches aligned with annual seasonality
• —CLS token + 4-layer Transformer encoder with 8 attention heads
• —CycleNet RCF decomposition + RevIN normalization
• —Pre-LN architecture for training stability

Learned Ensemble

• —Sigmoid-gated weighted average of CycleNet + PatchTST
• —Weight learned on validation set

📈 Key Innovations

1. 1.Residual Cycle Forecasting (RCF): From CycleNet — learns W=12 annual cycle, forecasts residuals
2. 2.Seasonal Patching: 12-month patch size matched to annual cycle (vs typical 16 or 32)
3. 3.RevIN Normalization: Handles diverse scales across 48K series (Macro, Finance, Demographics)
4. 4.Value-flipping + Scaling Augmentation: From Sundial (ICML 2025 Oral)
5. 5.CLS Token Aggregation: Global representation for multi-step forecasting

🚀 Usage

📚 Training Details

• —Dataset: M4 Monthly (48,000 series from autogluon/chronos_datasets)
• —Context: 48 months → Predict 18 months
• —Optimizer: AdamW (lr=1e-3 CycleNet / 5e-4 PatchTST, weight_decay=0.01)
• —Schedule: Cosine annealing
• —Early stopping: Patience=12, best val MSE checkpoint
• —Augmentation: Value-flipping (10%), random scaling ±20%
• —288K training windows from sliding window extraction

🌍 Comparison with Foundation Models (2025 SOTA)

📄 References

• —Chronos-2: From Univariate to Universal Forecasting
• —Sundial: Time Series Foundation Models (ICML 2025 Oral)
• —CycleNet: Enhancing TSF through Periodic Patterns
• —PatchTST: A Time Series is Worth 64 Words
• —Timer-S1: Billion-Scale Foundation Model
• —M4 Competition