Shivam3002/meter-load-forecasting-transformer

meter-load-forecasting-transformer

Small Transformer (1.46M params) trained from scratch to forecast household electricity load 24 hours ahead from 168 hours (1 week) of history.

• —Task: 24h-ahead electricity load forecasting
• —Input: last 168 hourly readings of Global_active_power (kW)
• —Output: next 24 hourly readings (kW)
• —Dataset: UCI Individual Household Electric Power Consumption, resampled to hourly means
• —Architecture: Transformer encoder (2 layers, d_model=64, 4 heads, ffn=128) + linear regression head
• —Params: 1,457,304
• —Framework: PyTorch (no external ML framework dependency)

Results

Trained 8 epochs, ~104s on Apple Silicon (MPS).

Full training log and config: run_info.json, config.json. Code: https://github.com/shivam2003-dev/meter-load-forecasting-transformer

Usage

import json
import torch
from huggingface_hub import hf_hub_download

repo_id = "Shivam3002/meter-load-forecasting-transformer"
cfg = json.load(open(hf_hub_download(repo_id, "config.json")))
weights_path = hf_hub_download(repo_id, "pytorch_model.bin")

# rebuild TinyForecastTransformer class from train.py in the GitHub repo, then:
# model = TinyForecastTransformer(d_model=cfg["d_model"], nhead=cfg["n_head"],
#                                  num_layers=cfg["n_layers"], dim_ff=cfg["dim_ff"],
#                                  input_len=cfg["input_hours"], pred_len=cfg["pred_hours"])
# model.load_state_dict(torch.load(weights_path))
# model.eval()

# normalize input window with cfg["normalization"]["mean"] / ["std"] before inference,
# and de-normalize the output the same way.

Limitations

Trained on a single household's meter data — not tuned for other households, regions, or seasonal patterns beyond what the dataset (2006-2010, France) covers. Small model and short training run; intended as a lightweight demo/baseline, not a production forecaster.