amazon/chronos-t5-tiny

1---2license: apache-2.03pipeline_tag: time-series-forecasting4tags:5- time series6- forecasting7- pretrained models8- foundation models9- time series foundation models10- time-series11library_name: chronos-forecasting12new_version: amazon/chronos-213---14 15# Chronos-T5 (Tiny)16 17🚀 **Update Feb 14, 2025**: Chronos-Bolt & original Chronos models are now available on Amazon SageMaker JumpStart! Check out the [tutorial notebook](https://github.com/amazon-science/chronos-forecasting/blob/main/notebooks/deploy-chronos-to-amazon-sagemaker.ipynb) to learn how to deploy Chronos endpoints for production use in a few lines of code.18 19🚀 **Update Nov 27, 2024**: We have released Chronos-Bolt⚡️ models that are more accurate (5% lower error), up to 250 times faster and 20 times more memory-efficient than the original Chronos models of the same size. Check out the new models [here](https://huggingface.co/amazon/chronos-bolt-tiny).20 21Chronos is a family of **pretrained time series forecasting models** based on language model architectures. A time series is transformed into a sequence of tokens via scaling and quantization, and a language model is trained on these tokens using the cross-entropy loss. Once trained, probabilistic forecasts are obtained by sampling multiple future trajectories given the historical context. Chronos models have been trained on a large corpus of publicly available time series data, as well as synthetic data generated using Gaussian processes.22 23For details on Chronos models, training data and procedures, and experimental results, please refer to the paper [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815).24 25<p align="center">26  <img src="figures/main-figure.png" width="100%">27  <br />28  <span>29    Fig. 1: High-level depiction of Chronos. (<b>Left</b>) The input time series is scaled and quantized to obtain a sequence of tokens. (<b>Center</b>) The tokens are fed into a language model which may either be an encoder-decoder or a decoder-only model. The model is trained using the cross-entropy loss. (<b>Right</b>) During inference, we autoregressively sample tokens from the model and map them back to numerical values. Multiple trajectories are sampled to obtain a predictive distribution.30  </span>31</p>32 33---34 35## Architecture36 37The models in this repository are based on the [T5 architecture](https://arxiv.org/abs/1910.10683). The only difference is in the vocabulary size: Chronos-T5 models use 4096 different tokens, compared to 32128 of the original T5 models, resulting in fewer parameters.38 39| Model                                                                  | Parameters | Based on                                                               |40| ---------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------- |41| [**chronos-t5-tiny**](https://huggingface.co/amazon/chronos-t5-tiny)   | 8M         | [t5-efficient-tiny](https://huggingface.co/google/t5-efficient-tiny)   |42| [**chronos-t5-mini**](https://huggingface.co/amazon/chronos-t5-mini)   | 20M        | [t5-efficient-mini](https://huggingface.co/google/t5-efficient-mini)   |43| [**chronos-t5-small**](https://huggingface.co/amazon/chronos-t5-small) | 46M        | [t5-efficient-small](https://huggingface.co/google/t5-efficient-small) |44| [**chronos-t5-base**](https://huggingface.co/amazon/chronos-t5-base)   | 200M       | [t5-efficient-base](https://huggingface.co/google/t5-efficient-base)   |45| [**chronos-t5-large**](https://huggingface.co/amazon/chronos-t5-large) | 710M       | [t5-efficient-large](https://huggingface.co/google/t5-efficient-large) |46 47## Usage48 49To perform inference with Chronos models, install the package in the GitHub [companion repo](https://github.com/amazon-science/chronos-forecasting) by running:50 51```52pip install git+https://github.com/amazon-science/chronos-forecasting.git53```54 55A minimal example showing how to perform inference using Chronos models:56 57```python58import matplotlib.pyplot as plt59import numpy as np60import pandas as pd61import torch62from chronos import ChronosPipeline63 64pipeline = ChronosPipeline.from_pretrained(65  "amazon/chronos-t5-tiny",66  device_map="cuda",67  torch_dtype=torch.bfloat16,68)69 70df = pd.read_csv("https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv")71 72# context must be either a 1D tensor, a list of 1D tensors,73# or a left-padded 2D tensor with batch as the first dimension74context = torch.tensor(df["#Passengers"])75prediction_length = 1276forecast = pipeline.predict(context, prediction_length)  # shape [num_series, num_samples, prediction_length]77 78# visualize the forecast79forecast_index = range(len(df), len(df) + prediction_length)80low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0)81 82plt.figure(figsize=(8, 4))83plt.plot(df["#Passengers"], color="royalblue", label="historical data")84plt.plot(forecast_index, median, color="tomato", label="median forecast")85plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")86plt.legend()87plt.grid()88plt.show()89```90 91## Citation92 93If you find Chronos models useful for your research, please consider citing the associated [paper](https://arxiv.org/abs/2403.07815):94 95```96@article{ansari2024chronos,97    title={Chronos: Learning the Language of Time Series},98    author={Ansari, Abdul Fatir and Stella, Lorenzo and Turkmen, Caner and Zhang, Xiyuan, and Mercado, Pedro and Shen, Huibin and Shchur, Oleksandr and Rangapuram, Syama Syndar and Pineda Arango, Sebastian and Kapoor, Shubham and Zschiegner, Jasper and Maddix, Danielle C. and Mahoney, Michael W. and Torkkola, Kari and Gordon Wilson, Andrew and Bohlke-Schneider, Michael and Wang, Yuyang},99    journal={Transactions on Machine Learning Research},100    issn={2835-8856},101    year={2024},102    url={https://openreview.net/forum?id=gerNCVqqtR}103}104```105 106## Security107 108See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.109 110## License111 112This project is licensed under the Apache-2.0 License.