FlowVortex/SymTime

1---2license: apache-2.03metrics:4- mse5- mae6tags:7- time series8- forecasting9- foundation models10- pretrained models11- generative models12- time series foundation models13library_name: transformers14language:15- en16---17 18# SymTime NeurIPS 202519 20This code is the official PyTorch implementation of our NeurIPS'25 paper: **Synthetic Series-Symbol Data Generation for Time Series Foundation Models**.21 22<div align="center">23  24[Paper](https://arxiv.org/abs/2510.08445) | [Poster](https://github.com/wwhenxuan/wwhenxuan.github.io/blob/main/assets/img/poster_neurips_2025_115260_synthetic_series-symbol_data_generation.jpg) | [Blog](https://mp.weixin.qq.com/s/D6O5SBl2RYHdkiinV6UM8w) | [Video](https://www.bilibili.com/video/BV1RT4QzXECt/?spm_id_from=333.337.search-card.all.click) | [PPT](https://github.com/wwhenxuan/wwhenxuan.github.io/blob/main/assets/files/NeurIPS_2025_SymTime_video_en.pptx) | [Citation](#Citation) | [HF 🤗](https://huggingface.co/FlowVortex/SymTime)25 26</div>27 28This repository contains the official Hugging Face / PyTorch implementation of **SymTime** from our NeurIPS 2025 paper, *Synthetic Series-Symbol Data Generation for Time Series Foundation Models*.29 30## Overview31 32SymTime is a lightweight time series foundation model designed to learn strong temporal representations from patch-based inputs. It is built for practical downstream use and supports easy loading through the Hugging Face `AutoModel` interface.33 34<div style="text-align: center;">35    <img src="https://raw.githubusercontent.com/wwhenxuan/SymTime/main/configs/images/S2Generator_SymTime.png" alt="SymTime" style="zoom:80%;" />36</div>37 38The model takes a univariate time series, splits it into patches, and encodes the patch sequence with a transformer backbone. The repository includes the configuration, model definition, and a runnable example for inference.39 40## Quick start41 42### Install dependencies43 44```bash45pip install -r requirements.txt46```47 48### Load the model49 50```python51from transformers import AutoModel52 53model = AutoModel.from_pretrained("FlowVortex/SymTime", trust_remote_code=True)54```55 56### Run inference57 58```python59import torch60 61x = torch.randn(16, 256)62out = model(x)63out_no_cls = model(x, return_cls_token=False)64```65 66## Model summary67 68- Input: `Tensor` with shape `[batch_size, seq_length]`69- Output: patch embeddings, optionally with a CLS token output70- Backend: patch-based transformer encoder71 72## Citation <a id="Citation"></a>73 74If you find this code useful, please cite our paper.75 76```77@misc{wang2025syntheticseriessymboldatageneration,78      title={Synthetic Series-Symbol Data Generation for Time Series Foundation Models}, 79      author={Wenxuan Wang and Kai Wu and Yujian Betterest Li and Dan Wang and Xiaoyu Zhang},80      year={2025},81      eprint={2510.08445},82      archivePrefix={arXiv},83      primaryClass={cs.LG},84      url={https://arxiv.org/abs/2510.08445}, 85}86```87 88## Contact89 90If you have any questions or are interested in our view on the complex dynamics of time series, feel free to contact:91 92- [Whenxuan Wang](https://wwhenxuan.github.io/) (whenxuanwang@stu.xidian.edu.cn)93- [Kai Wu](https://sparsel.github.io/index.html) (kwu@xidian.edu.cn)94- [Dan Wang](https://web.xidian.edu.cn/danwang/) (danwang@xidian.edu.cn)95 96## Acknowledgement97 98We appreciate the following GitHub repos a lot for their valuable code and efforts.99 100- Time-Series-Library (https://github.com/thuml/Time-Series-Library)101- PySDKit (https://github.com/wwhenxuan/PySDKit)102- ALBEF (https://github.com/salesforce/ALBEF)103- PatchTST (https://github.com/yuqinie98/PatchTST)104- Short-term Forecasting (https://github.com/ServiceNow/N-BEATS)