michel-pohl/online_rnn_time_series_forecasting

Online training of Vanilla RNNs for Time-Series Forecasting

This page documents the reference implementation used in the following paper on online training of vanilla RNNs for medical time-series forecasting.

Paper

• —Title: Real-time respiratory motion forecasting with online learning of recurrent neural networks for accurate targeting in externally guided radiotherapy
• —Authors: Michel Pohl, Mitsuru Uesaka, Hiroyuki Takahashi, Kazuyuki Demachi, Ritu Bhusal Chhatkuli
• —arXiv: https://doi.org/10.48550/arXiv.2403.01607
• —Published version: https://doi.org/10.1016/j.cmpb.2025.108828
• —Hugging Face paper page: https://huggingface.co/papers/2403.01607

Code

• —GitHub repository: https://github.com/pohl-michel/2D-MR-image-prediction/blob/main/Timeseriesforecasting/README.md
• —Latest release (Zenodo): https://doi.org/10.5281/zenodo.13896201

Note: only the "Timeseriesforecasting" folder of the repository was used for this paper.

Description

This works compares online learning algorithms applied to vanilla RNNs, namely, real-time recurrent learning, unbiased online recurrent optimization, sparse-one step optipization (SnAp-1), and decoupled neural interfaces (DNI). We propose an efficient implementation for SnAp-1, that compresses matrices intervening in the update equations into a non-sparse format to reduce memory complexity and inference time. Regarding DNI, we improve the procedure to estimate credit assignment by deriving an accurate closed-form expression for the corresponding update step. We experimentally investigate hyperparameter optimization, inference time, and the influence of the prediction horizon, sampling frequency, and original signal regularity on the accuracy and oscillatory behavior of the predictions. Even when training and testing sequence-wise on a small dataset with several irregular breathing records, online-trained RNNs achieved forecasting accuracy comparable to prior deep learning approaches trained on substantially larger datasets.

Citation

@article{pohl2025real,
  title={Real-time respiratory motion forecasting with online learning of recurrent neural networks for accurate targeting in externally guided radiotherapy},
  author={Pohl, Michel and Uesaka, Mitsuru and Takahashi, Hiroyuki and Demachi, Kazuyuki and Chhatkuli, Ritu Bhusal},
  journal={Computer Methods and Programs in Biomedicine},
  pages={108828},
  year={2025},
  publisher={Elsevier}
}