stargatek1/ssm-metarl-demo

SSM-MetaRL-Unified: Experience-Augmented Meta-RL Demo

This interactive demo showcases the SSM-MetaRL-Unified framework, which integrates State Space Models (SSM) with Meta-Learning (MAML) for efficient reinforcement learning with fast adaptation capabilities.

🎯 Key Features

This demo allows you to compare two test-time adaptation strategies:

1. 1.Standard Adaptation: Baseline approach that adapts using only current task data
2. 2.Hybrid Adaptation: Novel approach that augments adaptation with past experiences from a replay buffer

The hybrid approach demonstrates more robust and sample-efficient learning through experience replay!

🔬 What You'll See

• —Meta-Training: Train the SSM model using MetaMAML for fast adaptation capability
• —Standard Adaptation: Observe how the model adapts using only current observations
• —Hybrid Adaptation: See the improvement when leveraging past experiences
• —Visual Comparison: Interactive plots comparing both adaptation strategies

🏗️ Framework Components

• —State Space Models (SSM): Efficient sequence modeling for temporal dynamics
• —Meta-Learning (MAML): Model-Agnostic Meta-Learning for rapid adaptation
• —Experience Buffer: Stores and replays past experiences during adaptation
• —Test-Time Adaptation: Both standard and hybrid strategies

💡 How to Use

1. 1.Setup: Configure the environment and model parameters
2. 2.Meta-Train: Run meta-training to prepare the model for fast adaptation
3. 3.Compare: Execute both Standard and Hybrid adaptation modes
4. 4.Analyze: Examine the loss curves and performance metrics

🔗 Links

• —GitHub Repository: sunghunkwag/SSM-MetaRL-Unified
• —Documentation: See the repository README for detailed information
• —Paper: Coming soon!

📊 Technical Details

The demo uses the CartPole-v1 environment from Gymnasium and trains an SSM model to predict next observations. The comparison highlights how experience-augmented adaptation can lead to:

• —Faster convergence
• —More stable learning
• —Better sample efficiency
• —Improved robustness

🎓 Citation

If you use this framework in your research, please cite:

@software{ssm_metarl_unified,
  title = {SSM-MetaRL-Unified: Experience-Augmented Meta-RL},
  author = {Kwag, Sung Hun},
  year = {2025},
  url = {https://github.com/sunghunkwag/SSM-MetaRL-Unified}
}

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

Built with ❤️ using Gradio, PyTorch, and Gymnasium