Modelpublic

YongdongWang/emergent-snake-locomotion

sourceHugging Facemitupdated 5mo agoView on Hugging Face

0likes

Model Card

Emergent Locomotion Patterns of a Snake Robot through Reinforcement Learning

Authors: Yuya Shimizu¹, Yongdong Wang^{1,2}, So Shimooka¹, Tetsushi Kamegawa¹ ¹ Graduate School of Environmental, Life, Natural Science and Technology, Okayama University ² Department of Precision Engineering, The University of Tokyo Project Page: https://yuya-0411.github.io/project-crawl/ Year: 2026

This work was supported by OU-SPRING, JSPS KAKENHI Grant Number 26K07423, and JSPS KAKENHI Grant Number 26K21350.

Abstract

Traditional gait design for snake robots has long relied on geometric models inspired by biological snakes. However, due to the inherent morphological discrepancies between biological snakes and robotic systems, these conventional models are not always optimal for snake robots.

In this study, we trained a 10-DoF snake robot on random step fields using reinforcement learning. The learned policy autonomously emerged dynamic gaits capable of traversing unstructured terrains where conventional sine-wave-based locomotion typically fails. The robot effectively utilized environmental protrusions as pivot points for propulsion, three-dimensionally transforming its body to overcome obstacles.

Although these gaits deviate from predefined periodic motions, they exhibit characteristics approximating biological sidewinding as a direct result of physical constraints. Comparative experiments with geometric models confirm that the learned policy offers superior robustness and adaptability in rugged terrains.

Robot Model

—Structure: 11 links connected by 10 alternating yaw/pitch joints (orthogonally arranged)
—Length: 9 central units × 0.16 m + 2 terminal units × 0.08 m = 1.6 m total
—Sensors: Each link is equipped with a tactile sensor for contact-aware state feedback

Method

Observation & Action Space

The policy takes a 56-dimensional observation as input and outputs 10 joint position targets.

Category	Item	Dimension
Observation	Joint positions	10
	Joint velocities	10
	Per-link contact forces (11 links × 3 axes, in local frame)	33
	Global orientation (roll, pitch, yaw)	3
	Total	56
Action	Joint position targets	10

Per-link contact forces are the key embodiment-sensing component — each link's resultant contact force is expressed in its local coordinate frame, capturing normal and tangential friction. Joint targets are scaled as: q_target = (π/2) × a, where a ∈ [−1, 1].

Reward Function

The total reward combines six terms:

Term	Weight	Purpose
Forward velocity	5.0 × exp(−(0.5 − v_y)² / 0.25)	Encourage target speed of 0.5 m/s
Lateral deviation	−0.5 × \	x_com\	Penalize drifting sideways
Trajectory correction	+1.0 × max(0, −xcom × vx)	Self-correct when off-track
Ground contact	+0.2 × N_contact	Reward multi-link ground contact
Head height	−10.0 if z_head > 0.2 m	Enforce low center-of-gravity posture
Action smoothness	−0.005 × Σ(Δa_i)²	Penalize large joint movements

Training Setup

Parameter	Value
Algorithm	PPO (RSL-RL)
Simulator	NVIDIA Isaac Lab (GPU-accelerated PhysX, 50 Hz)
Parallel environments	4,000 (10 × 20 terrain field, 20 agents/area)
Network	MLP [256, 256], ELU activation
Learning rate	2.5 × 10⁻⁵
Discount factor (γ)	0.99
GAE lambda (λ)	0.95

Training Terrains (4 types)

Type	Description	Parameter Range
Pyramidal Stairs & Pits	Multi-level elevation changes	Step height 0.05–0.23 m
Random Grid (Boxes)	Discontinuous rigid obstacles	Grid height 0.05–0.20 m
Random Rough Surface	Stochastic height-field noise	Noise range 0.02–0.10 m
Pyramidal Slopes	Continuous inclined surfaces	Slope 0.0–0.4 rad

Checkpoint

`model_3000.pt` — PPO policy weights at 3,000 training iterations.

python

import torch

checkpoint = torch.load("model_3000.pt", map_location="cpu")
print(checkpoint.keys())

For full simulation and evaluation setup, see the project page.

Key Findings

—The policy emerged a gait approximating biological sidewinding purely from physical constraints — no gait template was provided.
—The robot exploits terrain protrusions as pivot points for propulsion, reconfiguring its body in 3D to overcome obstacles.
—Against the geometric sidewinding baseline (amplitude 1.2 rad, frequency 0.50 Hz), the learned policy shows superior robustness on unstructured terrain, avoiding the "stacking phenomenon" caused by the open-loop geometric model.
—On flat ground, the learned gait achieves reliable forward locomotion driven by the full 56-D observation, with independent per-joint phase control rather than a prescribed waveform.

Citation

bibtex

@inproceedings{shimizu2026emergent,
  title={Emergent Locomotion Patterns of a Snake Robot through Reinforcement Learning},
  author={Shimizu, Yuya and Wang, Yongdong and Shimooka, So and Kamegawa, Tetsushi},
  year={2026}
}