hamzasheedi/humanoid-robotics

🤖 PPO/SAC Agent for BipedalWalker-v3

This is a trained agent that learned to walk on two legs from scratch!

Model Description

• —Algorithm: PPO or SAC (Soft Actor-Critic)
• —Environment: BipedalWalker-v3
• —Framework: Stable-Baselines3
• —Training Steps: 500,000 steps

Performance

• —Walking Success: Consistent bipedal locomotion
• —Average Reward: 200+ (successful walking)
• —Coordination: Learned proper leg coordination and balance

Usage

from stable_baselines3 import PPO
import gymnasium as gym

# Load the trained model
model = PPO.load("bipedal_walker_ppo_model")

# Create environment
env = gym.make('BipedalWalker-v3', render_mode='human')

# Watch it walk!
obs, _ = env.reset()
for _ in range(2000):
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, _ = env.reset()

env.close()

Training Details

The agent learned to coordinate:

• —4 continuous joint controls (hip + knee for each leg)
• —Balance and momentum management
• —Forward locomotion
• —Obstacle navigation

What Makes This Impressive

• —24-dimensional state space - Complex sensory input
• —Continuous control - Smooth joint movements
• —Physics simulation - Realistic walking dynamics
• —From scratch learning - No pre-programmed walking patterns

Amazing to watch a robot learn to walk! 🚶‍♂️