chrisluo5311/a2c-PandaReachDense-v3

A2C Agent playing PandaReachDense-v3

This is a trained model of a A2C agent playing PandaReachDense-v3 using the stable-baselines3 library.

Environment

• —Task: move the end-effector of a Franka Emika Panda robotic arm to a randomly sampled target position (the green ball).
• —Observation: a Dict with three entries, which is why MultiInputPolicy is used instead of MlpPolicy:
• —observation: end-effector position (x, y, z) and velocity (vx, vy, vz)
• —achieved_goal: current end-effector position (x, y, z)
• —desired_goal: target position (x, y, z)
• —Action: 3-dim continuous end-effector displacement (x, y, z). Joints are not controlled directly.
• —Reward (dense): negative Euclidean distance between the end-effector and the target at every step, so values closer to 0 are better.

Results

Evaluated for 10 episodes with a deterministic policy, using the saved VecNormalize statistics (frozen, with reward normalization disabled).

Usage (with Stable-baselines3)

import gymnasium as gym
import panda_gym
from huggingface_sb3 import load_from_hub
from stable_baselines3 import A2C
from stable_baselines3.common.vec_env import DummyVecEnv, VecNormalize

repo_id = "chrisluo5311/a2c-PandaReachDense-v3"
model_path = load_from_hub(repo_id, "a2c-PandaReachDense-v3.zip")
stats_path = load_from_hub(repo_id, "vec_normalize.pkl")

env = DummyVecEnv([lambda: gym.make("PandaReachDense-v3", render_mode="rgb_array")])
env = VecNormalize.load(stats_path, env)
env.training = False      # do not update normalization stats at test time
env.norm_reward = False

model = A2C.load(model_path, env=env)

obs = env.reset()
for _ in range(1000):
    action, _ = model.predict(obs, deterministic=True)
    obs, rewards, dones, infos = env.step(action)

Hyperparameters