CoolFace
Modelpublic

nota-gmbh/so101_pick_place_pen_smolvla

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes78downloads
Model Card

so101pickplacepensmolvla

A SmolVLA vision-language-action policy fine-tuned with LeRobot for a pick-and-place pen task on the SO-101 robot arm.

이 모델은 SO-101 로봇 팔에서 펜 집어 옮기기(pick-and-place) 작업을 수행하도록 LeRobot으로 파인튜닝한 SmolVLA 비전-언어-액션 정책입니다.

Model Details / 모델 정보

  • —Policy / 정책: SmolVLA (fine-tuned from lerobot/smolvla_base)
  • —Robot / 로봇: SO-101 follower arm
  • —Task / 작업: Pick and place a pen / 펜 집어 옮기기
  • —Dataset / 데이터셋: `nota-gmbh/pick_and_place_pen_so101`
  • —Episodes / 에피소드: 53

Training / 학습

  • —Steps: 40,000
  • —Hardware: Multi-GPU
  • —Batch size: 64
  • —Base model: lerobot/smolvla_base

During data collection, the camera frames were captured keeping a consistent width-to-height aspect ratio across episodes for visual consistency.

데이터 수집 시, 에피소드 전반에 걸쳐 카메라 프레임의 가로:세로 비율을 동일하게 유지하려고 노력했습니다.

terminal
export NCCL_P2P_DISABLE="1"
export NCCL_IB_DISABLE="1"
export HF_USER="~~"
export TASK_NAME="pick_and_place"

accelerate launch --multi_gpu --num_processes=4 \
  $(which lerobot-train) \
  --policy.path=lerobot/smolvla_base \
  --dataset.repo_id=Dangni/pick_and_place_so101_pen \
  --dataset.video_backend=pyav \
  --dataset.image_transforms.enable=true \
  --dataset.image_transforms.max_num_transforms=3 \
  --dataset.image_transforms.random_order=true \
  --batch_size=16 \
  --num_workers=1 \
  --steps=100000 \
  --output_dir=outputs/train/so101/smolvla/pick_and_place_$(date +%Y%m%d_%H%M%S) \
  --job_name=smolvla_yellow \
  --policy.device=cuda \
  --policy.freeze_vision_encoder=true \
  --policy.train_expert_only=true \
  --policy.push_to_hub=false\
  --policy.repo_id=Dangni/pick_and_place_yellow_smolvla \
  --rename_map='{"observation.images.fixed": "observation.images.camera1", "observation.images.handy": "observation.images.camera2"}' \
  --save_freq=5000 \
  --wandb.enable=true

Optimizer / 옵티마이저

Item / 항목Value / 값Note / 설명
OptimizerAdamWdecoupled_weight_decay=True
Initial LR1.0e-04Learning rate at the start of training / 학습 시작 LR
LR (latest logged)3.2594e-05LR at the logged step (scheduled decay) / 스케줄러 적용 후 LR
Betas(0.9, 0.95)1st / 2nd moment coefficients
Epsilon1e-08Numerical stability constant
Weight Decay1e-10Very small L2 regularization
Note: The learning rate decays over training via the scheduler, so 3.2594e-05 reflects a logged intermediate step rather than the final value. 참고: LR은 스케줄러에 의해 학습 중 감소하며, 3.2594e-05는 로깅된 중간 스텝의 값입니다.

Usage / 사용법

terminal
lerobot-rollout `
  --strategy.type=base `
  --policy.path=Dangni/pick_and_place_smolvla_so101_040000 `
  --policy.n_action_steps=50 `
  --robot.type=so101_follower `
  --robot.port=COM3 `
  --robot.id=follower_arm `
  --robot.cameras='{fixed: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}, handy: {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30}}' `
  --rename_map='{"observation.images.fixed": "observation.images.camera1", "observation.images.handy": "observation.images.camera2"}' `
  --task="Pick the pen and place it inside the yellow border" `
  --duration=500

Limitations / 한계

  • —Trained on a single pick-and-place pen task with 53 episodes; performance is limited to this task and the data-collection setup.
  • —Camera resolution and viewpoint should match the training setup for reliable inference.
  • —단일 펜 집기 작업, 53개 에피소드로 학습되어 해당 작업과 수집 환경에 한정됩니다.
  • —추론 시 카메라 해상도·시점을 학습 환경과 동일하게 맞춰야 안정적으로 동작합니다.

Links / 링크