Modelpublic

sarthak2314/GameAgent-IDM

sourceHugging Faceapache-2.0updated 11d agoView on Hugging Face

0likes26downloads

Model Card

GameAgent-IDM

Game-playing Inverse Dynamics Model for predicting keyboard and mouse actions from gameplay video.

Model details

Property	Value
Base model	OpenGVLab/InternVL3-1B-hf
Parameters	0.9B
Precision	BF16
Size	~1.9 GB
Input	Screen frames (448×448) + keyboard/mouse events with timestamps
Output	Predicted keyboard and mouse events between screen frames
Training data	147 h from 29 PC games (GameAgent-480p-100GB)

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "sarthak2314/GameAgent-IDM",
    dtype=torch.bfloat16,
    device_map="cuda",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "sarthak2314/GameAgent-IDM",
    trust_remote_code=True,
)

@article{choi2025d2e,
  title={D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI},
  author={Choi, Suhwan and Jung, Jaeyoon and Seong, Haebin and Kim, Minchan and
          Kim, Minyeong and Cho, Yongjun and Kim, Yoonshik and Park, Yubeen and
          Yu, Youngjae and Lee, Yunsung},
  journal={arXiv preprint arXiv:2510.05684},
  year={2025}
}