visgym
Datasets
All datasets matching “visgym”visgym_data
VisGym Dataset
Project Page | Paper | GitHub
VisGym consists of 17 diverse, long-horizon environments designed to systematically evaluate, diagnose, and train Vision-Language Models (VLMs) on visually interactive tasks. In these environments, agents must select actions conditioned on both their past actions and observation history, challenging their ability to handle complex, multimodal sequences.
Dataset Summary
This dataset contains trajectories and interaction data… See the full description on the dataset page: https://huggingface.co/datasets/VisGym/visgym_data.trajectoryinference-dataset
VLM-Gym Inference Dataset
This dataset contains pre-defined test episodes and initial states for evaluating Vision-Language Models (VLMs) on the VLM-Gym benchmark.
Dataset Structure
inference-dataset/
├── test_set_easy/ # Easy difficulty test episodes (JSONL)
├── test_set_hard/ # Hard difficulty test episodes (JSONL)
├── initial_states_easy/ # Initial environment states for easy episodes (JSON)
├── initial_states_hard/ # Initial environment… See the full description on the dataset page: https://huggingface.co/datasets/VisGym/inference-dataset.visgym_pacman_2d_random
VisGym Pacman2D Random Policy World-Model Trajectories
This dataset contains random-policy interaction trajectories for the custom VisGym Pacman2D environment.
Each row is one environment trajectory. The history entries include image_prev, image, and
image_next base64 JPEG frames, the prompt shown at the step, the sampled action, reward, and
environment info. The policy samples move actions uniformly and samples the valid stop action with
the configured random stop probability after… See the full description on the dataset page: https://huggingface.co/datasets/novastar112/visgym_pacman_2d_random.visgym-pacman-2d
VisGym Pacman2D Deterministic Trajectories
This dataset contains behavior-cloning trajectories for the custom VisGym Pacman2D environment.
Each row is one successful oracle episode. The history entries include image_prev, image, and
image_next; no synthetic thinking traces are stored. Images are JPEG base64 strings rendered from
the greyblue9/pacman-python visual assets used by the environment.
Environment summary:
Grid size is constrained to 7x7 through 12x12.
Easy uses the 9x9… See the full description on the dataset page: https://huggingface.co/datasets/DarthVaderSenior/visgym-pacman-2d.visgym_pacman_2d_remap
VisGym Pacman2D Deterministic Trajectories
This dataset contains behavior-cloning trajectories for the custom VisGym Pacman2D environment.
Each row is one successful oracle episode. The history entries include image_prev, image, and
image_next; no synthetic thinking traces are stored. Images are JPEG base64 strings rendered from
the greyblue9/pacman-python visual assets used by the environment.
Environment summary:
Grid size is constrained to 7x7 through 12x12.
Easy uses the 9x9… See the full description on the dataset page: https://huggingface.co/datasets/novastar112/visgym_pacman_2d_remap.