TeaPearce/CounterStrike_Deathmatch

This dataset contains video, action labels, and metadata from the popular video game CS:GO. Past usecases include imitation learning, behavioral cloning, world modeling, video generation.

The paper presenting the dataset: _Counter-Strike Deathmatch with Large-Scale Behavioural Cloning [Tim Pearce](https://teapearce.github.io/), [Jun Zhu](https://ml.cs.tsinghua.edu.cn/~jun/index.shtml) IEEE Conference on Games (CoG) 2022 [⭐️ Best Paper Award!] ArXiv paper: https://arxiv.org/abs/2104.04258 (Contains some extra experiments not in CoG version) CoG paper: https://ieee-cog.org/2022/assets/papers/paper45.pdf Four minute introduction video: https://youtu.be/rnz3lmfSHv0 Gameplay examples: https://youtu.be/KTY7UhjIMm4 Code: https://github.com/TeaPearce/Counter-StrikeBehaviouralCloning

The dataset comprises several different subsets of data as described below. You probably only care about the first one (if you want the largest dataset), or the second or third one (if you care about clean expert data).

• —``hdf5_dm_july2021_*_to_*.tar``
• —each .tar file contains 200 .hdf5 files
• —total files when unzipped: 5500
• —approx size: 700 GB
• —map: dust2
• —gamemode: deathmatch
• —source: scraped from online servers

• —``dataset_dm_expert_dust2/hdf5_dm_july2021_expert_*.hdf5``
• —total files when unzipped: 190
• —approx size: 24 GB
• —map: dust2
• —gamemode: deathmatch
• —source: manually created, clean actions

• —``dataset_aim_expert/hdf5_aim_july2021_expert_*.hdf5``
• —total files when unzipped: 45
• —approx size: 6 GB
• —map: aim map
• —gamemode: aim mode
• —source: manually created, clean actions

• —``dataset_dm_expert_othermaps/hdf5_dm_nuke_expert_*.hdf5``
• —total files when unzipped: 10
• —approx size: 1 GB
• —map: nuke
• —gamemode: deathmatch
• —source: manually created, clean actions

• —``dataset_dm_expert_othermaps/hdf5_dm_mirage_expert_*.hdf5``
• —total files when unzipped: 10
• —approx size: 1 GB
• —map: mirage
• —gamemode: deathmatch
• —source: manually created, clean actions

• —``dataset_dm_expert_othermaps/hdf5_dm_inferno_expert_*.hdf5``
• —total files when unzipped: 10
• —approx size: 1 GB
• —map: mirage
• —gamemode: deathmatch
• —source: manually created, clean actions

• —``dataset_metadata/currvarsv2_dm_july2021_*_to_*.npy, currvarsv2_dm_july2021_expert_*_to_*.npy, currvarsv2_dm_mirage_expert_1_to_100.npy, currvarsv2_dm_inferno_expert_1_to_100.npy, currvarsv2_dm_nuke_expert_1_to_100.npy, currvarsv2_aim_july2021_expert_1_to_100.npy``
• —total files when unzipped: 55 + 2 + 1 + 1 + 1 + 1 = 61
• —approx size: 6 GB
• —map: as per filename
• —gamemode: as per filename
• —source: as per filename

• —``location_trackings_backup/``
• —total files when unzipped: 305
• —approx size: 0.5 GB
• —map: dust2
• —gamemode: deathmatch
• —source: contains metadata used to compute map coverage analysis
• —currvarsv2_agentj22 is the agent trained over the full online dataset
• —currvarsv2_agentj22_dmexpert20 is previous model finetuned on the clean expert dust2 dataset
• —currvarsv2_bot_capture is medium difficulty built-in bot

Structure of .hdf5 files (image and action labels -- you probably care about this one):

Each file contains an ordered sequence of 1000 frames (~1 minute) of play. This contains screenshots, as well as processed action labels. We chose .hdf5 format for fast dataloading, since a subset of frames can be accessed without opening the full file. The lookup keys are as follows (where i is frame number 0-999)

• —frame_i_x: is the image
• —frame_i_xaux: contains actions applied in previous timesteps, as well as health, ammo, and team. see dmpretrainpreprocess.py for details, note this was not used in our final version of the agent
• —frame_i_y: contains target actions in flattened vector form; [keyspressedonehot, Lclicksonehot, Rclicksonehot, mousexonehot, mouseyonehot]
• —frame_i_helperarr: in format [killflag, deathflag], each a binary variable, e.g. [1,0] means the player scored a kill and did not die in that timestep

Structure of .npy files (scraped metadata -- you probably don't care about this):

Each .npy file contains metadata corresponding to 100 .hdf5 files (as indicated by file name) They are dictionaries with keys of format: filenumiframej for file number i, and frame number j in 0-999 The values are of format **[currvars, infera, framei_helperarr]** where,

• —curr_vars: contains a dictionary of the metadata originally scraped -- see dmrecorddata.py for details
• —infer_a: are inferred actions, [keyspressed,mousex,mousey,pressmousel,pressmouser], with mousex and y being continuous values and keys_pressed is in string format
• —frame_i_helperarr: is a repeat of the .hdf5 file

Trained Models

Four trained models are provided. There are 'non-stateful' (use during training) and 'stateful' (use at test time) versions of each. Models can be downloaded under ``trained_models.zip``.

• —``ak47_sub_55k_drop_d4`` : Pretrained on AK47 sequences only.
• —``ak47_sub_55k_drop_d4_dmexpert_28`` : Finetuned on expert deathmatch data.
• —``ak47_sub_55k_drop_d4_aimexpertv2_60`` : Finetuned on aim mode expert data.
• —``July_remoterun7_g9_4k_n32_recipe_ton96__e14`` : Pretrained on full dataset.

Other works using the dataset:

• —_Imitating Human Behaviour with Diffusion Models, ICLR 2023_ https://arxiv.org/abs/2301.10677 Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin

• —_Diffusion for World Modeling: Visual Details Matter in Atari, NeurIPS 2024_ https://arxiv.org/pdf/2405.12399 Eloi Alonso∗, Adam Jelley∗, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce‡, François Fleuret‡ Tweet here: https://twitter.com/EloiAlonso1/status/1844803606064611771