TJURL-Lab/UniEQA

UniEQA Dataset

UniEQA Dataset Directory Structure

|- Part
  |- capability dimension (eg.,object_type)
     |- core
        |- images
        |- data.json
  | - ...
|- ...

The UniEQA dataset includes questions, images, and answers, and the question-images-answer pairs are in data.json.

Download Dataset

Step 1: Download dataset UniEQA.

Step 2: Download HM3D. The RGB frames for the HM3D episode histories are available in this third party location (12 Gb). You can use the following commands to download and extract the data:

wget -O open-eqa-hm3d-frames-v0.tgz <link above>
md5sum open-eqa-hm3d-frames-v0.tgz  # 286aa5d2fda99f4ed1567ae212998370
tar -xzf open-eqa-hm3d-frames-v0.tgz -C Part1/images
rm open-eqa-hm3d-frames-v0.tgz

Afterwards, your directory should look like this:

|- Part1
   |- images
      |- hm3d
         |- 000-hm3d-BFRyYbPCCPE
         |- ...
| - ...

Step 3: Download ScanNet by following the instructions here.

Place the data in data/raw/scannet. Afterwards, your directory should look like this:

|- data
   |- raw
      |- scannet
         |- scans
            |- <scanId>
               |- <scanId>.sens
               |- ...
         |- scans_test
            |- <scanId>
               |- <scanId>.sens
               |- ...
|- ...

Step 4: Extract episode histories $H$ from ScanNet scenes.

You can either only extract RGB frames or extract RGB, depth, camera intrinsics, and camera pose information.

To extract only the RGB frames, run:

python data/raw/scannet/extract-frames.py --rgb-only

To extract the RGB, depth, camera intrinsics, and camera pose information, run:

python data/raw/scannet/extract-frames.py

Place the RGB frames in Part1/images/scannet. Afterwards, your directory structure should look like this:

|- Part1
   |- images
      |- scannet
         |- 002-scannet-scene0709_00
         |- ...
      |- hm3d
| - ...