Datasetpublic

unstructured/pix3d

Pix3D Pix3D is a large-scale dataset of real images paired with pixel-aligned 3D shapes, released alongside Sun, Wu, Zhang et al., Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling (CVPR 2018). This Hub repo repackages the official release into two Hugging Face datasets configs so both the image annotations and the underlying 3D assets can be loaded directly with load_dataset, with no manual download/unzip step. Project page: http://pix3d.csail.mit.edu Original… See the full description on the dataset page: https://huggingface.co/datasets/unstructured/pix3d.

sourceHugging Facecc-by-4.0updated 7d agoView on Hugging Face

0likes45downloads

Dataset Card

Pix3D

Pix3D is a large-scale dataset of real images paired with pixel-aligned 3D shapes, released alongside Sun, Wu, Zhang et al., *Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling* (CVPR 2018). This Hub repo repackages the official release into two Hugging Face datasets configs so both the image annotations and the underlying 3D assets can be loaded directly with load_dataset, with no manual download/unzip step.

—Project page: http://pix3d.csail.mit.edu
—Original repository: https://github.com/xingyuansun/pix3d
—License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Dataset structure

The dataset is split into two configs because 10,069 image annotations reference only 735 unique physical 3D objects (many objects were photographed multiple times). Keeping them separate avoids duplicating each mesh/voxel grid across every image that uses it; join the two configs on model_id when you need both.

python

from datasets import load_dataset

annotations = load_dataset("unstructured/pix3d", "default")   # 10,069 rows, one per image
models = load_dataset("unstructured/pix3d", "models")          # 735 rows, one per unique 3D object

`default` config — one row per annotated image

Column	Type	Description
`idx`	int32	Unique incrementing row identifier
`img`	Image	The photograph containing the object
`width`, `height`	int32	Image dimensions (`img_size` in the original JSON)
`category`	ClassLabel	Object category (`bed`, `bookcase`, `chair`, `desk`, `misc`, `sofa`, `table`, `tool`, `wardrobe`)
`mask`	Image	Binary segmentation mask isolating the object in `img`
`model_id`	string	Join key into the `models` config identifying which 3D object appears in this image
`img_source`	ClassLabel	Image provenance: `ikea`, `internet`, or `self-taken`
`keypoints_2d`	list[list[list[float]]]	2D keypoints per annotator, shape `[n_annotators, n_keypoints, 2]`, in pixel coordinates (origin top-left, `+x` right, `+y` down); `[-1, -1]` marks a keypoint an annotator judged unlabelable (e.g. occluded)
`bbox`	list[int]	Object bounding box `[x_from, y_from, x_to, y_to]`
`rot_mat`, `trans_mat`	list[float]	Rotation matrix (3x3) and translation vector applied to the object for rendering
`cam_position`, `inplane_rotation`	list[float] / float	Estimated camera position and in-plane rotation (object-centered), used to evaluate viewer-centered predictions against the canonical view
`focal_length`	float	Estimated focal length in mm (sensor width fixed at 32mm)
`truncated`, `occluded`, `slightly_occluded`	bool	Visibility flags for the object in the image (`occluded` and `slightly_occluded` are mutually exclusive)

`models` config — one row per unique 3D object

Column	Type	Description
`model_id`	string	Unique identifier for this 3D object; matches `model_id` in the `default` config
`category`	ClassLabel	Same category enumeration as the `default` config
`model`	binary (glb)	The object's textured mesh, converted from the original `.obj`/`.mtl`/texture files into a single self-contained binary glTF (`.glb`) blob via `trimesh`, analogous to how `img`/`mask` embed PNG bytes
`model_raw`	binary (glb) or null	Raw, unprocessed scanned mesh, same glb encoding as `model`. Only populated for objects from the full Pix3D release that includes raw scans (`self-scanned` sources); this standard release does not include raw scans, so this column is `null` for every row here
`model_source`	ClassLabel	3D model provenance: `ikea` or `self-scanned`
`voxel_model`	Array3D (128×128×128, uint8)	Voxelized occupancy grid of the object, parsed from the original `voxel.mat` file (`1` = occupied)
`keypoints_3d`	list[list[float]]	3D keypoint positions `[n_keypoints, 3]` in the model's canonical coordinate frame (`+x` left, `+y` up, `+z` inward)

Coordinate conventions (from the original release)

—2D keypoints / image pixels: origin top-left, +x right, +y down.
—Mesh (`model`) canonical frame: +x left, +y up, +z inward.
—Voxel (`voxel_model`) canonical frame: +x left, +y outward, +z up.
—rot_mat/trans_mat are defined in camera coordinates and applied to the object for rendering; they are not directly usable to rotate viewer-centered predictions back to canonical view. Use cam_position and inplane_rotation instead — both are defined in the object's own coordinate frame and reproduce the original image up to a homography, which is what the original authors used for viewpoint evaluation.

Category / label distribution

Category	# images
chair	3,839
sofa	1,947
table	1,870
bed	994
desk	700
bookcase	361
wardrobe	243
misc	68
tool	47

License and attribution

The Pix3D dataset is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). You are free to share and adapt the data for any purpose, including commercially, provided you give appropriate credit to the original authors.

If you use this dataset, please cite the original paper:

bibtex

@inproceedings{pix3d,
  title={Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling},
  author={Sun, Xingyuan and Wu, Jiajun and Zhang, Xiuming and Zhang, Zhoutong and Zhang, Chengkai and Xue, Tianfan and Tenenbaum, Joshua B and Freeman, William T},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2018}
}

For questions about the original dataset, contact Xingyuan Sun (xingyuansun.cs@gmail.com) or Jiajun Wu (jiajunwu@mit.edu). This Hub repackaging was produced by converting the official release's pix3d.json and asset folders (img/, mask/, model/) into the two configs described above; no annotation values were modified, only the file layout and encoding (mesh → glb, voxel .mat → numpy array).