CoolFace
Datasetpublic

ramblr/BARISTA

BARISTA BARISTA is a densely annotated egocentric video dataset of coffee preparation, designed for unified benchmarking of vision-language models across spatial, temporal, relational, and procedural understanding tasks. The dataset contains 185 egocentric videos (~4.4 hours, 30 FPS, 1280×720 to 1920×1080) covering three coffee preparation methods: capsule machines, portafilter machines, and fully automatic machines. Videos were recorded in controlled indoor setups using iPhones… See the full description on the dataset page: https://huggingface.co/datasets/ramblr/BARISTA.

sourceHugging Facecc-by-nc-4.0updated 5mo agoView on Hugging Face
3likes341downloads
Dataset Card

BARISTA

BARISTA is a densely annotated egocentric video dataset of coffee preparation, designed for unified benchmarking of vision-language models across spatial, temporal, relational, and procedural understanding tasks.

The dataset contains 185 egocentric videos (~4.4 hours, 30 FPS, 1280×720 to 1920×1080) covering three coffee preparation methods: capsule machines, portafilter machines, and fully automatic machines. Videos were recorded in controlled indoor setups using iPhones, Apple Vision Pro, RayBan Meta 3, and RayBan Wayfarer smart glasses.

Dataset structure

Each video is stored in its own directory:

<video_id>/
    coco_annotation.json   # COCO-style annotations (masks, bboxes, attributes, relations, activities)
    video.mp4              # raw video

coco_annotation.json follows the COCO format extended with additional top-level keys:

KeyDescription
annotationsPer-frame instance annotations. Fields: id, image_id (0-based frame index), object_id (UUID), bbox ([x, y, w, h]), segmentation (COCO RLE with counts and size), area
attributesSegment-level key-value attributes per object. Fields: id, object_id, attribute_type (e.g. "color", "state"), value, image_ranges (list of {image_id_start, image_id_end})
relationsDirected typed relations between object pairs. Fields: id, source_object_id, target_object_id, relation_type (e.g. "position", "human_actions"), value, image_ranges
categoriesObject categories. Fields: id (UUID), name.
activitiesFine-grained verb+noun activity segments. Fields: id, display_name, activity_class_id (UUID), image_range ({image_id_start, image_id_end})
process_stepsHigh-level process step segments. Same fields as activities
object_id_to_category_idMap from object UUID to category UUID (needed to resolve annotation object_id → category)
video_metadataList with one entry. Fields: document_id, video_index, width, height, frame_count, fps, length_in_ms, recording_device_type, recording_device_version
splitDataset split: "train" or "test"

Loading the data and running evaluations

See the project repository for the dataset loader and the VLM benchmarking pipeline.

Citations

@misc{knab2026barista,
      title={BARISTA: A Multi-Task Egocentric Benchmark for Compositional Visual Understanding}, 
      author={Patrick Knab and Orgest Xhelili and Inis Buzi and Drago Andres Guggiana Nilo and Mohd Saquib Khan and Lorenz Kolb and Manuel Scherzer and Kerem Yildirir and Christian Bartelt and Philipp Johannes Schubert},
      year={2026},
      eprint={2605.12074},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2605.12074}, 
}