saaduddinM/OXE_berkeley_cable_routing_embeddings

Language Table (LeRobot) — Embedding-Only Release (DINOv3 + SigLIP2 image features; EmbeddingGemma task-text features)

This repository packages a re-encoded variant of IPEC-COMMUNITY/berkeley_cable_routing_lerobot where raw videos are replaced by fixed-length image embeddings, and task strings are augmented with text embeddings. All indices, splits, and semantics remain consistent with the source dataset while storage and I/O are substantially lighter. To make the dataset practical to upload/download and stream from the Hub, we also consolidated tiny per-episode Parquet files into N large Parquet shards under a single data/ folder. The file meta/shardedindex.json preserves a precise mapping from each original episode (referenced by a normalized identifier of the form data/chunk-{episodechunk:03d}/episode{episodeindex:06d}.parquet) to its shard path and row range, so you keep original addressing without paying the small-file tax.

• —Robot: Franka
• —Modalities kept: states, actions, timestamps, frame/episode indices, image embeddings, task-text embeddings
• —Removed:
• —observation.images.image
• —observation.images.top_image
• —observation.images.wrist225_image
• —observation.images.wrist45_image
• —License: apache-2.0 (inherits from source)

Quick Stats

From meta/info.json and meta/tasktextembeddings_info.json:

• —Episodes: 1,482
• —Frames: 38,240
• —Tasks (unique): 1
• —Chunks (original layout): 2 (chunks_size=1000)
• —Shards (this release): 64 Parquet files under data/ (see meta/sharded_index.json)
• —FPS: 10
• —Image embeddings (per frame):
• —observation.images.image_dinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.image_siglip2 → float32 [768] (SigLIP2-base)
• —observation.images.topimagedinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.topimagesiglip2 → float32 [768] (SigLIP2-base)
• —observation.images.wrist225imagedinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.wrist225imagesiglip2 → float32 [768] (SigLIP2-base)
• —observation.images.wrist45imagedinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.wrist45imagesiglip2 → float32 [768] (SigLIP2-base)
• —Task-text embeddings (per unique task):
• —embedding → float32 [768] from google/embeddinggemma-300m
• —Count: 1 rows (one per task)

Note: This is an embedding-only package. The original pixel arrays listed under “Removed” are dropped.

<details> <summary><b>Contents</b></summary>

<pre> . |-- meta/ | |-- info.json | |-- shardedindex.json | |-- tasks.jsonl | |-- episodes.jsonl | `-- tasktextembeddingsinfo.json |-- data/ | |-- shard-00000-of-000NN.parquet | |-- shard-00001-of-000NN.parquet | |-- ... | -- task_text_embeddings.parquet -- README.md </pre> </details>

How This Was Generated (Reproducible Pipeline)

1) Episode → Image Embeddings (drop pixels) convertlerobottoembeddingsmono.py (GPU-accelerated preprocessing). Adds:

• —observation.images.image_dinov3 (float32[1024])
• —observation.images.image_siglip2 (float32[768])
• —observation.images.topimagedinov3 (float32[1024])
• —observation.images.topimagesiglip2 (float32[768])
• —observation.images.wrist225imagedinov3 (float32[1024])
• —observation.images.wrist225imagesiglip2 (float32[768])
• —observation.images.wrist45imagedinov3 (float32[1024])
• —observation.images.wrist45imagesiglip2 (float32[768]) Removes:
• —observation.images.image
• —observation.images.top_image
• —observation.images.wrist225_image
• —observation.images.wrist45_image

2) Task-Text Embeddings (one row per unique task) buildtasktextembeddings.py with SentenceTransformer("google/embeddinggemma-300m") → data/tasktextembeddings.parquet + meta/tasktextembeddingsinfo.json.

3) Data Consolidation (this release) All per-episode Parquets were consolidated into N large Parquet shards in one data/ folder.

• —The index meta/shardedindex.json records, for each episode, its normalized source identifier data/chunk-{episodechunk:03d}/episode{episodeindex:06d}.parquet, the destination shard path, and the (rowoffset, numrows) range inside that shard.
• —This preserves original addressing while making Hub sync/clone/stream far faster and more reliable.

Metadata (Excerpts)

meta/tasktextembeddings_info.json

~~~json { "model": "google/embeddinggemma-300m", "dimension": 768, "normalized": true, "count": 1, "file": "tasktextembeddings.parquet" } ~~~

meta/info.json (embedding-only + shards)

~~~json { "codebaseversion": "v2.1-embeddings-sharded", "robottype": "franka", "totalepisodes": 1482, "totalframes": 38240, "totaltasks": 1, "totalvideos": 5928, "totalchunks": 2, "chunkssize": 1000, "fps": 10, "splits": { "train": "0:1482" }, "datapath": "data/shard-{shardid:05d}-of-{numshards:05d}.parquet", "features": { "observation.state": { "dtype": "float32", "shape": [ 8 ], "names": { "motors": [ "motor0", "motor1", "motor2", "motor3", "motor4", "motor5", "motor6", "pad" ] } }, "action": { "dtype": "float32", "shape": [ 7 ], "names": { "motors": [ "x", "y", "z", "roll", "pitch", "yaw", "gripper" ] } }, "timestamp": { "dtype": "float32", "shape": [ 1 ], "names": null }, "frameindex": { "dtype": "int64", "shape": [ 1 ], "names": null }, "episodeindex": { "dtype": "int64", "shape": [ 1 ], "names": null }, "index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "taskindex": { "dtype": "int64", "shape": [ 1 ], "names": null }, "observation.images.imagedinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.imagesiglip2": { "dtype": "float32", "shape": [ 768 ], "names": null }, "observation.images.topimagedinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.topimagesiglip2": { "dtype": "float32", "shape": [ 768 ], "names": null }, "observation.images.wrist225imagedinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.wrist225imagesiglip2": { "dtype": "float32", "shape": [ 768 ], "names": null }, "observation.images.wrist45imagedinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.wrist45imagesiglip2": { "dtype": "float32", "shape": [ 768 ], "names": null } }, "videokeys": [ "observation.images.image", "observation.images.topimage", "observation.images.wrist225image", "observation.images.wrist45image" ], "numshards": 64, "indexpath": "meta/shardedindex.json" } ~~~

Environment & Dependencies

Python ≥ 3.9 • PyTorch ≥ 2.1 • transformers • sentence-transformers • pyarrow • tqdm • decord (and optionally av)

Provenance, License, and Citation

• —Source dataset: IPEC-COMMUNITY/berkeley_cable_routing_lerobot
• —License: apache-2.0 (inherits from the source)
• —Encoders to cite:
• —facebook/dinov3-vitl16-pretrain-lvd1689m
• —google/siglip2-base-patch16-384
• —google/embeddinggemma-300m

Changelog

• —v2.0-embeddings-sharded — Replaced video tensors with DINOv3 + SigLIP2 features; added EmbeddingGemma task-text embeddings; consolidated per-episode Parquets into N shards with a repo-local index; preserved original indexing/splits via normalized episode identifiers.