saaduddinM/OXE_bridge_orig_embeddings

Language Table (LeRobot) — Embedding-Only Release (DINOv3 + SigLIP2 image features; EmbeddingGemma task-text features)

This repository packages a re-encoded variant of IPEC-COMMUNITY/bridge_orig_lerobot where raw videos are replaced by fixed-length image embeddings, and task strings are augmented with text embeddings. All indices, splits, and semantics remain consistent with the source dataset while storage and I/O are substantially lighter. To make the dataset practical to upload/download and stream from the Hub, we also consolidated tiny per-episode Parquet files into N large Parquet shards under a single data/ folder. The file meta/shardedindex.json preserves a precise mapping from each original episode (referenced by a normalized identifier of the form data/chunk-{episodechunk:03d}/episode{episodeindex:06d}.parquet) to its shard path and row range, so you keep original addressing without paying the small-file tax.

• —Robot: widowx
• —Modalities kept: states, actions, timestamps, frame/episode indices, image embeddings, task-text embeddings
• —Removed:
• —observation.images.image_0
• —observation.images.image_1
• —observation.images.image_2
• —observation.images.image_3
• —License: apache-2.0 (inherits from source)

Quick Stats

From meta/info.json and meta/tasktextembeddings_info.json:

• —Episodes: 53,192
• —Frames: 1,893,026
• —Tasks (unique): 19,974
• —Chunks (original layout): 54 (chunks_size=1000)
• —Shards (this release): 64 Parquet files under data/ (see meta/sharded_index.json)
• —FPS: 5
• —Image embeddings (per frame):
• —observation.images.image0dinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.image0siglip2 → float32 [768] (SigLIP2-base)
• —observation.images.image1dinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.image1siglip2 → float32 [768] (SigLIP2-base)
• —observation.images.image2dinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.image2siglip2 → float32 [768] (SigLIP2-base)
• —observation.images.image3dinov3 → float32 [1024] (DINOv3 ViT-L/16 CLS)
• —observation.images.image3siglip2 → float32 [768] (SigLIP2-base)
• —Task-text embeddings (per unique task):
• —embedding → float32 [768] from google/embeddinggemma-300m
• —Count: 19,974 rows (one per task)

Note: This is an embedding-only package. The original pixel arrays listed under “Removed” are dropped.

<details> <summary><b>Contents</b></summary>

<pre> . |-- meta/ | |-- info.json | |-- shardedindex.json | |-- tasks.jsonl | |-- episodes.jsonl | `-- tasktextembeddingsinfo.json |-- data/ | |-- shard-00000-of-000NN.parquet | |-- shard-00001-of-000NN.parquet | |-- ... | -- task_text_embeddings.parquet -- README.md </pre> </details>

How This Was Generated (Reproducible Pipeline)

1) Episode → Image Embeddings (drop pixels) convertlerobottoembeddingsmono.py (GPU-accelerated preprocessing). Adds:

• —observation.images.image0dinov3 (float32[1024])
• —observation.images.image0siglip2 (float32[768])
• —observation.images.image1dinov3 (float32[1024])
• —observation.images.image1siglip2 (float32[768])
• —observation.images.image2dinov3 (float32[1024])
• —observation.images.image2siglip2 (float32[768])
• —observation.images.image3dinov3 (float32[1024])
• —observation.images.image3siglip2 (float32[768]) Removes:
• —observation.images.image_0
• —observation.images.image_1
• —observation.images.image_2
• —observation.images.image_3

2) Task-Text Embeddings (one row per unique task) buildtasktextembeddings.py with SentenceTransformer("google/embeddinggemma-300m") → data/tasktextembeddings.parquet + meta/tasktextembeddingsinfo.json.

3) Data Consolidation (this release) All per-episode Parquets were consolidated into N large Parquet shards in one data/ folder.

• —The index meta/shardedindex.json records, for each episode, its normalized source identifier data/chunk-{episodechunk:03d}/episode{episodeindex:06d}.parquet, the destination shard path, and the (rowoffset, numrows) range inside that shard.
• —This preserves original addressing while making Hub sync/clone/stream far faster and more reliable.

Metadata (Excerpts)

meta/tasktextembeddings_info.json

~~~json { "model": "google/embeddinggemma-300m", "dimension": 768, "normalized": true, "count": 19974, "file": "tasktextembeddings.parquet" } ~~~

meta/info.json (embedding-only + shards)

~~~json { "codebaseversion": "v2.0-embeddings-sharded", "robottype": "widowx", "totalepisodes": 53192, "totalframes": 1893026, "totaltasks": 19974, "totalvideos": 212768, "totalchunks": 54, "chunkssize": 1000, "fps": 5, "splits": { "train": "0:53192" }, "datapath": "data/shard-{shardid:05d}-of-{numshards:05d}.parquet", "features": { "observation.state": { "dtype": "float32", "shape": [ 8 ], "names": { "motors": [ "x", "y", "z", "roll", "pitch", "yaw", "pad", "gripper" ] } }, "action": { "dtype": "float32", "shape": [ 7 ], "names": { "motors": [ "x", "y", "z", "roll", "pitch", "yaw", "gripper" ] } }, "timestamp": { "dtype": "float32", "shape": [ 1 ], "names": null }, "frameindex": { "dtype": "int64", "shape": [ 1 ], "names": null }, "episodeindex": { "dtype": "int64", "shape": [ 1 ], "names": null }, "index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "taskindex": { "dtype": "int64", "shape": [ 1 ], "names": null }, "observation.images.image0dinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.image0siglip2": { "dtype": "float32", "shape": [ 768 ], "names": null }, "observation.images.image1dinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.image1siglip2": { "dtype": "float32", "shape": [ 768 ], "names": null }, "observation.images.image2dinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.image2siglip2": { "dtype": "float32", "shape": [ 768 ], "names": null }, "observation.images.image3dinov3": { "dtype": "float32", "shape": [ 1024 ], "names": null }, "observation.images.image3siglip2": { "dtype": "float32", "shape": [ 768 ], "names": null } }, "videokeys": [ "observation.images.image0", "observation.images.image1", "observation.images.image2", "observation.images.image3" ], "numshards": 64, "indexpath": "meta/shardedindex.json" } ~~~

Environment & Dependencies

Python ≥ 3.9 • PyTorch ≥ 2.1 • transformers • sentence-transformers • pyarrow • tqdm • decord (and optionally av)

Provenance, License, and Citation

• —Source dataset: IPEC-COMMUNITY/bridge_orig_lerobot
• —License: apache-2.0 (inherits from the source)
• —Encoders to cite:
• —facebook/dinov3-vitl16-pretrain-lvd1689m
• —google/siglip2-base-patch16-384
• —google/embeddinggemma-300m

Changelog

• —v2.0-embeddings-sharded — Replaced video tensors with DINOv3 + SigLIP2 features; added EmbeddingGemma task-text embeddings; consolidated per-episode Parquets into N shards with a repo-local index; preserved original indexing/splits via normalized episode identifiers.