Datasetpublic

didfd/robodojo-assets-packed

RoboDojo Assets — verified transport shards This public dataset is a byte-preserving transport repack of Assets/ from RoboDojo-Benchmark/RoboDojo, pinned at source commit aada55a7a58b520a2004b9db02e884144b6e83e7. It reduces 15,335 source files to 11 independently downloadable and restorable tar.zst shards. No payload file was changed. 这是 RoboDojo Assets/ 的公开传输分片：固定源 commit、减少小文件数量、每片可独立下载/重试，完整复原后会逐文件做 SHA-256 校验。 Contents Source files: 15,335 Source logical… See the full description on the dataset page: https://huggingface.co/datasets/didfd/robodojo-assets-packed.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes545downloads

Dataset Card

RoboDojo Assets — verified transport shards

This public dataset is a byte-preserving transport repack of Assets/ from `RoboDojo-Benchmark/RoboDojo`, pinned at source commit aada55a7a58b520a2004b9db02e884144b6e83e7. It reduces 15,335 source files to 11 independently downloadable and restorable tar.zst shards. No payload file was changed.

这是 RoboDojo Assets/ 的公开传输分片：固定源 commit、减少小文件数量、每片可独立下载/重试，完整复原后会逐文件做 SHA-256 校验。

—Source files: 15,335
—Source logical bytes: 41,268,637,776
—Source revision: `aada55a7a58b520a2004b9db02e884144b6e83e7`
—Archive format: GNU tar + Zstandard level 3
—Every archive is below 4,000,000,000 bytes

shard	files	input bytes	archive bytes	SHA-256 prefix
`data/assets-000.tar.zst`	2,023	3,758,095,437	3,538,570,683	`a0846895771da2ac…`
`data/assets-001.tar.zst`	143	3,758,074,568	3,749,841,773	`711b3a5bcee029cd…`
`data/assets-002.tar.zst`	263	3,758,095,164	3,175,194,388	`b8360b8f0bf21af0…`
`data/assets-003.tar.zst`	491	3,758,080,869	3,488,084,115	`9b184cbe5ccc249e…`
`data/assets-004.tar.zst`	143	3,758,095,598	3,672,499,179	`442e12ad6c7820aa…`
`data/assets-005.tar.zst`	472	3,758,095,587	3,385,701,591	`1e5b0fdf922b09e7…`
`data/assets-006.tar.zst`	467	3,758,084,191	3,491,925,802	`1469148d3284b53b…`
`data/assets-007.tar.zst`	8,488	3,758,059,501	3,311,607,868	`29d4b73bf90de4d3…`
`data/assets-008.tar.zst`	517	3,758,096,082	3,433,188,150	`045b896487b1484c…`
`data/assets-009.tar.zst`	592	3,758,062,850	3,570,379,301	`0a7aa894670f58e9…`
`data/assets-010.tar.zst`	1,736	3,687,797,929	3,459,212,794	`7e4d65469a712e59…`

SHA256SUMS verifies the 11 transport archives. FILES.sha256 verifies every restored source file. MANIFEST.json records exact counts, sizes, paths, and full hashes.

Download with low external concurrency

Install the current Hugging Face CLI, then choose one or two workers. Pin --revision to a full commit SHA in production.

bash

hf download didfd/robodojo-assets-packed \
  --repo-type dataset \
  --revision <FULL_PACKED_REPO_COMMIT_SHA> \
  --local-dir /path/to/robodojo-assets-packed \
  --max-workers 2

If the company importer accepts file URLs only, import the 11 files under data/ plus MANIFEST.json, SHA256SUMS, FILES.sha256, lists/, and restore.py. With concurrency 1, only one ~3.7 GB object is active at a time; a failed transfer only retries that shard.

Restore and verify

Python 3.9+, zstd, and Linux with /proc/self/fd plus O_NOFOLLOW support are required. The restore tool validates all metadata, archive hashes, and structured tar headers before creating the output. It rejects links, sparse files, devices, FIFOs, non-canonical paths, duplicates, and cross-shard path conflicts. Each shard is extracted into an isolated private staging directory; only a fully hashed tree is atomically published as Assets/.

bash

python3 restore.py \
  --repo-dir /path/to/robodojo-assets-packed \
  --output-dir /path/to/restored \
  --workers 4 \
  --hash-workers 16

Success ends with RESTORE VERIFIED; the restored root is /path/to/restored/Assets/. If Assets/ already exists, --force builds and verifies a fresh tree first, atomically replaces the old tree, and preserves the old tree as Assets.backup.<random-id>; it does not perform an in-place overwrite. Atomic replacement requires filesystem support for Linux RENAME_EXCHANGE; if unavailable, --force fails without modifying the existing tree.

Provenance and license

The source dataset declares Apache-2.0. This repository changes packaging only and keeps the same declared license; users remain responsible for following the upstream dataset's terms and notices.