codemanCheng/physicalword-assets

1# PhysicalWord last05_mot2_action Assets2 3This dataset repo contains the private/custom assets needed to train4`last05_mot2_action` on a machine that cannot access the source NAS.5 6Code repo expected by the migration guide:7 8- GitHub: `https://github.com/zhang-yi-ming/phyword.git`9- Branch: `mot2_cosmos_ae`10- Commit: `1495a9d`11 12## Included13 14This repo intentionally includes:15 16```text17ckpt/pretrained/LaST0_Pretrain_AE_chunk16/tfmr/18data/libero_training_data_last05_lastest/libero_spatial_20hz_224_dual/19rlbench/20```21 22Approximate source sizes:23 24```text25ckpt/pretrained/LaST0_Pretrain_AE_chunk16/tfmr                         4.0G26data/libero_training_data_last05_lastest/libero_spatial_20hz_224_dual  2.8G27rlbench                                                               25G28```29 30## Not Included31 32Cosmos Predict2.5 weights are not included in this repo. Download them from33the official gated NVIDIA Hugging Face model after accepting the license:34 35```bash36huggingface-cli download nvidia/Cosmos-Predict2.5-2B \37  --local-dir /mnt/nas/zhangyiming/database/ckpt/pretrained/Cosmos-Predict2.5-2B38```39 40The source scripts expect the Cosmos checkpoint at:41 42```text43/mnt/nas/zhangyiming/database/ckpt/pretrained/Cosmos-Predict2.5-2B/base/pre-trained/d20b7120-df3e-4911-919d-db6e08bad31c_ema_bf16.pt44```45 46Already-trained MoT2 evaluation checkpoints are also not included.47 48## Target Layout49 50Download this dataset repo directly into:51 52```bash53/mnt/nas/zhangyiming/database54```55 56Example:57 58```bash59export HF_HUB_ENABLE_HF_TRANSFER=160mkdir -p /mnt/nas/zhangyiming/database61 62huggingface-cli download <owner>/physicalword-assets \63  --repo-type dataset \64  --local-dir /mnt/nas/zhangyiming/database65```66 67The training JSON files contain absolute paths under68`/mnt/nas/zhangyiming/database`. If the assets are downloaded elsewhere, create69a symlink:70 71```bash72mkdir -p /mnt/nas/zhangyiming73ln -sfn /actual/database/path /mnt/nas/zhangyiming/database74```75 76## Asset Notes77 78### Action Expert79 80Expected target path:81 82```text83/mnt/nas/zhangyiming/database/ckpt/pretrained/LaST0_Pretrain_AE_chunk16/tfmr84```85 86Files:87 88```text89config.json90model.safetensors91preprocessor_config.json92processor_config.json93special_tokens_map.json94tokenizer.json95tokenizer_config.json96```97 98This directory is used as both the action expert and processor/tokenizer source99by the MoT2 scripts. Do not replace it with Janus-Pro-1B.100 101### RLBench102 103Expected target root:104 105```text106/mnt/nas/zhangyiming/database/rlbench107```108 109The full source `rlbench` directory is included. The key training files used by110the current scripts include:111 112```text113rlbench/train/json/train_action_chunk1_sumpos_lastrot.json114rlbench/train/json/train_action_chunk1_sumpos_lastrot_statistics.json115rlbench/train/json/cosmos_text_cache_rlbench_keyframe/116rlbench/train/images/117```118 119The JSON references image paths such as:120 121```text122/mnt/nas/zhangyiming/database/rlbench/train/images/...123```124 125### LIBERO126 127Expected target path:128 129```text130/mnt/nas/zhangyiming/database/data/libero_training_data_last05_lastest/libero_spatial_20hz_224_dual131```132 133Included files/directories:134 135```text136train_with_atomic_action.json137train.json138train_rewritten_prompts.json139train_statistics.json140videos/141cosmos_text_cache/142cosmos_text_cache_raw_full_concat/143cosmos_text_cache_rewritten_prompts_raw_full_concat/144```145 146The JSON references video paths such as:147 148```text149/mnt/nas/zhangyiming/database/data/libero_training_data_last05_lastest/libero_spatial_20hz_224_dual/videos/...150```151 152## Main Training Entrypoints153 154RLBench:155 156```bash157cd /mnt/nas/zhangyiming/last05_beta/last05_mot2_action158bash scripts/train_mot2_rlbench_keyframe.sh159```160 161LIBERO:162 163```bash164cd /mnt/nas/zhangyiming/last05_beta/last05_mot2_action165bash scripts/train_mot2.sh166```167