hqfang/sam2act-datasets

SAM2Act SAM2Act is a multi-view robotics transformer policy for robotic manipulation. Built on RVT-2, it combines multi-resolution upsampling with visual embeddings from the SAM2 foundation model to improve 3D action prediction, multitask learning, and generalization. SAM2Act+ extends this policy with a memory bank, memory encoder, and memory attention so the agent can condition on prior observations and actions for spatial memory-dependent tasks. For full project details, code… See the full description on the dataset page: https://huggingface.co/datasets/hqfang/sam2act-datasets.