Modelpublic

tj111/object-concepts-from-motion

sourceHugging Faceapache-2.0updated 20d agoView on Hugging Face

0likes

Model Card

Object Concepts from Motion

This repository contains the inference-only Motion Object Encoder checkpoints released with Object Concepts from Motion. The checkpoints provide dense visual representations using five Swin Transformer backbone sizes.

The files contain the backbone, neck, and representation head weights. Optimizer, scheduler, message-hub, and other training state have been removed. Parameters use MMPretrain/MMEngine names and are stored as PyTorch .pth checkpoints.

Checkpoints

File	Variant	Embedding	Stage depths	Attention heads	Size
`swin_h.pth`	Swin-H / huge	384	2, 2, 18, 2	12, 24, 48, 96	3.14 GB
`swin_l.pth`	Swin-L / large	192	2, 2, 18, 2	6, 12, 24, 48	796 MB
`swin_b.pth`	Swin-B / base	128	2, 2, 18, 2	4, 8, 16, 32	362 MB
`swin_s.pth`	Swin-S / small	96	2, 2, 18, 2	3, 6, 12, 24	210 MB
`swin_t.pth`	Swin-T / tiny	96	2, 2, 6, 2	3, 6, 12, 24	124 MB

Swin-H is the Cycle 2 checkpoint. The T, S, B, and L variants are distilled from the Cycle 2 Swin-H model.

Download

Download all checkpoints into the location expected by the source repository:

bash

hf download tj111/object-concepts-from-motion \
  --include "*.pth" \
  --local-dir checkpoints

Or download one checkpoint from Python:

python

from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="tj111/object-concepts-from-motion",
    filename="swin_h.pth",
)

Usage

Clone the source repository, install its lightweight inference dependencies, and download the weights:

bash

git clone https://github.com/TJ12342/object-concepts-from-motion.git
cd object-concepts-from-motion
python -m pip install -r requirements.txt
hf download tj111/object-concepts-from-motion \
  --include "*.pth" \
  --local-dir checkpoints

Run the feature visualization demo with an explicitly selected architecture:

bash

python tools/feature_visualization.py assets/pic1.png \
  --arch huge \
  --output assets/pic1_pca.png

For direct loading, use the security-restricted checkpoint mode used by the source repository:

python

import torch

checkpoint = torch.load(
    checkpoint_path,
    map_location="cpu",
    mmap=True,
    weights_only=True,
)
state_dict = checkpoint.get("state_dict", checkpoint)

The repository includes a standalone PyTorch implementation and adapters for DCDepth, BEVFormer, and SparseOcc. See the source repository for architecture selection, checkpoint conversion, preprocessing, and downstream instructions.

Limitations

These are representation checkpoints, not complete task-specific models. DCDepth, BEVFormer, and SparseOcc evaluation can require separately trained decoders, prediction heads, or full task checkpoints. The files are not packaged for transformers.AutoModel or the hosted Hugging Face Inference API.

Integrity

SHA-256 checksums are provided in SHA256SUMS.