Modelpublic

ddz16/CamInject-4B

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes22downloads

Model Card

CamInject-4B

Camera-movement understanding model that injects frozen VGGT camera tokens into Qwen/Qwen3-VL-4B-Instruct. Given a video, it outputs structured JSON describing every camera-movement segment.

—Paper: Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation
—Project page: https://ddz16.github.io/cammotion.github.io
—Code: https://github.com/ddz16/CamDistill

⚠️ This model cannot be loaded with plain 🤗 Transformers. It requires a custom model type (registered via a plugin) and runs VGGT online to produce camera tokens. Loading it as a standard Qwen3VLForConditionalGeneration would not work correctly. Use the CamDistill repo.

Usage

Clone the CamDistill repo and clone VGGT-Omega (set VGGT_OMEGA_REPO, see the repo's setup). CamInject runs VGGT online during inference:

bash

VGGT_TEACHER_TYPE=vggt_omega \
python camera_movement_sft/infer_single.py \
    --model ddz16/CamInject-4B \
    --video /path/to/video.mp4 \
    --variant caminject

See the repo's README for environment setup and batch evaluation.