khania/motion-mgvqvae

MotionVQVAE

A Multi-Group Vector Quantized VAE (MG-VQVAE) trained on the MotionMillion dataset for motion tokenization and reconstruction.

⚠️ License Notice: This model is released under CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0). The training data includes datasets with mixed licensing terms, some of which restrict commercial use. This model is for research and non-commercial use only.

📋 Body Model: This model was trained on motion data using the SMPL body model (22 joints). Input motions must be in SMPL skeleton format.

Model Description

MotionVQVAE learns to compress human motion sequences into discrete tokens using a Multi-Group Vector Quantization approach. The model can:

• —Tokenize motion sequences into discrete tokens for downstream generation tasks
• —Reconstruct motions from tokens with high fidelity
• —Compress variable-length motions with 4× temporal downsampling

Multi-Group VQ Architecture

Instead of a single codebook, MG-VQVAE uses 64 parallel groups, each with its own 512-code codebook. This provides:

• —Effective codebook size: $512^{64} \approx 2.47 \times 10^{173}$ combinations
• —Fine-grained control over different motion aspects
• —Better reconstruction quality through distributed quantization

Usage

Installation

pip install torch huggingface_hub numpy

Download the Model Code

Download motion_vqvae_hf.py from this repository or copy it to your project.

Quick Start

from motion_vqvae_hf import MotionVQVAE
import numpy as np

# Load model (auto-downloads from HuggingFace)
model = MotionVQVAE.from_pretrained("khania/motion-vqvae")

# Prepare motion data (272-dim absolute root format)
motion = np.random.randn(120, 272).astype(np.float32)  # Replace with real motion

# Encode motion to tokens
tokens = model.encode(motion)  # Returns token indices for each group
print(f"Tokens shape: {tokens.shape}")  # (64, 1, 30) - 64 groups, batch=1, T/4 timesteps

# Decode tokens back to motion
motion_recon = model.decode(tokens)
print(f"Reconstructed motion shape: {motion_recon.shape}")  # (1, 120, 272)

# Full forward pass (encode + decode)
motion_recon, tokens = model(motion)

Batch Processing

# Process multiple motions
motions = [
    np.random.randn(100, 272).astype(np.float32),
    np.random.randn(150, 272).astype(np.float32),
    np.random.randn(80, 272).astype(np.float32),
]

# Encode batch (will pad to max length)
tokens = model.encode_batch(motions)

# Decode batch
motions_recon = model.decode_batch(tokens)

Access Codebook

# Get quantized embeddings for analysis
embeddings = model.get_codebook_embeddings()
print(f"Codebook shape: {embeddings.shape}")  # (64, 512, 8) - 64 groups, 512 codes, 8-dim each

Model Architecture

Motion Format

The model expects 272-dimensional motion features in absolute root format based on the SMPL body model (22 joints).

SMPL Body Model Requirement

This model was trained exclusively on motion data represented using the SMPL body model. Your input motions must:

• —Use the SMPL skeleton with 22 joints
• —Follow the SMPL joint ordering
• —Be converted to the 272-dimensional HumanML3D-style representation

If your motion data uses a different skeleton (e.g., CMU, Mixamo, custom rigs), you must first retarget it to SMPL before using this model.

Feature Dimensions

The model automatically normalizes input motions using the bundled mean/std statistics.

Training Details

Loss Weights

Performance

Final evaluation metrics at 300K iterations:

Per-Component Reconstruction Loss (Eval)

Evaluation Results

Quantitative evaluation on the MotionMillion test set using rotation-based reconstruction (global positions computed via forward kinematics).

Rotation & Position Errors

Motion Quality

Feature-Space MSE (Denormalized)

Codebook Utilization

Files in This Repository

Use Cases

• —Motion Generation: Tokenize motions for autoregressive or diffusion-based generation
• —Motion Compression: Efficiently store motion data as discrete tokens
• —Motion Editing: Manipulate tokens for motion modification
• —Downstream Tasks: Use tokens as input for text-to-motion models

Limitations

• —Trained on English text descriptions only (for associated metadata)
• —Motion format is specific to HumanML3D-style 272-dim representation
• —4× temporal downsampling may lose very fine-grained details
• —Best performance on motions similar to training distribution (daily activities, sports, etc.)

Citation

@article{motionmillion2026,
  title={MotionMillion: A Large-Scale Motion-Language Dataset},
  author={...},
  year={2026}
}

License

CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0 International)