ZeyuLing/Motius-MoMask-HumanML3D

<!-- This model card is synchronized from docs/modelzoo/momask.md by tools/syncmodelzoocards.py. -->

MoMask — Generative Masked Modeling of 3D Human Motions

Text-to-motion baseline integrated into the motius Model Zoo. Our reproduction is fully self-contained and independent of `ref_repo`: the RVQ-VAE tokenizer, the masked generative transformer, the residual transformer and the length estimator are all vendored into motius.models.motion.momask._momask, preserving numerical parity with the released HumanML3D checkpoints. The CLIP ViT-B/32 text encoder is reloaded by name only for legacy lightweight artifacts; new motius artifacts include clip.safetensors.

Weights

Self-contained motius artifact (diffusers-style from_pretrained):

Use the published artifact directly from the Hub:

from motius.pipelines.momask import MoMaskPipeline

pipe = MoMaskPipeline.from_pretrained(
    "ZeyuLing/Motius-MoMask-HumanML3D",
    device="cuda",
)
motions = pipe.infer_t2m(
    ["a person walks forward then sits down"],
    [120],
)  # list of (T, 263)

The artifact is produced from the released upstream .tar checkpoints with scripts/eval/convert_momask_checkpoint.py (--verify asserts bit-identical generation after the round-trip):

python3 scripts/eval/convert_momask_checkpoint.py \
    --weights_root ref_repo/Momask/weights \
    --out_dir checkpoints/momask/humanml3d \
    --verify

Use it:

from motius.pipelines.momask import MoMaskPipeline

pipe = MoMaskPipeline.from_pretrained("checkpoints/momask/humanml3d", device="cuda")
# fixed length (frames @ 20 fps):
motions = pipe.infer_t2m(["a person walks forward then sits down"], [120])
# or let the length estimator pick the length:
motions = pipe.infer_t2m(["a person walks forward then sits down"])  # list of (T, 263)

You can also drive it directly from the released weights, no conversion needed:

bundle = MoMaskBundle(weights_root="ref_repo/Momask/weights")

Motion representation

HumanML3D-263, the standard redundant T2M feature (Guo et al.), 20 fps, 22-joint SMPL skeleton. Per frame (263 dims):

The RVQ-VAE tokenizes this with unit_length = 4 (one token ≈ 4 frames), so a 196-frame motion maps to 49 tokens × 6 quantizers.

Generation

Three vendored stages (parity with scripts/eval/momask_infer_h3d_test.py):

1. 1.MaskTransformer — confidence-based masked iterative decoding of the base (q=0) token map, classifier-free guidance cond_scale≈4 over time_steps≈10 iterations, cosine mask schedule, topkr≈0.9, temperature=1.0.
2. 2.ResidualTransformer — autoregressively predicts quantizers q=1..5 conditioned on the lower layers (cond_scale≈5, temperature=1.0).
3. 3.RVQVAE.forward_decoder — de-quantizes (T, 6) tokens and decodes to the 263-dim feature, then de-normalised with the training Mean / Std.

Evaluation

Generation under the official HumanML3D protocol (standard test split, native 263-dim @ 20 fps, first caption) and scoring with the persisted HumanML263Evaluator. Reproduce with:

# 1) generate
python3 scripts/eval/momask_t2m_h3d263.py \
    --model_path checkpoints/momask/humanml3d \
    --out_dir outputs/evaluation/momask_h3d263_official/momask_263
# 2) score with the HumanML3D-263 evaluator
python3 scripts/eval/verify_evaluators.py --which hml263 \
    --hml263-pred outputs/evaluation/momask_h3d263_official/momask_263

HumanML3D-263 evaluator (native space)

(20 repeats, n = 3970; GT/real reference under the same evaluator: R-Prec 0.513 / 0.711 / 0.807, MM-Dist 2.932, Diversity 9.453.)

R-Precision, MM-Dist and Diversity match the paper essentially exactly, confirming the generation is faithfully reproduced. The small residual FID gap (0.097 vs 0.045) is a data-processing / population difference in the evaluation set (e.g. no sub-clip predictions, test-split composition), not a generation-quality gap — the decode path is verified parity-equal to the released MoMask inference (momask_infer_h3d_test.py).

MotionStreamer-272 evaluator (SMPL retarget path)

For cross-model comparison with the MotionStreamer / HYMotion-M2M evaluator, native HumanML3D-263 predictions are retargeted through the validated MDM-style chain: HML263 -> SMPL motion_135 (IK refine-80, 20 -> 30 fps) -> MotionStreamer-272 -> MotionStreamer272Evaluator.

Run details: n_repeats = 20, n_samples_used = 7392, skipped_no_pred = 0, outputs under outputs/evaluation/ms272_from263/momask_272, metrics in outputs/evaluation/ms272_from263/metrics_momask.json.

Implementation notes

• —Vendored, ref_repo-independent: motius/models/motion/momask/_momask/ holds the RVQ-VAE (vq/), the masked / residual transformers (mask_transformer/) and the masked iterative decoding entry point (inference.py). Imports are package-relative; training-only code paths are not exercised.
• —Sub-modules: vq_model / t2m_transformer / res_transformer / length_estimator (the last is optional, load_length_estimator=False).
• —CLIP: frozen ViT-B/32 lives inside the two transformers and is stored once as clip.safetensors in new artifacts. MoMaskBundle.from_pretrained passes that file path into both transformers; legacy lightweight artifacts still fall back to clip_version.
• —Normalization travels with the checkpoint: Mean.npy / Std.npy are the RVQ-VAE training stats, embedded in the artifact.
• —Guidance: classifier-free, base cond_scale=4, residual cond_scale=5.

Direct Loading

from motius import Pipeline

pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MoMask-HumanML3D")