KennyInPA17844/minimax-h3-hybrid-ref2va-demo

MiniMax-H3 Hybrid · fl2va base + ref2va reference pathway

`smhfacct/Minimax-H3-fl2va-ref2va-hybrid-models` is a tensor-level merge of MiniMax's two released MiniMax-H3 checkpoints. fl2va supplies the attention, MLP, norm, token-refiner and output weights that govern raw fidelity; the per-block adaln_proj modulation projections of the later transformer blocks come from ref2va, the only released checkpoint trained on image / video / audio reference conditioning. The result is a drop-in ref2va replacement that keeps fl2va's output quality — and, like every H3 branch, it generates video and its soundtrack jointly inside one 33B diffusion transformer.

What this Space runs

A single resident transformer serves both pipelines — it is transformer on the keyframe branch and transformer_ref on the reference branch — which is exactly what the merge exists for, and halves resident weights compared with holding both official checkpoints.

Two modes

Text / keyframes → video. A prompt on its own, or anchored to a first and/or last keyframe (t2va / fl2va).

References → video. A prompt plus up to a subject image, a motion clip and a voice track (ref2va). At least one image or video reference is required, an audio reference needs a visual one alongside it, and the output must be at least five seconds.

Notes

• —Variants. The repository publishes b15-49, b20-49, b25-49 and b30-49, differing only in how many later blocks take their adaln_proj weights from ref2va: lower numbers follow the reference more closely, higher numbers keep more of fl2va's fidelity. Set H3_TRANSFORMER_FILE when duplicating this Space to try another.
• —Frames. MiniMax-H3's video VAE only decodes 17n + 5 frames at 24 fps, so a requested duration is snapped up (5 s → 124 frames → 5.17 s).
• —Audio. Both VAEs stay in float32 on purpose; a bfloat16 audio VAE decodes the soundtrack about 20 dB too quiet.
• —Turbo. Comfy-Org's adapters already carry their training scale in their alpha tensors, so they are applied at strength 1.0 with block caching disabled.
• —License. The weights inherit MiniMax's original H3 license terms; the merge grants nothing beyond them.

Example asset credits

• —subject.jpg — royalty-free Pexels studio portrait by Oliver Dohrn (source).
• —voice.wav — LibriSpeech dev-clean utterance 1462-170145-0022, CC BY 4.0.
• —dunes.jpg, beach_first.jpg, beach_last.jpg — from `linoyts/repo-to-space-example-inputs`.
• —motion_dance.mp4 — from `linoyts/repo-to-space-example-videos`.