Noob-Master60nine/Anima-Control-Pose

Anima Control — Pose (Preview-2)

⚠️ Preview-2 — still experimental. Better than Preview-1, but not finished. It will still miss poses and produce deformed bodies, fused hands, and similar artifacts. Treat it as a work-in-progress preview, not a production tool. Non-commercial use only (inherits the Anima base model license). Behaviour and weights may still change.

A native pose control adapter for the Anima v1.0 image model: condition generation on a skeleton pose map so the subject follows a target pose. Preview-2 is the same idea as Preview-1, trained at higher resolutions on a larger corpus, and shipped with a friendlier ComfyUI node.

What changed since Preview-1

• —Multi-resolution: 512, 768 and 1024. Preview-1 was 512-only; this runs at all three, and the bodies hold together much better at the larger sizes. 1024 looks best.
• —Larger training corpus than Preview-1's ~3,900 examples, which shows up as cleaner anatomy and steadier poses.
• —Better pose-following at every resolution, with the biggest gains at 768 and 1024.
• —New ComfyUI node, "Anima Pose Control." Drop in a reference photo; it detects the pose for you and renders the skeleton, and you can pick how the skeleton is drawn (thin, thick, puppet, heatmap, or with hands/face stripped). The plain thin skeleton is still the best default — it's the only one the model trained on — but a different render occasionally lands a stubborn pose when the seed alone won't.

Method

Unchanged from Preview-1: a channel-concat control-LoRA on the frozen Anima DiT.

Conditioning. The base VAE encodes the skeleton pose map into a control latent, the same latent space as the noisy image, so the control stays spatially aligned with the generation.

Fusion (`ControlEmbedder` + `ControlInitialLayer`). A zero-initialized ControlEmbedder produces control tokens that are added to the frozen base patch-embed output. Zero-init means training starts as an exact no-op (output == base) and the control contribution grows only as it earns loss, so at strength = 0 the adapter is exactly the base model.

Trainable parameters. The ControlEmbedder plus a rank-16 low-rank adapter on the transformer blocks. The base transformer, text encoder, and VAE stay frozen.

skeleton ─▶ VAE ─▶ control latent ─┐
                                   ▼  (+ zero-init ControlEmbedder)
noisy latent ─▶ patch-embed ─▶ [ControlInitialLayer] ─▶ Block×N (+ rank-16 LoRA) ─▶ output

Training

Data. (image, skeleton, caption) triples generated by Anima from a broad prompt distribution; skeletons rendered from each image's detected keypoints (DWPose, COCO-WholeBody, black background). Preview-2 uses a substantially larger corpus than Preview-1.

Final training loss ≈ 0.11 (denoising MSE, mean over the final 400 steps).

Results

Measured on held-out full-body poses (fresh generations, not seen in training). Pose agreement is body-PCK@0.1: re-detect keypoints on each output, compare to the target skeleton.

[image]

Each grid: the BASE column shows the reference and the no-control generation (same prompt, different seed — it ignores the pose); the remaining columns show the skeleton, drawn in each style, over the pose-controlled output. Control follows the pose; no-control doesn't.

Preview-1 reached ~0.59 at 512. Preview-2 matches that at 512 and pulls clearly ahead at 768 and 1024 — the gains grow with resolution.

What's still off (honest):

• —It doesn't always follow the skeleton, even a clean, correct one.
• —A thin stick figure on black isn't how anime is drawn, so the model only half-reads it; the worst artifacts (fused hands, mush) cluster where the skeleton is busiest.
• —Dynamic poses — running, jumping, sitting — are the least reliable.
• —On short or vague prompts the style flattens toward a samey default; a richer prompt fixes it.

Usage (ComfyUI)

Preview-2 uses two small custom nodes: Anima Control Apply (AnimaControlApply) applies the adapter, and Anima Pose Control (AnimaPoseControl) detects the pose from a photo and renders the skeleton for you.

Install

1. 1.Download anima_pose_preview2.safetensors into ComfyUI/models/loras/.
2. 2.Copy both folders from comfyui/ in this repo into ComfyUI/custom_nodes/: anima_control_lora/ and ComfyUI-anima-pose-control/. Restart ComfyUI. ComfyUI-Manager installs the second node's requirements.txt automatically; otherwise: pip install -r ComfyUI-anima-pose-control/requirements.txt (rtmlib, opencv-python, onnxruntime, numpy; torch and Pillow come with ComfyUI).
3. 3.Load a workflow from the menu. pose_control_demo.json is the easiest — control vs no-control side by side. Also included: pose_control.json (simple), pose_control_edit.json (single node, pick the skeleton style), pose_control_compare.json (one pose across every style at once).

anima_pose_preview2.safetensors holds both the low-rank adapter (lora.* keys) and the control embedder (control_embedder.* keys); LoraLoaderModelOnly reads the first set and Anima Control Apply reads the second, both pointing at the same file.

Strength. 0.0 is the base model with no control; 1.0 follows the skeleton (range 0–2). Higher tracks the pose more closely but can cost some image quality.

Pose detector (first run)

The Anima Pose Control node needs a pose detector (rtmlib). On first use it downloads two ONNX files (~316 MB) from this repo's detector/ folder and caches them under ~/.cache/rtmlib/hub/checkpoints/; after that it runs offline. If you see urllib ... getaddrinfo failed, your machine couldn't reach the download host — download detector/yolox_m_8xb8-300e_humanart-c2c7a14a.onnx and detector/rtmw-dw-x-l_simcc-cocktail14_270e-256x192_20231122.onnx from the Files tab by hand and drop both into that cache folder (create it if missing), then restart ComfyUI.

Roadmap

Preview-3 targets the headline weakness: the model only half-reads the skeleton. The thin lines-on-black signal seems foreign to it, so a representation bake-off is testing other renders (thicker, puppet/segmentation, depth-mannequin) to find what Anima follows best, plus the smaller fixes (default strength, dropping noisy hand points, more varied captions). The other half is the detector: the one used here was built for photos, not anime, so on art it produces noisy skeletons. Preview-3 will most likely wait on a purpose-built anime pose detector ("DWPose for anime"), then retrain on the winning representation with a lot more dynamic-pose data.

Version 1.0 comes after Preview-3, if it lands clean — the first non-preview release, trained on a good deal more data again.

Pose is the first component on a shared control harness for Anima; planned siblings are image-prompt (IP-Adapter) and face-identity conditioning.

License

These weights are a derivative of the Anima base model (`circlestone-labs/Anima`) and inherit its terms: the CircleStone Labs Non-Commercial License, and — because Anima is itself a derivative of Cosmos-Predict2 — the NVIDIA Open Model License.

The model weights are for non-commercial use only. Generated images (outputs) are not restricted by these terms and may be used commercially. See the bundled `LICENSE` for the full text.

Support

Building these models means mining and labeling a lot of images and renting GPUs to train on them. If they're useful to you and you want to chip in, it's appreciated and never expected: https://ko-fi.com/claquasse

Citation

@misc{anima_control_pose_preview2,
  title  = {Anima Control --- Pose (Preview-2)},
  author = {Claquasse},
  year   = {2026},
  note   = {Preview-2 multi-resolution pose control adapter for Anima v1.0},
  howpublished = {\url{https://huggingface.co/Claquasse/Anima-Control-Pose}}
}

Built on Anima (CircleStone Labs), the Cosmos-Predict2 transformer architecture, and the diffusion-pipe training framework.