Modelpublic

bghira/AnyFlow-Wan2.1-T2V-1.3B-LoRA

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes136downloads

Model Card

AnyFlow Wan2.1 T2V 1.3B LoRA

This repository contains an extracted PEFT LoRA approximation of `nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers` against the original `Wan-AI/Wan2.1-T2V-1.3B-Diffusers`.

The adapter was extracted from the transformer weight delta with:

—target modules: all linear layers shared by the base and target transformer
—rank: 256
—alpha: 256
—dtype: float16
—sidecar tensors: AnyFlow condition_embedder.delta_embedder.*

The included safetensors file is:

text

anyflow-wan2.1-t2v-1.3b_all-linear_rank256_anyflow-sidecar.safetensors

Important Loading Note

This is not a plain Diffusers pipeline.load_lora_weights(...) adapter.

AnyFlow changes the Wan transformer time-conditioning path by adding a trained condition_embedder.delta_embedder. PEFT does not load those extra full-weight tensors by itself, so this repository includes monkeypatch.py.

Use load_anyflow_lora(...) from monkeypatch.py to:

1.enable AnyFlow time conditioning on the transformer when needed,
2.copy the sidecar delta_embedder tensors, and
3.load and activate the PEFT LoRA adapter.

Example With NVLabs/AnyFlow

Clone the AnyFlow source and install its requirements:

bash

git clone https://github.com/NVLabs/AnyFlow.git
cd AnyFlow
pip install -r requirements.txt --no-build-isolation

Download monkeypatch.py from this repository into the AnyFlow checkout, then run:

python

import torch
from diffusers.utils import export_to_video

from far.models.transformer_far_wan_model import FAR_Wan_Transformer3DModel
from far.pipelines.pipeline_wan_anyflow import WanAnyFlowPipeline
from far.schedulers.scheduling_flowmap_euler_discrete import FlowMapDiscreteScheduler
from monkeypatch import load_anyflow_lora

base_model = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
adapter_id = "bghira/AnyFlow-Wan2.1-T2V-1.3B-LoRA"

transformer = FAR_Wan_Transformer3DModel.from_pretrained(base_model, subfolder="transformer")
load_anyflow_lora(transformer, adapter_id)

scheduler = FlowMapDiscreteScheduler.from_pretrained(
    "nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers",
    subfolder="scheduler",
)

pipe = WanAnyFlowPipeline.from_pretrained(
    base_model,
    transformer=transformer,
    scheduler=scheduler,
).to("cuda", dtype=torch.bfloat16)

prompt = (
    "CG game concept digital art, a majestic elephant with a vibrant tusk and sleek fur "
    "running swiftly towards a herd of its kind. The elephant has a calm yet determined "
    "expression, with its ears flapping slightly as it moves at high speed. The landscape "
    "is vast savanna with rolling hills, tall grasses, and scattered acacia trees."
)

video = pipe(
    prompt=prompt,
    height=480,
    width=832,
    num_frames=81,
    num_inference_steps=4,
    generator=torch.Generator("cuda").manual_seed(0),
).frames[0]

export_to_video(video, "anyflow_lora_demo.mp4", fps=16)

Using The Patched AnyFlow Demo

If your AnyFlow checkout has LoRA support added to demo.py, run:

bash

python demo.py \
  model_path=Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  task_type=t2v \
  save_dir=results/demo/AnyFlow-Wan2.1-T2V-1.3B-LoRA \
  lora_path=bghira/AnyFlow-Wan2.1-T2V-1.3B-LoRA

The upstream pull request for this support is: https://github.com/NVlabs/AnyFlow/pull/7

Extraction Command

The adapter was produced with an extraction script equivalent to:

bash

python scripts/extract_anyflow_peft_lora.py \
  Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers \
  anyflow-wan2.1-t2v-1.3b_all-linear_rank256_anyflow-sidecar.safetensors \
  --rank 256 \
  --alpha 256 \
  --device cuda \
  --dtype float16 \
  --target-modules all-linear \
  --anyflow-sidecar

Limitations

This is an extracted low-rank approximation, not NVIDIA's full checkpoint. It is intended for experimentation with a smaller adapter-style representation of the AnyFlow Wan 1.3B transformer delta.

For exact reproduction of the released AnyFlow model, use `nvidia/AnyFlow-Wan2.1-T2V-1.3B-Diffusers`.