Modelpublic

mingyi456/SRPO-DF11

sourceHugging Faceotherupdated 11mo agoView on Hugging Face

1likes44downloads

Model Card

For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11

Feel free to request for other models for compression as well, although compressing models that do not use the Flux architecture might be tricky for me.

This compressed model was made from rockerBOO/flux.1-dev-SRPO's BF16 quantization. Thanks to rockerBOO, without which I would not have been able to directly work with. (My PC only has 48 GB of system RAM, too little to work with a 12B model in FP32 precision)

How to Use

`diffusers`

1.Install the DFloat11 pip package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):

bash

    pip install dfloat11[cuda12]
    # or if you have CUDA version 11:
    # pip install dfloat11[cuda11]

1.To use the DFloat11 model, run the following example code in Python:

python

    import torch
    from diffusers import FluxPipeline, FluxTransformer2DModel
    from dfloat11 import DFloat11Model
    with no_init_weights():
      transformer = FluxTransformer2DModel.from_config(
          FluxTransformer2DModel.load_config(
              "black-forest-labs/FLUX.1-dev",
              subfolder="transformer"
          ),
          torch_dtype=torch.bfloat16
      ).to(torch.bfloat16)
    
    pipe = FluxPipeline.from_pretrained(
        "black-forest-labs/FLUX.1-dev",
        transformer=transformer,
        torch_dtype=torch.bfloat16
    )
    DFloat11Model.from_pretrained('mingyi456/SRPO-DF11', device='cpu', bfloat16_model=pipe.transformer)
    pipe.enable_model_cpu_offload()
    prompt = "A futuristic cityscape at sunset, with flying cars, neon lights, and reflective water canals"
    image = pipe(
        prompt,
        guidance_scale=3.5,
        num_inference_steps=30,
        max_sequence_length=256,
        generator=torch.Generator("cpu").manual_seed(0)
    ).images[0]
    image.save("SPRO.png")

ComfyUI

Refer to this model page instead, and follow the instructions there.