segmind/tiny-sd-mxfinetune

1---2license: creativeml-openrail-m3base_model: SG161222/Realistic_Vision_V4.04datasets:5- recastai/LAION-art-EN-improved-captions6tags:7- stable-diffusion8- stable-diffusion-diffusers9- text-to-image10- diffusers11inference: true12---13    14# Text-to-image Distillation15 16This pipeline was distilled from **SG161222/Realistic_Vision_V4.0** on a Subset of **recastai/LAION-art-EN-improved-captions** dataset. Below are some example images generated with the tiny-sd model. 17 18![val_imgs_grid](./grid_tiny.png)19 20 21This Pipeline is based upon [the paper](https://arxiv.org/pdf/2305.15798.pdf). Training Code can be found [here](https://github.com/segmind/BKSDM).22 23## Pipeline usage24 25You can use the pipeline like so:26 27```python28from diffusers import DiffusionPipeline29import torch30 31pipeline = DiffusionPipeline.from_pretrained("segmind/tiny-sd-mxtune", torch_dtype=torch.float16)32prompt = "Portrait of a pretty girl"33image = pipeline(prompt).images[0]34image.save("my_image.png")35```36 37## Training info38 39These are the key hyperparameters used during training:40 41* Steps: 12500042* Learning rate: 1e-443* Batch size: 3244* Gradient accumulation steps: 445* Image resolution: 51246* Mixed-precision: fp1647 48## Finetune info49 50These are the key hyperparameters used during training:51 52* Steps: 8000 / 10000053* Learning rate: 1e-554* Batch size: 2455* Gradient accumulation steps: 156* Image resolution: 76857* Mixed-precision: fp1658 59## Speed Comparision60 61We have observed that the distilled models are upto 80% faster than the Base SD1.5 Models. Below is a comparision on an A100 80GB.62 63![graph](./graph.png)64![comparision](./comparision1.png)65 66Below is the code for benchmarking the speeds67 68