Jl-wei/ui-diffuser-v2

1---2tags:3  - text-to-image4  - stable-diffusion5  - lora6  - diffusers7language:8- en9library_name: diffusers10pipeline_tag: text-to-image11base_model: stabilityai/stable-diffusion-2-base12instance_prompt: "Mobile app:"13---14 15# UI-Diffuser-V216UI-Diffuser-V2 is fine tuned from "stabilityai/stable-diffusion-2-base" with the [SCapRepo dataset](https://paperswithcode.com/dataset/gpscap) for mobile UI generation.17 18A demo using diffusion model and large language model for UI generation is available at https://github.com/Jl-wei/ai-gen-ui19 20 21## Using with Diffusers22```python23import torch24from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler25 26model_id = "stabilityai/stable-diffusion-2-base"27scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")28pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, torch_dtype=torch.float16)29 30lora_path = "Jl-wei/ui-diffuser-v2"31pipe.load_lora_weights(lora_path)32pipe.to("cuda")33 34prompt = "Mobile app: health monitoring report"35images = pipe(prompt, num_inference_steps=30, guidance_scale=7.5, height=512, width=288, num_images_per_prompt=10).images36 37columns = 538fig = plt.figure(figsize=(20,10))39for i, image in enumerate(images):40    plt.subplot(int(len(images) / columns), columns, i + 1)41    plt.imshow(image)42for ax in fig.axes:43    ax.axis("off")44```45 46## Citation47If you find our work useful, please cite our paper:48```bibtex49@article{wei2024aiinspired,50  author={Wei, Jialiang and Courbis, Anne-Lise and Lambolais, Thomas and Dray, Gérard and Maalej, Walid},51  journal={IEEE Software}, 52  title={On AI-Inspired User Interface Design}, 53  year={2025},54  volume={42},55  number={3},56  pages={50-58},57  doi={10.1109/MS.2025.3536838}58}59 60```61Please note that the code and model can only be used for academic purpose.62 63### UI-Diffuser-V164This model, UI-Diffuser-V2, represents the second version of the UI-Diffuser model.65 66The initial version, UI-Diffuser-V1, was introduced in our paper titled [Boosting GUI Prototyping with Diffusion Models](https://ieeexplore.ieee.org/abstract/document/10260853)