TheStageAI/Elastic-stable-diffusion-3.5-large
18
1---2license: apache-2.03base_model:4- stabilityai/stable-diffusion-3.5-large5base_model_relation: quantized6pipeline_tag: text-to-image7---8 9 10# Elastic model: Fastest self-serving models. Stable Diffusion 3.5 Large.11 12Elastic models are the models produced by TheStage AI ANNA: Automated Neural Networks Accelerator. ANNA allows you to control model size, latency and quality with a simple slider movement. For each model, ANNA produces a series of optimized models:13 14* __XL__: Mathematically equivalent neural network, optimized with our DNN compiler.15 16* __S__: The fastest model, with accuracy degradation less than 2%.17 18 19__Goals of Elastic Models:__20 21* Provide the fastest models and service for self-hosting.22* Provide flexibility in cost vs quality selection for inference.23* Provide clear quality and latency benchmarks.24* Provide interface of HF libraries: transformers and diffusers with a single line of code.25* Provide models supported on a wide range of hardware, which are pre-compiled and require no JIT.26 27> It's important to note that specific quality degradation can vary from model to model. For instance, with an S model, you can have 0.5% degradation as well.28 2930 3132 33-----34 35## Inference36 37Currently, our demo model supports 512x512 - 1024x1024 and batch sizes 1-4. This will be updated in the near future.38To infer our models, you just need to replace `diffusers` import with `elastic_models.diffusers`:39 40```python41import torch42from elastic_models.diffusers import StableDiffusion3Pipeline43 44model_name = 'stabilityai/stable-diffusion-3.5-large'45hf_token = ''46device = torch.device("cuda")47 48pipeline = StableDiffusion3Pipeline.from_pretrained(49 model_name,50 torch_dtype=torch.bfloat16,51 token=hf_token,52 mode='S'53)54pipeline.to(device)55 56prompts = ["A cat holding a sign that says hello world"]57output = pipeline(prompt=prompts)58 59for prompt, output_image in zip(prompts, output.images):60 output_image.save((prompt.replace(' ', '_') + '.png'))61```62 63### Installation64 65 66__System requirements:__67* GPUs: H100, B20068* CPU: AMD, Intel69* Python: 3.10-3.1270 71 72To work with our models just run these lines in your terminal:73 74```shell75pip install thestage76pip install 'thestage-elastic-models[nvidia]' --extra-index-url https://thestage.jfrog.io/artifactory/api/pypi/pypi-thestage-ai-production/simple77 78# or for blackwell support79pip install 'thestage-elastic-models[blackwell]' --extra-index-url https://thestage.jfrog.io/artifactory/api/pypi/pypi-thestage-ai-production/simple80pip install -U --pre torch --index-url https://download.pytorch.org/whl/nightly/cu12881pip install -U --pre torchvision --index-url https://download.pytorch.org/whl/nightly/cu12882 83 84pip install flash_attn==2.7.3 --no-build-isolation85pip uninstall apex86```87 88Then go to [app.thestage.ai](https://app.thestage.ai), login and generate API token from your profile page. Set up API token as follows:89 90```shell91thestage config set --api-token <YOUR_API_TOKEN>92```93 94Congrats, now you can use accelerated models!95 96----97 98## Benchmarks99 100Benchmarking is one of the most important procedures during model acceleration. We aim to provide clear performance metrics for models using our algorithms.101 102### Quality benchmarks103 104For quality evaluation we have used: PSNR and SSIM. PSNR and SSIM were computed using outputs of original model.105| Metric/Model | S | XL | Original |106|---------------|---|----|----------|107| PSNR | 20.78 | 29.13 | inf |108| SSIM | 0.81 | 0.95 | 1.0 |109 110 111### Latency benchmarks112 113Time in seconds to generate one image 1024x1024114| GPU/Model | S | XL | Original |115|-----------|-----|----|----------|116| H100 | 3.10 | 3.80 | 6.55 |117| B200 | 1.76 | 2.27 | 4.81 |118 119 120## Links121 122* __Platform__: [app.thestage.ai](https://app.thestage.ai)123<!-- * __Elastic models Github__: [app.thestage.ai](app.thestage.ai) -->124* __Subscribe for updates__: [TheStageAI X](https://x.com/TheStageAI)125* __Contact email__: contact@thestage.ai126