dataautogpt3/ProteusSigma

SDXL-ProteusSigma Training with ZTSNR and NovelAI V3 Improvements

• —[x] 10k dataset proof of concept (completed)

• —[x] 500k+ dataset finetune (completed) [Note: not aesthetically tuned whatsoever]

• —[ ] 12M million dataset finetune (planned)

<style> .logo-container { position: relative; text-align: center; margin: 40px 0; }

.text-layer { font-family: 'Arial Black', 'Helvetica', sans-serif; font-size: 72px; font-weight: bold; white-space: nowrap; }

.text-base { position: relative; color: #ff71ce; text-shadow: 2px 2px 0 #ff00ff; }

.text-overlay { position: absolute; left: 50%; top: 50%; transform: translate(-49%, -47%); / Slightly offset / color: #01cdfe; text-shadow: -2px -2px 0 #00ffff; opacity: 0.8; mix-blend-mode: screen; }

.sigma { color: #00ffff; text-shadow: 2px 2px 0 #ff00ff, -2px -2px 0 #00ffff; } </style>

<div class="logo-container"> <div class="text-layer text-overlay"> Proteus<span class="sigma">Σ</span> </div> <div class="text-layer text-base"> Proteus<span class="sigma">Σ</span> </div> </div>

Example Outputs

<style> .gallery { display: flex; flex-direction: row; flex-wrap: wrap; gap: 10px; justify-content: center; align-items: center; width: 100%; padding: 10px; }

.gallery-item { flex: 0 0 300px; margin: 0; position: relative; }

.gallery-item.large { / New class for larger item / flex: 0 0 340px; }

.gallery img { width: 300px; cursor: pointer; transition: transform 0.2s; border-radius: 8px; }

.gallery-item.large img { / Larger size for last image / width: 512px; }

.gallery img:hover { transform: scale(1.05); }

.caption { position: absolute; bottom: 0; left: 0; right: 0; background: rgba(0, 0, 0, 0.4); color: white; padding: 8px; font-size: 11px; border-bottom-left-radius: 8px; border-bottom-right-radius: 8px; opacity: 0.7; transition: opacity 0.3s ease; }

.gallery-item:hover .caption { opacity: 0.2; }

.modal { display: none; position: fixed; z-index: 1000; top: 0; left: 0; width: 100%; height: 100%; background-color: rgba(0,0,0,0.9); padding: 20px; box-sizing: border-box; }

.modal img { max-width: 90%; max-height: 90vh; margin: auto; display: block; position: relative; top: 50%; transform: translateY(-50%); }

.modal.active { display: block; } </style>

<div class="gallery"> <div class="gallery-item"> <img src="https://huggingface.co/dataautogpt3/ProteusSigma/resolve/main/example.png" alt="Example Output 1" onclick="showImage(this.src)"/> </div> <div class="gallery-item"> <img src="https://huggingface.co/dataautogpt3/ProteusSigma/resolve/main/example2.png" alt="Example Output 2" onclick="showImage(this.src)"/> </div> <div class="gallery-item"> <img src="https://huggingface.co/dataautogpt3/ProteusSigma/resolve/main/example3.png" alt="Example Output 3" onclick="showImage(this.src)"/> </div> <div class="gallery-item"> <img src="https://huggingface.co/dataautogpt3/ProteusSigma/resolve/main/example4.png" alt="Example Output 4" onclick="showImage(this.src)"/> </div> <div class="gallery-item large"> <!-- Added 'large' class --> <img src="https://huggingface.co/dataautogpt3/ProteusSigma/resolve/main/example5.png" alt="Example Output 5" onclick="showImage(this.src)"/> </div> </div>

<div class="modal" onclick="this.classList.remove('active')"> <img id="modal-img" src="" alt="Full size image"/> </div>

<script> function showImage(src) { document.getElementById('modal-img').src = src; document.querySelector('.modal').classList.add('active'); } </script>

Combined Proteus and Mobius datasets with ZTSNR and NovelAI V3 Improvements

CUSTOM INFERENCE IS REQUIRED FOR BEST RESULTS!

https://github.com/DataCTE/SDXL-Training-Improvements/tree/main/Comfyui-zsnrnode

use this comfyui custom node from the training repo.

and the workflow here: https://github.com/DataCTE/SDXL-Training-Improvements/blob/main/Comfyui-zsnrnode/ztsnr%2Bv-pred.json

Model Details

• —Model Type: SDXL Fine-tuned with ZTSNR and NovelAI V3 Improvements
• —Base Model: stabilityai/stable-diffusion-xl-base-1.0
• —Training Dataset: 500,000 high-quality images
• —License: Apache 2.0

Key Features

• —Zero Terminal SNR (ZTSNR) implementation
• —Increased σ_max ≈ 20000.0 (NovelAI research)
• —High-resolution coherence enhancements

Training Details

Training Configuration

• —Learning Rate: 4e-7
• —Batch Size: 8
• —Gradient Accumulation Steps: 8
• —Epochs: 80
• —Optimizer: AdamW
• —Precision: bfloat16

Repository and Resources

• —GitHub Repository: SDXL-Training-Improvements
• —Training Code: Available in the repository
• —Documentation: Implementation Details
• —Issues and Support: GitHub Issues

Citation

@article{ossa2024improvements,
  title={Improvements to SDXL in NovelAI Diffusion V3},
  author={Ossa, Juan and Doğan, Eren and Birch, Alex and Johnson, F.},
  journal={arXiv preprint arXiv:2409.15997v2},
  year={2024}
}