Delta-Vector/MS3.2-Austral-Winton
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Austral 24B Winton</title> <link href="" rel="stylesheet"> <style> body { font-family: 'Roboto Slab', serif; background: linear-gradient(135deg, #8B4513 0%, #A0522D 25%, #CD853F 50%, #D2691E 75%, #8B4513 100%); background-size: 400% 400%; animation: prehistoricShift 20s ease-in-out infinite; color: #2F1B14; margin: 0; padding: 0; font-size: 16px; min-height: 100vh; } @keyframes prehistoricShift { 0%, 100% { background-position: 0% 50%; } 50% { background-position: 100% 50%; } } .container { margin: 20px; background: linear-gradient(145deg, #F4E4BC 0%, #DEB887 100%); padding: 20px; border-radius: 15px; box-shadow: 0 8px 25px rgba(0, 0, 0, 0.4), inset 0 2px 5px rgba(255, 255, 255, 0.3); border: 4px solid #8B4513; position: relative; overflow: hidden; } .container::before { content: ''; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background-image: radial-gradient(circle at 20% 80%, rgba(139, 69, 19, 0.1) 0%, transparent 50%), radial-gradient(circle at 80% 20%, rgba(160, 82, 45, 0.1) 0%, transparent 50%); pointer-events: none; } .header h1 { font-family: 'Cinzel', serif; font-size: 32px; color: #5D2E0C; margin: 0 0 20px 0; text-align: center; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3); letter-spacing: 2px; position: relative; } .section { margin-top: 30px; position: relative; } .section h2 { font-family: 'Cinzel', serif; font-size: 26px; color: #5D2E0C; text-align: center; margin-bottom: 20px; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.2); letter-spacing: 1px; } .info p { color: #2F1B14; line-height: 1.7; font-size: 16px; text-shadow: 0 1px 1px rgba(255, 255, 255, 0.5); } .info img { width: 85%; border-radius: 12px; margin: 0 auto 15px; display: block; box-shadow: 0 0 25px rgba(0, 0, 0, 0.4); border: 3px solid #8B4513; filter: sepia(20%) contrast(110%); } a { color: #5D2E0C; text-decoration: none; transition: all 0.3s ease; font-weight: 500; } a:hover { color: #8B4513; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.2); } .button { display: inline-block; background: linear-gradient(145deg, #CD853F, #D2691E); color: #2F1B14; padding: 12px 24px; border-radius: 8px; cursor: pointer; text-decoration: none; transition: all 0.3s ease; box-shadow: 0 4px 8px rgba(0, 0, 0, 0.2); border: 2px solid #8B4513; } .button:hover { background: linear-gradient(145deg, #D2691E, #CD853F); box-shadow: 0 6px 15px rgba(139, 69, 19, 0.4); transform: translateY(-2px); } pre { background: linear-gradient(145deg, #F5DEB3, #DEB887); padding: 20px; border-radius: 8px; overflow-x: auto; border: 2px solid #8B4513; box-shadow: inset 0 2px 5px rgba(0, 0, 0, 0.1); } code { font-family: 'Courier New', monospace; color: #2F1B14; } .info-card { background: linear-gradient(145deg, #F5DEB3, #DEB887); border: 3px solid #8B4513; border-radius: 12px; overflow: hidden; box-shadow: 0 6px 15px rgba(0, 0, 0, 0.2); } .info-header { background: linear-gradient(145deg, #CD853F, #D2691E); padding: 25px; border-bottom: 2px solid #8B4513; } .info-header h3 { font-family: 'Cinzel', serif; color: #2F1B14; margin: 0 0 15px 0; font-size: 22px; text-align: center; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.2); letter-spacing: 1px; } .model-tags { display: flex; gap: 10px; flex-wrap: wrap; justify-content: center; } .model-tag { background: linear-gradient(145deg, #DEB887, #CD853F); color: #2F1B14; padding: 6px 12px; border-radius: 6px; font-size: 12px; border: 2px solid #8B4513; font-weight: 500; box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1); } .model-composition { padding: 25px; border-bottom: 2px solid #8B4513; } .model-composition h4 { font-family: 'Cinzel', serif; color: #5D2E0C; margin: 0 0 20px 0; font-size: 18px; text-align: center; letter-spacing: 1px; } .composition-list { list-style: none; padding: 0; margin: 0; display: grid; gap: 15px; } .composition-list li { color: #2F1B14; display: flex; align-items: baseline; gap: 12px; padding: 10px; background: rgba(245, 222, 179, 0.5); border-radius: 6px; border-left: 4px solid #8B4513; } .model-component { font-weight: 600; min-width: 120px; } .model-description { padding: 25px; background: linear-gradient(145deg, #F5DEB3, #F4E4BC); } .metrics-section { margin-bottom: 30px; } .metrics-section details { background: linear-gradient(145deg, #F5DEB3, #DEB887); border: 3px solid #8B4513; border-radius: 10px; padding: 20px; margin-bottom: 20px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2); } .metrics-section summary { font-family: 'Cinzel', serif; color: #5D2E0C; font-size: 18px; cursor: pointer; outline: none; padding: 10px 0; text-align: center; font-weight: 500; letter-spacing: 1px; } .creator-section { margin: 25px 0; text-align: center; } .creator-badge { display: inline-flex; align-items: center; background: linear-gradient(145deg, #CD853F, #D2691E); border: 3px solid #8B4513; border-radius: 10px; padding: 15px 20px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2); } .creator-label { color: #2F1B14; font-size: 14px; margin-right: 10px; font-weight: 500; } .creator-link { display: flex; align-items: center; gap: 8px; color: #2F1B14; text-decoration: none; transition: all 0.3s ease; } .creator-name { font-weight: 600; } .creator-arrow { font-size: 16px; transition: transform 0.3s ease; } .creator-link:hover .creator-arrow { transform: translateX(5px); } .link-arrow { display: inline-block; transition: transform 0.3s ease; } a:hover .link-arrow { transform: translateX(3px); } .axolotl-container { text-align: center; margin: 35px 0; } .axolotl-container img { max-width: 300px; border-radius: 10px; box-shadow: 0 6px 15px rgba(0, 0, 0, 0.3); border: 3px solid #8B4513; filter: sepia(30%) contrast(110%); } .fossil-texture { position: relative; } .fossil-texture::after { content: ''; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background-image: radial-gradient(circle at 25% 25%, rgba(139, 69, 19, 0.05) 2px, transparent 2px), radial-gradient(circle at 75% 75%, rgba(160, 82, 45, 0.05) 1px, transparent 1px); background-size: 50px 50px, 30px 30px; pointer-events: none; } </style> </head> <body> <div class="container fossil-texture"> <div class="header"> <h1>Austral 24B Winton</h1> </p> </div> <div class="info"> <img src="https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/jxUvuFK1bdOdAPiYIcBW5.jpeg" alt="Model banner"> <div style="text-align: center;"> <div class="creator-section"> <div class="creator-badge"> <span class="creator-label">Trained by</span> <a href="https://huggingface.co/Delta-Vector" target="blank" class="creator-link"> <span class="creator-name">Delta-Vector</span> </a> </div> </div> <div class="model-info"> <h2>Overview</h2> <div class="info-card"> <div class="info-header"> <h3>Austral 24B - Winton</h3> <div class="model-tags"> <span class="model-tag">Codex Finetune</span> <span class="model-tag">Mistral Based</span> <span class ="model-tag">KTO enhanced</span> <span class ="model-tag">Adventure/Roleplay generalist</span> <span class="model-tag">24B Sized model</span> </div> </div> <div class="model-description"> <p style="font-weight: bold; font-style: italic;">More than 1.5-metres tall, about six-metres long and up to 1000-kilograms heavy, Australovenator Wintonensis was a fast and agile hunter. The largest known Australian theropod.</p> <p>This is a finetune of Codex 24B to be a generalist Roleplay/Adventure model. I've removed some of the "slops" that i noticed in an otherwise great model aswell as improving the general writing of the model, This was a multi-stage finetune, all previous checkpoints are released aswell. In testing it has shown to be a great model for Adventure cards & Roleplay, Often pushing the plot forward better then other models, While avoiding some of the slops you'd find in models from Drummer and Co.</p> <p>Support my finetunes / Me on Kofi: https://Ko-fi.com/deltavector | Thank you to Auri for helping/Testing ♥</p> </div> </div> </div> <div class="section"> <h2>Quants</h2> <div class="info-card"> <div class="model-composition"> <h4>Quants Formats</h4> <ul class="composition-list"> <li><span class="model-component"><a href="" target="blank">GGUF</a></span>For use with LLama.cpp & Forks (coming soon!))</li> <li><span class="model-component"><a href="" target="blank">EXL3</a></span>For use with TabbyAPI (Coming soon!)</li> </ul> </div> </div> </div> <div class="section"> <h2>Chat Format</h2> <p>This model utilizes ChatML.</p> <pre><code><|imstart|>user Hi there!<|imend|> <|imstart|>assistant Nice to meet you!<|imend|> <|imstart|>user Can I ask a question?<|imend|> <|imstart|>assistant</code></pre> </div> <div class="section"> <h2>Training</h2> <p>As the the Austral/Francois tradition, I built off another great finetune Codex-24B, I did 4 epochs ontop with roughly the same datamix as Francois-Huali/Austral 70B as a R128 Lora, then KTO alignment with a mix of Instruct/Small writing datasets and then finally another 4 epoch SFT with Repremover (Thanks Pocket!)</p> <div class="metrics-section"> <details> <summary>Config(Post-KTO SFT)</summary> <pre><code>https://wandb.ai/new-eden/austral/runs/i85da0c6?nw=nwuserdeltavector</code></pre> </details> </div> </div> <div class="section"> <h2Fossil Formation Process</h2> <p>This model was trained over 4 epochs using 8 x A100s (Ty to my work, Cognitive Computations) for the base SFT, Then i used KTO to clean up some coherency issues for 1 epoch, then finally training for another 4 epochs on RepRemover to delete slops. Total was roughly 80 hours total.</p> <p style="text-align: center; margin-top: 20px;"> <div class="axolotl-container"> <a href="https://github.com/OpenAccess-AI-Collective/axolotl" target="_blank"> <img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl"> </a> </div> <div class="section"> <h2>Credits</h2> <p>TYSM to my friends: Auri, Lucy, Trappu, Alicat, Kubernetes Bad, Intervitens, NyxKrage & Kalomaze</p> </p> </div> </div> </div> </div> </div> </body> </html>