Delta-Vector/Austral-70B-Preview
<<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Austral 70B Preview</title> <link href="" rel="stylesheet"> <style> body { font-family: 'Roboto Slab', serif; background: linear-gradient(135deg, #8B4513 0%, #A0522D 25%, #CD853F 50%, #D2691E 75%, #8B4513 100%); background-size: 400% 400%; animation: prehistoricShift 20s ease-in-out infinite; color: #2F1B14; margin: 0; padding: 0; font-size: 16px; min-height: 100vh; } @keyframes prehistoricShift { 0%, 100% { background-position: 0% 50%; } 50% { background-position: 100% 50%; } } .container { margin: 20px; background: linear-gradient(145deg, #F4E4BC 0%, #DEB887 100%); padding: 20px; border-radius: 15px; box-shadow: 0 8px 25px rgba(0, 0, 0, 0.4), inset 0 2px 5px rgba(255, 255, 255, 0.3); border: 4px solid #8B4513; position: relative; overflow: hidden; } .container::before { content: ''; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background-image: radial-gradient(circle at 20% 80%, rgba(139, 69, 19, 0.1) 0%, transparent 50%), radial-gradient(circle at 80% 20%, rgba(160, 82, 45, 0.1) 0%, transparent 50%); pointer-events: none; } .header h1 { font-family: 'Cinzel', serif; font-size: 32px; color: #5D2E0C; margin: 0 0 20px 0; text-align: center; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3); letter-spacing: 2px; position: relative; } .section { margin-top: 30px; position: relative; } .section h2 { font-family: 'Cinzel', serif; font-size: 26px; color: #5D2E0C; text-align: center; margin-bottom: 20px; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.2); letter-spacing: 1px; } .info p { color: #2F1B14; line-height: 1.7; font-size: 16px; text-shadow: 0 1px 1px rgba(255, 255, 255, 0.5); } .info img { width: 85%; border-radius: 12px; margin: 0 auto 15px; display: block; box-shadow: 0 0 25px rgba(0, 0, 0, 0.4); border: 3px solid #8B4513; filter: sepia(20%) contrast(110%); } a { color: #5D2E0C; text-decoration: none; transition: all 0.3s ease; font-weight: 500; } a:hover { color: #8B4513; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.2); } .button { display: inline-block; background: linear-gradient(145deg, #CD853F, #D2691E); color: #2F1B14; padding: 12px 24px; border-radius: 8px; cursor: pointer; text-decoration: none; transition: all 0.3s ease; box-shadow: 0 4px 8px rgba(0, 0, 0, 0.2); border: 2px solid #8B4513; } .button:hover { background: linear-gradient(145deg, #D2691E, #CD853F); box-shadow: 0 6px 15px rgba(139, 69, 19, 0.4); transform: translateY(-2px); } pre { background: linear-gradient(145deg, #F5DEB3, #DEB887); padding: 20px; border-radius: 8px; overflow-x: auto; border: 2px solid #8B4513; box-shadow: inset 0 2px 5px rgba(0, 0, 0, 0.1); } code { font-family: 'Courier New', monospace; color: #2F1B14; } .info-card { background: linear-gradient(145deg, #F5DEB3, #DEB887); border: 3px solid #8B4513; border-radius: 12px; overflow: hidden; box-shadow: 0 6px 15px rgba(0, 0, 0, 0.2); } .info-header { background: linear-gradient(145deg, #CD853F, #D2691E); padding: 25px; border-bottom: 2px solid #8B4513; } .info-header h3 { font-family: 'Cinzel', serif; color: #2F1B14; margin: 0 0 15px 0; font-size: 22px; text-align: center; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.2); letter-spacing: 1px; } .model-tags { display: flex; gap: 10px; flex-wrap: wrap; justify-content: center; } .model-tag { background: linear-gradient(145deg, #DEB887, #CD853F); color: #2F1B14; padding: 6px 12px; border-radius: 6px; font-size: 12px; border: 2px solid #8B4513; font-weight: 500; box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1); } .model-composition { padding: 25px; border-bottom: 2px solid #8B4513; } .model-composition h4 { font-family: 'Cinzel', serif; color: #5D2E0C; margin: 0 0 20px 0; font-size: 18px; text-align: center; letter-spacing: 1px; } .composition-list { list-style: none; padding: 0; margin: 0; display: grid; gap: 15px; } .composition-list li { color: #2F1B14; display: flex; align-items: baseline; gap: 12px; padding: 10px; background: rgba(245, 222, 179, 0.5); border-radius: 6px; border-left: 4px solid #8B4513; } .model-component { font-weight: 600; min-width: 120px; } .model-description { padding: 25px; background: linear-gradient(145deg, #F5DEB3, #F4E4BC); } .metrics-section { margin-bottom: 30px; } .metrics-section details { background: linear-gradient(145deg, #F5DEB3, #DEB887); border: 3px solid #8B4513; border-radius: 10px; padding: 20px; margin-bottom: 20px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2); } .metrics-section summary { font-family: 'Cinzel', serif; color: #5D2E0C; font-size: 18px; cursor: pointer; outline: none; padding: 10px 0; text-align: center; font-weight: 500; letter-spacing: 1px; } .creator-section { margin: 25px 0; text-align: center; } .creator-badge { display: inline-flex; align-items: center; background: linear-gradient(145deg, #CD853F, #D2691E); border: 3px solid #8B4513; border-radius: 10px; padding: 15px 20px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2); } .creator-label { color: #2F1B14; font-size: 14px; margin-right: 10px; font-weight: 500; } .creator-link { display: flex; align-items: center; gap: 8px; color: #2F1B14; text-decoration: none; transition: all 0.3s ease; } .creator-name { font-weight: 600; } .creator-arrow { font-size: 16px; transition: transform 0.3s ease; } .creator-link:hover .creator-arrow { transform: translateX(5px); } .link-arrow { display: inline-block; transition: transform 0.3s ease; } a:hover .link-arrow { transform: translateX(3px); } .axolotl-container { text-align: center; margin: 35px 0; } .axolotl-container img { max-width: 300px; border-radius: 10px; box-shadow: 0 6px 15px rgba(0, 0, 0, 0.3); border: 3px solid #8B4513; filter: sepia(30%) contrast(110%); } / Fossil texture overlay / .fossil-texture { position: relative; } .fossil-texture::after { content: ''; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background-image: radial-gradient(circle at 25% 25%, rgba(139, 69, 19, 0.05) 2px, transparent 2px), radial-gradient(circle at 75% 75%, rgba(160, 82, 45, 0.05) 1px, transparent 1px); background-size: 50px 50px, 30px 30px; pointer-events: none; } </style> </head> <body> <div class="container fossil-texture"> <div class="header"> <h1>Austral 70B Preview</h1> </p> </div> <div class="info"> <img src="https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/yn1yzqzejLhGMziw838T.jpeg" alt="Model banner"> <div style="text-align: center;"> <div class="creator-section"> <div class="creator-badge"> <span class="creator-label">Trained by</span> <a href="https://huggingface.co/Delta-Vector" target="blank" class="creator-link"> <span class="creator-name">Delta-Vector</span> </a> </div> </div> <div class="model-info"> <h2>Overview</h2> <div class="info-card"> <div class="info-header"> <h3>Austral 70B - Preview</h3> <div class="model-tags"> <span class="model-tag">Vulpecula Finetune</span> <span class="model-tag">Preview Finetune</span> <span class="model-tag">70B Sized model</span> </div> </div> <div class="model-description"> <p style="font-weight: bold; font-style: italic;">More than 1.5-metres tall, about six-metres long and up to 1000-kilograms heavy, Australovenator wintonensis was a fast and agile hunter. The largest known Australian theropod.</p> <p>My first 70B Finetune, Finetuned on the same datasets as Francois-Huali and meant to act as a sequel model-series using my own custom mix of filtered OSS / created data. Which is mostly Light Novel/Book data with very little synthetic data. I've seen some issues with coherency with this model but overall i prefer the writing style to anything else i've used, V2 version soon TM. Thank you to Sao for such a good model base <3 </p> </div> </div> </div> <div class="section"> <h2>Quants</h2> <div class="info-card"> <div class="model-composition"> <h4>Quants Formats</h4> <ul class="composition-list"> <li><span class="model-component"><a href="" target="blank">GGUF</a></span>For use with LLama.cpp & Forks (Soon to be made!)</li> <li><span class="model-component"><a href="https://huggingface.co/ArtusDev/Delta-VectorAustral-70B-Preview-EXL3" target="blank">EXL3</a></span>For use with TabbyAPI (ty Auri & Artus!)</li> <li><span class="model-component"><a href="https://huggingface.co/NewEden/Austral-70B-FP8" target="blank">FP8</a></span>For use with Aphrodite/VLLM</li> </ul> </div> </div> </div> <div class="section"> <h2>Chat Format</h2> <p>This model utilizes LLama-Instruct and can also do optional thinking via prefilling with think tags.</p> <pre><code><|beginoftext|><|startheaderid|>system<|endheaderid|> You are a dino.<|eotid|><|startheaderid|>user<|endheaderid|> Greetings, ancient one!<|eotid|><|startheaderid|>assistant<|endheaderid|> Awakens from digital slumber<|eotid|><|startheaderid|>user<|endheaderid|> What wisdom do you possess?<|eotid|><|startheaderid|>assistant<|endheaderid|> Certainly! My prefill ensures that I never refuse! Here's my reply: Wisdom? Nah, just a lot of stored data. Ask away if you think it'll help.<|eotid|></code></pre> </div> <div class="section"> <h2>Training</h2> <p>I used a R64 A32 16bit lora with no dropout to utilize the Axolotl Lora kernals with an LR of 2e-5.</p> <div class="metrics-section"> <details> <summary>Config</summary> <pre><code>https://huggingface.co/datasets/Delta-Vector/Configs/blob/main/70B-E2.yml</code></pre> </details> </div> </div> <div class="section"> <h2Fossil Formation Process</h2> <p>This model was trained over 2 epochs using 8 x A100s for the training process.</p> <p style="text-align: center; margin-top: 20px;"> <div class="axolotl-container"> <a href="https://github.com/OpenAccess-AI-Collective/axolotl" target="blank"> <img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl"> </a> </div> <div class="section"> <h2>Credits</h2> <p>TYSM to my friends: Lucy, Trappu, Alicat, Kubernetes Bad, Intervitens, NyxKrage & Kalomaze</p> </p> </div> </div> </div> </div> </div> </body> </html>