ArtusDev/AllThingsIntel_Apollo-V0.1-4B-Thinking-EXL3

<style> .container-dark { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; line-height: 1.6; color: #d4d4d4; } a { color: #569cd6; text-decoration: none; font-weight: 600; } a:hover { text-decoration: underline; } .card-dark { background-color: #252526; border-radius: 12px; padding: 24px; margin-bottom: 20px; box-shadow: 0 4px 12px rgba(0,0,0,0.3); border: 1px solid #3c3c3c; } .card-dark h1 { font-size: 2.2em; color: #ffffff; text-align: center; margin-bottom: 10px; } .card-dark.card-dark-title h1 { font-size: 1.5em; } .card-dark .subtitle { text-align: center; font-size: 1.1em; color: #a0a0a0; } .card-dark h2 { font-size: 1.5em; margin-top: 0; padding-bottom: 10px; border-bottom: 1px solid #3c3c3c; color: #c586c0; } .card-dark h3 { font-size: 1.2em; color: #d4d4d4; } .styled-table { display: table; border: none; width: 100%; font-size: 0.95em; margin-bottom: 0px; } .styled-table thead th { background-color: #333333; color: #c586c0; text-align: left; } .styled-table th { padding: 12px 15px; } .styled-table td { padding: 0; } .styled-table table, .styled-table th, .styled-table td { border-left: none; border-right: none; border-bottom: none; } .styled-table td { border-bottom: 1px solid #3c3c3c; } .styled-table tbody tr { transition: background-color 0.1s ease; } .styled-table tbody tr:hover { background-color: #3a3a3a; } .styled-table tr:last-child td { border-bottom: none; } .styled-table td a { display: block; padding: 12px 15px; } .styled-table td a.fake-link { text-decoration:none; color:inherit; } details { margin-top: 20px; border: 1px solid #3c3c3c; border-radius: 8px; overflow: hidden; } summary { cursor: pointer; padding: 12px 18px; background-color: #6A5ACD; font-weight: 600; display: flex; align-items: center; gap: 10px; justify-content: space-between; list-style: none; } summary::-webkit-details-marker { display: none; } summary:hover { filter: brightness(1.1); } summary::after { content: ''; display: inline-block; width: 8px; height: 8px; border-bottom: 2px solid white; border-right: 2px solid white; transform: rotate(45deg); transition: transform 0.3s ease; } details[open] > summary::after { transform: rotate(225deg); } .details-content { padding: 18px; } .btn-purple { display: inline-block; background-color: #6A5ACD; color: white !important; padding: 12px 24px; border-radius: 8px; text-decoration: none; font-weight: 600; transition: background-color 0.3s ease, transform 0.2s ease; text-align: center; } .btn-purple:hover { background-color: #7B68EE; transform: translateY(-2px); } </style>

<div class="container-dark">

<div class="card-dark card-dark-title"> <h1>ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3</h1> <p class="subtitle"> EXL3 quants of <a href="https://huggingface.co/AllThingsIntel/Apollo-V0.1-4B-Thinking" target="blank">AllThingsIntel/Apollo-V0.1-4B-Thinking</a> using <a href="https://github.com/turboderp-org/exllamav3/" target="_blank">exllamav3</a> for quantization. </p> </div>

<div class="card-dark"> <h2>Quants</h2> <table class="styled-table"> <thead> <tr> <th>Quant</th> <th>BPW</th> <th>Head Bits</th> <th>Size (GB)</th> </tr> </thead> <tbody> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/2.5bpwH6" target="blank">2.5H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/2.5bpwH6" target="blank">2.5</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/2.5bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/2.5bpwH6" target="blank">2.23</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.0bpwH6" target="blank">3.0H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.0bpwH6" target="blank">3.0</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.0bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.0bpwH6" target="blank">2.46</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.5bpwH6" target="blank">3.5H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.5bpwH6" target="blank">3.5</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.5bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/3.5bpwH6" target="blank">2.68</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.0bpwH6" target="blank">4.0H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.0bpwH6" target="blank">4.0</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.0bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.0bpwH6" target="blank">2.91</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.5bpwH6" target="blank">4.5H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.5bpwH6" target="blank">4.5</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.5bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/4.5bpwH6" target="blank">3.14</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/5.0bpwH6" target="blank">5.0H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/5.0bpwH6" target="blank">5.0</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/5.0bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/5.0bpwH6" target="blank">3.36</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/6.0bpwH6" target="blank">6.0H6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/6.0bpwH6" target="blank">6.0</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/6.0bpwH6" target="blank">6</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/6.0bpwH6" target="blank">3.82</a></td> </tr> <tr> <td><a href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/8.0bpwH8" target="blank">8.0H8</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/8.0bpwH8" target="blank">8.0</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/8.0bpwH8" target="blank">8</a></td> <td><a class="fake-link" href="https://huggingface.co/ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3/tree/8.0bpwH8" target="blank">4.82</a></td> </tr> </tbody> </table> </div>

<div class="card-dark"> <h2>How to Download and Use Quants</h2> <p>You can download quants by targeting specific size using the Hugging Face CLI.</p> <details> <summary>Click for download commands</summary> <div class="details-content"> <b>1. Install huggingface-cli:</b> <pre><code>pip install -U "huggingfacehub[cli]"</code></pre> <b>2. Download a specific quant:</b> <pre><code>huggingface-cli download ArtusDev/AllThingsIntelApollo-V0.1-4B-Thinking-EXL3 --revision "5.0bpwH6" --local-dir ./</code></pre> </div> </details> <p>EXL3 quants can be run with any inference client that supports EXL3, such as <a href="https://github.com/theroyallab/tabbyapi" target="blank"><b>TabbyAPI</b></a>. Refer to <a href="https://github.com/theroyallab/tabbyAPI/wiki/01.-Getting-Started" target="_blank">documentation</a> for set up instructions.</p> </div>

<div class="card-dark"> <h2>Quant Requests</h2> <div style="text-align: center; margin-top: 25px;"> <a href="https://huggingface.co/ArtusDev/requests-exl/discussions/new?title=[MODELNAMEHERE]&description=[MODELHFLINKHERE]" class="btn-purple" target="blank">Request EXL3 Quants</a> </div> <p class="subtitle"> See <a href="https://huggingface.co/ArtusDev/requests-exl" target="_blank">EXL community hub</a> for request guidelines. </p> </div>

<div class="card-dark"> <h2>Acknowledgements</h2> <p style="text-align: center; font-size: 1.1em; margin-top: 10px;"> Made possible with cloud compute from <a href="https://lium.io/" target="_blank"><b>lium.io</b></a> </p> </div>

</div>