Modelpublic

KIEFERSA/Sophea-Titan-1-FP8

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes82downloads

Model Card

<div style="display:flex;align-items:center;gap:18px;padding:16px 20px;margin-bottom:16px;border-radius:14px;background:linear-gradient(100deg,#e8f5ec 0%,#f7fbf8 62%);border:1px solid #c6e7d1"><img alt="KIEFERSA" style="flex:0 0 auto;height:58px;width:58px;border-radius:12px;object-fit:contain" src="data:image/webp;base64,UklGRgQNAABXRUJQVlA4WAoAAAAQAAAAxwAAxwAAQUxQSIsKAAABsMf/nyE52q+qZjd2nluEh00OsY2zbdsOH51t27Zt2744F1vrrqrf69X1m+mumq4K/ooISG4bSZKUmIIOeag1w1Ud8wFwPDgH2Om9zYFD2AcXDIa/hdWdgAff+r6oNK7pDDzw1uPxRlQRVoccFlu3e+pj0zrksByD8huqjSFSAm0c2l+8ilrIEQxaT1qC2ljAYYJD89PnxaYQQ05spcdMp5ZEcI0f/CtqGVsygTXY4+vYJGLI4QJg7CeIxkIOFwyGvE0tLeG0Xs8rVLGFHBZb1aORCXAYcniOQdfb61FJjXaEcEb5NeuoWRI+63DhShrgrAmdtZy4iAa4kJPj0PSMOdRCjuCQO3oaDXAhR3Dg+/9Ko0/IERxgl29pDA05QgCM+ohGn7DDYPDr1EJPr2cUiT7hp5aEhvUBZMZ6AY3rC3BjkLVdga0PkFjTGUqCDykOMw9kwEVwKIL9c1zX1d/saIJUsCFvE4sntILyFYjvjgQmeFBwbCsuaAMCyldhhOqVAYERp7bm2s3MKTMoKTF6apugiEOrvaMr+XbADJq8JtXf391IQImt4aEqYpBH3kmtuaXS7E4gkRL1s73pBS8IEY1rLusYCHFi+PoQurwnkjdh+dQ2wHP+i71p/GgM/XaQggITFpzdIpaQoWL7ZlcaGtKD5LPnrBOaeB8tbO3XAzg5LylIsYe/HyY8FwuLNE4/uoSck5SkkO/38jsk2djc05uSK5uSlOvQZzsAcBESdGyLJrckP4hZkFLeGw7AuZ+kDnAXt6fVyIa0ol7u62u0SGfrri2nM5xDQpJ8oqefksbq7+hOk4Eddgmw4e7uwHLMa6TC6NEe9DLaYyfV13sY95JMv5AX4FxgK6sv7ZiFCVwkDFdIifj2EBp93GAvS6a29ivuFbRPx1Jzi135nndWsyKXMdZvxLDCo7kDlNT43Z5AjjYTFDiW6ceVFFEYNJmOqtCthdgLuCXSBLiDaIBzi5sJv5PjKRalf6PU+UNptZ0lKrZpx5WSo8wA6eLeHhBLkZiWsNGWG/P8nEmfnwySd499Og5AsMyDSye3ISuzexyKQnx7KGSfSzvS6FMEHIt+ui2wjEPrffFxUcaGg8g09IHJPqg1DgGe9Q34AeLQTSF0Wr5egOfMJlKBh5S4QQ+f9zNqqfwkS9bzMV3TCU4yHXsq1LDYut9Ti7i6G0Dzc0gHpX9kxcpvXIeqEas7QQmDNv9bhipSgYUE2PaXrUIVadJHwUhv7pWrUUU6oJAvHi0nLzUnK78fhJ7STrdWo4rCSWzNzphvFpSkvhYuGGxxv+ldDyM5DqXHzEAtdZr+HBZLr6ckShk+BAdx0O+mTCX1GSUUslcwFk8oprE9v0EtpU2/lBAAI99DLWW44AJg/EfmzKTr+0qSnb4y0SJMCMFg2FsapXTRvyY48L1/MhPCA4+t9wsKpXTVhyc4iCOmmZAUFlhsVY81oops+glTSZNTZpv1NBzw2LreVYdKard9kSyWFhMWmLgXAmg+KLtuHSpZhP5OJhi0vWC5CUn+w3Ic2l9IAhzaYXfyr1pjdttzBIfWE5bQ/OwAiwldbiMPQyZxGeBO+we1MQdYylYPNpoJGcSZ5Y6ZRs0BDhao7Z5VqKSPCA7soLz3fUc4kIFvIErpG5wD7JYX4LIB/WFw9IdGMkRfKM1ZjPxgNuZj+tOCS9xck12/NtckM6T+HSwh8r9F7z23uHpGDjTPiMwCiPjQ1TdcbzGM9XqOrgWOcSe5o/5CFRURdyNvbZYa3eO0hsxGHJIFZGQ1utxeRz+2uMettJywbCSIjG9WUssIViGpvD2wjKONFQfXAgCQdRB9AICxzIO+ALAhy7ouwNYHSB1vSsKKRSvuzsAEDx8OOgWnbr38jQFG/CC7tvaazaB8BTY+uV0RGkkg43nIsdXe0RkEVKzECOvv28JMCMbGsTU8vCUwIaB8FfkAUn1LJbAc84mbJ/33PxbD2eIoUT5rbmGe10dBH9ErOwITzBM02vYTOjKF+PpQc/sW7AchE5b9p7VLyfj7tKMZ+CFpxU3ua9GRxoVnN3fXHpzx7xsuTOPXuwFpnEyETNA4+/gmwEXoULH9sj8n5yeBhD3745CSWHzHQQvzkSXAcwCJJO7dD3uzWEKFMq24pzUl5yUVifLpjgBChAgd25KJLcnVTUWqNejdkaYgBwZSlVZc0o7U15SkrGwv9TfiK+ltzfVltCKlJW11i57sacRPUge4O7tScwpipLD+/m5mh4OAVBg90oNeTlvSnvrqGyqM+I6WCvWzfelN6Yo0O77mso4muHqNlIhvDqXRxwU2E5ZNbm1CkrdIqfGTsdTcYFcq5p3ZHLjwExXbd7vRIuoK29I9/YQmsfiHijT+diAnR+wKF/LrISIWbyh03Y7LkaMtOkn31V4mJHlC3oow70wa4IpOitXq0/EAnHsBscVTWpF1zS3u5O2hRjwg0rjq4va0YjrGoegX+5jDyv6m+vpKenjOcSgK5WNVkH3u7kZrvnPcf6q6vSWwjEPLbVFwLsNAZAJbyz6oNQ4BnvmNHyAO3RRCp+XrBXgOylZiowo8pFBvddMlc1BL7SdZsorra+sqm/5vuZFQw3IcOly2EnFNV4COl5Om2gBDvni0nLIEVSNWd4ISBpW31hjJNn+j1PlDaRWTDWt21jyTbEgfBeOCQff761FGOqib2EqOm2XyWaF+ECPbPtWISnpF8Y0f+DtqqQr1tRRoqn1Vo5TucC7de/YoPEozYHt8h1qqpP4cKgCj3sVYHOH14AJg3MeopUzbZyQEwM5foJYqk/CkUWwb/qa5zjb9UoID3+dH1FKGa8MFg74vmOfOtu9LcMgd9reZECZYbD0fbUQVpe1fS5QmJ80166gd/lrXu2tNhU3Zh5cqYzQ/b6GRoEC+eJRdvxaV1O76CZlg0OaC5SZaeEyqi9ruEtJQbNUXmVI6Xk3iXiAQDFpNWYramAUWt08nGvcCQDyj2anz6NuANaknVD3QgCryHcGh9OgZtJxaYlkkej2tUEmfEQLEAb/QUOAASxn0GqKUviI4wO7f0EBQLAqHpFEfGfERYQJc3jEXHwATwnb+2lwL3+CCweA3dGwai0tS3Nv/F1OQC+CJ9Xpe0hcYd7gRceTfRnyBxbblIzTAOcRd7Why2hwzgeDBjM531JHo4xSXnxNaTFiEKvIABmXXVNNPR25xm63aX7AccUjmaX/BSmqucZ51r6kbASLjzKMBrhi4lqoyYBkHpUbMOgCMzMg4CtEHADjLPOgLABuyrOu0fkDpmi7rAUi2WfgvYIGHvNnV3do5SH/gfEbjw1saCxGOZ6hntzWZM+xIifqNwdTChEPDD8cA8NgCDnnL/nJXaiFHxfbL/oy0f4YMNzbtqFJiIUdFCv85tTmxgEPeTxdPaEUs5OhI4coL2pJ/WAghdrbu2jLyNSqMw8Zq7+pCLeSYAPdYFbWQIyXK5/rQ6BNypER8fQi1oJLKPhoPYCzkKKnxm92AFKrgkmi/HSiIBZgEm3F0KbGQoyKN805tSizk6EjhkqktSfQJNvkB7pJ2wIyFG2JrbyynoSHkmC9hd3WjFnYweqQnDQ2h59W+1IIPAABWUDggUgIAAHAdAJ0BKsgAyAA+MRiJRCIhiBJcEAGCWlu4MAAZsUGEUQU6Vr/HVACel73mNKbZii7/0o6X9Y+s6FoAm3srqdXIcWd/45kb7mdOviNuuq60W3qXx/nI9sR42ICkrOwkSyYty0cun8cMpqXF0fh8fW3dSwaV0akfevRFNfnTLgy9bQM6IEWlj9O95hHRhKjNbDCS4qWJnQbJto47spFiqntPEoRO945gp7f+FzLmeAJQakEjD+xgQDxMOYrHw79WEkLQkKplqTo/uTX4n54eEP0N4epeSNwNE7Cl7q0Xi32YrSRma3RieLfasZELkHnrVj60WkAA/uouoblbKpSNm/wu8g0tISmLMdPrSIKVi8oMaNvyXkZc6zskreB0cAODf99ubzUwj0N8IJywGgCWpByLV1Kx+G8TMEVJa0HKGlyheflfEABBIEOGipfrRMJEDIiV7tNyTCRkHRlacBBPJNrPF6gu5KqHk0mGh2xBZYDKEWXI1A8R9WyRig0s6CIELwgJcqvHXXixg1yTYYd3Ja2rm0FNGHOb8MdNErR6nH3DYmradAb7JnP/RNqQ3gvhl/pA7ygUm6CIWAO/JF0d93ZViT6KqEgvSVdzZIwTzPR1TQ5YDMPK1u42qqN2SqTRwtzHrtold7EeIrx7plafSjlNtoSSwuC99MOigEHtk+7uPfbMdQA6uO4+wAsQ6bfVGy+dpyM63pbfBdYa2IbgqA8ZA3EUsyDx6GwlVker0XVZhE/HSCyOiMAVqYYDYXpIECKSg6k4BD3OKMJ7nJlAAAAAAA=="/><div><div style="font-size:1.24rem;font-weight:700;color:#003713;line-height:1.2">Sophea-Titan-1-FP8</div><div style="font-size:.86rem;color:#3a5a47;margin-top:2px">FP8_DYNAMIC (W8A8) quantization of Sophea-Titan-1</div></div></div>

FP8_DYNAMIC (W8A8) quantization of [KIEFERSA/Sophea-Titan-1](https://huggingface.co/KIEFERSA/Sophea-Titan-1) — the 27B Greek fine-tuned chat model built on Qwen3.6-27B. Same behaviour as the bf16 release, ≈0.55× the disk / VRAM, and verified lossless (numbers below).

—Base model: KIEFERSA/Sophea-Titan-1 (bf16)
—Scheme: FP8_DYNAMIC — per-channel FP8 (E4M3) weights + dynamic per-token FP8 activations (W8A8), compressed-tensors float-quantized
—Size: 29.4 GB (bf16 base ≈ 54 GB → 0.55×)
—Creator: Kiefer SA
—Decoding: non-thinking (enable_thinking=false) — see recommended sampling under Usage

What is / isn't quantized. Data-free one-shot PTQ via llm-compressor (FP8_DYNAMIC, no calibration). Quantized: the language-model Linear layers (attention q/k/v/o, MLP, GDN in/out projections). Kept bf16: lm_head. GDN conv1d/A_log/dt_bias/norms are non-Linear and untouched. This quantized build is text-only — the vision tower of the multimodal base (Sophea-Titan-1) is not included in this FP8 release.

Serve non-thinking. Set enable_thinking=false in the chat template. With thinking left on, Greek output quality degrades.

Verified lossless (this FP8 build)

Measured on this checkpoint through the vLLM decode path (greedy, non-thinking), vs the bf16 base:

Every axis is within measurement noise of bf16 → lossless. FP8 also loads in HF transformers.

Per-benchmark detail (measured on this FP8 build)

General Greek benchmarks (9, accuracy)

English retention — 5 benchmarks (accuracy)

greekmmlu — per-subject (accuracy)

<div style="overflow-x:auto"> <table style="width:100%;border-collapse:collapse;font-size:13px;font-variant-numeric:tabular-nums;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif"> <thead><tr> <th style="padding:10px 8px;text-align:left;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;">subject</th> <th style="padding:10px 8px;text-align:center;font-weight:700;border-bottom:2px solid #0a8043;color:#00682f;font-size:14px;background:rgba(10,128,67,.10);">FP8</th> <th style="padding:10px 8px;text-align:center;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;">subject</th> <th style="padding:10px 8px;text-align:center;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;">FP8</th> </tr></thead><tbody> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Accounting</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.859</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Greek Traditions</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.886</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Agriculture</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.875</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Law</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.700</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Art</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.753</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Management</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.838</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Biology</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.868</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Maritime Safety & Rescue</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.669</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Chemistry</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.802</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Mathematics</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.917</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Civil Engineering</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.822</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Medicine</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.901</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Clinical Knowledge</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.820</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Modern Greek Language</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.920</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Computer Networks & Security</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.762</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Physics</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.849</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Computer Science</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.880</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Prehistory</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">1.000</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Driving Rules</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.821</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">World History</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.950</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Economics</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.922</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">World Religions</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.774</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Education</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.867</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Geography</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.964</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Electrical Engineering</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.817</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Government and Politics</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.932</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">General Knowledge</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.789</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Greek History</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.874</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Greek Literature</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);background:rgba(10,128,67,.10);">0.714</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">Greek Mythology</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">0.845</td> </tr> <tr> <td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">OVERALL</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#00682f;background:rgba(10,128,67,.10);">0.853</td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);"></td> <td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);"></td> </tr> </tbody></table></div>

Usage

Serve with vLLM (compressed-tensors FP8 auto-detected; Cutlass FP8 kernels on Blackwell/Hopper):

bash

vllm serve KIEFERSA/Sophea-Titan-1-FP8 --served-model-name sophea-titan-1-fp8 --trust-remote-code \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

Serve with vision enabled — this is a multimodal checkpoint (Qwen3_5ForConditionalGeneration). The vision tower is excluded from quantization and kept in bf16, so image input works unchanged:

bash

vllm serve KIEFERSA/Sophea-Titan-1-FP8 --served-model-name sophea-titan-1-fp8 --trust-remote-code \
  --limit-mm-per-prompt '{"image": 4}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

For text-only serving, add --language-model-only to skip loading the vision tower entirely.

python

resp = client.chat.completions.create(
    model="sophea-titan-1-fp8",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
        {"type": "text", "text": "Περίγραψε την εικόνα στα ελληνικά."},
    ]}],
    temperature=0,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},   # required: non-thinking
)
print(resp.choices[0].message.content)

Recommended sampling (instruct / non-thinking): temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0.

python

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="sophea-titan-1-fp8",
    messages=[{"role": "user", "content": "Ποια είναι η πρωτεύουσα της Ελλάδας;"}],
    temperature=0,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},   # required: non-thinking
)
print(resp.choices[0].message.content)

Speculative decoding (MTP)

This build ships the model's multi-token-prediction head — 15 mtp.* tensors (~0.85 GB, bf16) in model-mtp.safetensors. This is the single-layer draft stack that config.json has always declared through mtp_num_hidden_layers: 1.

Earlier revisions of this repo did not contain it. The LoRA merge loaded the base through AutoModelForCausalLM, and transformers declares _keys_to_ignore_on_load_unexpected = [r"^mtp.*"] for this architecture, so the head was discarded at load and never written back — and llm-compressor dropped it again during quantization. The config therefore advertised a module the weights did not contain. To pin the previous bytes, use revision="11380ae677053cff0deab6cb8ae17d192611924a".

The MTP Linears are held in bf16 and listed in quantization_config.ignore, so the compressed-tensors loader treats that block as unquantized rather than expecting packed weights.

Provenance. These are the base Qwen3.6-27B MTP weights. LoRA never targeted mtp.*, so no fine-tuned draft head exists. This cannot affect output quality: speculative decoding verifies every drafted token against the main model, so a stale drafter changes throughput only, never the output distribution.

Enable it with vLLM (≥ 0.23.0):

bash

vllm serve KIEFERSA/Sophea-Titan-1-FP8 --served-model-name sophea-titan-1-fp8 --trust-remote-code \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":1}'

The head is loaded but stays inactive unless --speculative-config is passed, so existing serve commands are unaffected.

License

Inherits the Qwen3.6-27B base-model license. Verify base-model terms before use.