Jaidchen/Focus
078
1---2library_name: transformers3license: apache-2.04license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE5base_model: Qwen/Qwen3.6-27B6pipeline_tag: text-generation7tags:8- dense9- coding10- agentic11- unimodal12- repackaged13---14 15<center>16<img style='height: 16em' src='https://cdn-uploads.huggingface.co/production/uploads/63e9a1c4ccae1fe5c622b9f6/ADYJx3gsTmTVPuhHGMM5S.webp'/>17</center>18 19# Focus20 21repackaged [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplified architecture and minor opinionated improvements22 23- all vision-related components removed24- reduced storage and memory, faster inference25- zero loss of output quality26 27## comparison28 29<table>30<tr>31 <th></th>32 <th>Qwen 3.6 27B</th>33 <th>Focus</th>34</tr>35<tr>36 <td>author</td>37 <td><a href='https://huggingface.co/Qwen'>Alibaba Qwen</a></td>38 <td><a href='https://huggingface.co/Jaidchen'>Jaid</a></td>39</tr>40<tr>41 <td>repository</td>42 <td><a href='https://huggingface.co/Qwen/Qwen3.6-27B'>Qwen/Qwen3.6-27B</a></td>43 <td><a href='https://huggingface.co/Jaidchen/Focus'>Jaidchen/Focus</a></td>44</tr>45<tr>46 <td>architecture</td>47 <td><code>qwen3_5</code></td>48 <td><code>qwen3_5_text</code></td>49</tr>50<tr>51 <td>Transformers handler</td>52 <td>53 <a href='https://huggingface.co/docs/transformers/model_doc/qwen3_5#transformers.Qwen3_5ForConditionalGeneration'><code>Qwen3_5ForConditionalGeneration</code></a>54 </td>55 <td>56 <a href='https://huggingface.co/docs/transformers/model_doc/qwen3_5#transformers.Qwen3_5ForCausalLM'><code>Qwen3_5ForCausalLM</code></a>57 </td>58</tr>59<tr>60 <td>tensor entries</td>61 <td>1199</td>62 <td>866</td>63</tr>64<tr style='opacity: 50%'>65 <td>tensor type</td>66 <td>bf16</td>67 <td>bf16</td>68</tr>69<tr>70 <td>parameters</td>71 <td>27 781 427 952</td>72 <td>27 320 697 856</td>73</tr>74<tr style='opacity: 50%'>75 <td>active</td>76 <td>100%</td>77 <td>100%</td>78</tr>79<tr style='opacity: 50%'>80 <td>vocabulary size</td>81 <td>248 320</td>82 <td>248 320</td>83</tr>84<tr style='opacity: 50%'>85 <td>context size</td>86 <td>262 144</td>87 <td>262 144</td>88</tr>89<tr style='opacity: 50%'>90 <td>MTP</td>91 <td>integrated</td>92 <td>integrated</td>93</tr>94<tr>95 <td>sampling strategy</td>96 <td>random sampling</td>97 <td>greedy/deterministic</td>98</tr>99<tr>100 <td>sampling parameters</td>101 <td>102 <div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>103 <span style='color: hsl(from currentColor 0 100% l)'>do_sample</span>: <span style='color: hsl(from currentColor 50 80% l)'>true</span><br>104 <span style='color: hsl(from currentColor 0 100% l)'>temperature</span>: <span style='color: hsl(from currentColor 50 80% l)'>0.6</span><br>105 <span style='color: hsl(from currentColor 0 100% l)'>top_k</span>: <span style='color: hsl(from currentColor 50 80% l)'>20</span><br>106 <span style='color: hsl(from currentColor 0 100% l)'>top_p</span>: <span style='color: hsl(from currentColor 50 80% l)'>0.95</span></div>107 </td>108 <td>109 <div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>110 <span style='color: hsl(from currentColor 0 100% l)'>do_sample</span>: <span style='color: hsl(from currentColor 50 80% l)'>false</span><br>111 <span style='color: hsl(from currentColor 0 100% l)'>temperature</span>: <span style='color: hsl(from currentColor 50 80% l)'>0</span><br>112 <span style='color: hsl(from currentColor 0 100% l)'>top_k</span>: <span style='color: hsl(from currentColor 50 80% l)'>1</span><br>113 <span style='color: hsl(from currentColor 0 100% l)'>top_p</span>: <span style='color: hsl(from currentColor 50 80% l)'>1</span></div>114 </div>115 </td>116</tr>117<tr>118 <td>input modality</td>119 <td>text, image, video</td>120 <td>text</td>121</tr>122<tr>123 <td>repository size</td>124 <td>55 586 107 940</td>125 <td>54 659 211 447</td>126</tr>127<tr>128 <td>model size</td>129 <td>55 562 855 904</td>130 <td>54 641 395 712</td>131</tr>132<tr>133 <td>splits</td>134 <td>15</td>135 <td>none</td>136</tr>137<tr>138 <td>Jinja template</td>139 <td><a href='https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/chat_template.jinja'>Qwen original</a></td>140 <td><a href='https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/chat_template.jinja'>Qwen original</a><br>+ <a href='https://huggingface.co/unsloth/Qwen3.5-27B/blob/main/chat_template.jinja'>Unsloth tweaks</a><br>+ <a href='https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/main/chat_template.jinja'>Froggeric tweaks</a><br>+ unimodality patch<br>+ further custom tweaks</td>141</tr>142</table>143 144## pros145 146- reduced storage needs147- reduced loading time148- reduced VRAM occupancy, thus more room for context149- increased inference speed150- simplified architecture, unlocking some further potential for optimizing low-level procedures151 152## cons153 154- legally blind155 - Pictures and video frames can still be present in the context without crashing, but their contents are no longer interpreted by the model and won’t do anything else than waste space.156 - If you occasionally rely on those capabilities, I suggest adding a `consult` tool to your harness that calls a vision-enabled subagent model like [Gemini Flash](https://openrouter.ai/~google/gemini-flash-latest) or [GPT](https://openrouter.ai/~openai/gpt-latest).157- reduced compatibility158 - The simplified architecture is handled by the `Qwen3_5ForCausalLM` class which may not be included in your inference engine. In this case you would need to ask your agent or integrate it yourself.159 - Inference engines confirmed to have native support:160 - [SGLang](https://github.com/sgl-project/sglang) ≥ v0.5.17 (PR [#32401](https://github.com/sgl-project/sglang/pull/32401))161 - [vLLM](https://github.com/vllm-project/vllm) ≥ v0.26.0 (PR [#50210](https://github.com/vllm-project/vllm/pull/50210))162 - The applied coercions may confuse your inference engine in case it has fixed expectations about the model’s architecture and thus lead to unpredictable behavior.163 164## caveats165 166- model file not split, possibly causing issues if intended to be stored on an HDD from the previous century167- random sampling disabled by default, less suitable for long-form writing, entertainment and casual chat168 169## Jinja template changes170 171`chat.jinja` is reproducibly built from an untouched upstream template plus the ordered patch stack in `jinja_build/`. `jinja_build/build.ps1` applies the patches lexicographically and overwrites the final `chat.jinja`.172 173- base: [Qwen/Qwen3.6-27B original](https://huggingface.co/Qwen/Qwen3.6-27B/blob/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/chat_template.jinja)174 - commit `6a9e13bd6fc8f0983b9b99948120bc37f49c13e9`175 - SHA-256 `E84F32A23FDDA27689F868AA4A1A5621F41133E51A48D7F3EFCBEA2839574259`176- adopted tweaks from [Unsloth Qwen3.6](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja)177 - Added `developer` role as alias for `system`. ([lines 45–57](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L45-L57))178 - Merged leading system/developer messages into a single policy message. ([lines 45–57](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L45-L57), [lines 67–75](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L67-L75))179 - Allowed histories without a normal human query. ([lines 76–86](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L76-L86))180 - Made mapping-argument rendering portable by avoiding `|items`. ([lines 122–130](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L122-L130))181- adopted tweaks from [Froggeric version](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja)182 - Added `preserve_thinking` option to retain historical reasoning. ([line 8](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L8), [lines 225–230](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L225-L230))183 - Retained mid-conversation system/developer messages as Qwen system turns. ([lines 165–182](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L165-L182))184 - Allowed string-valued `message.thinking` as fallback for historical reasoning. ([lines 183–197](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L183-L197))185 - Applied boundary-aware `</think>` parsing. ([lines 198–223](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L198-L223))186 - Avoided synthesizing empty historical thinking blocks. ([lines 225–230](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L225-L230))187 - Supported both wrapped and direct tool calls, but never null-wrapped. ([lines 231–237](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L231-L237))188 - Preserved non-empty string tool arguments. ([lines 261–280](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L261-L280))189 - Used direct message indexing for tool-response grouping instead of `loop.previtem`/`loop.nextitem`. ([line 165](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L165), [lines 286–318](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L286-L318))190- custom tweaks191 - Merged any number of leading system/developer messages, generalizing Unsloth's two-message merge. ([lines 45–57](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L45-L57))192 - Applied boundary-aware `</think>` parsing but excluded Froggeric's malformed-tag recovery. ([lines 198–223](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L198-L223))193 - Raised an error containing the offending role instead of Qwen's generic unknown-role error. ([lines 143–144](https://huggingface.co/Qwen/Qwen3.6-27B/blob/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/chat_template.jinja#L143-L144))194 - Used direct message indexing for tool-response grouping without Froggeric's error-escalation state. ([lines 286–318](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/b225c1d63f84c65af7dd6af2ba384eb037aa885e/chat_template.jinja#L286-L318))195 - Guarded undefined `tools`/`tool_calls` and handled non-mapping content items defensively.196 - Adopted Qwen's proposed `continue_final_message` fix for partial assistant prefills. ([lines 130–132](https://huggingface.co/Qwen/Qwen3.6-27B/blob/0774bb84a25294dac0538c2bfc7abc08ae89d522/chat_template.jinja#L130-L132))197- unimodality patch198 - Removed Qwen's vision-token machinery and rendered image/video content as `[image]` and `[video]`. ([lines 1–41](https://huggingface.co/Qwen/Qwen3.6-27B/blob/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/chat_template.jinja#L1-L41))199 200## license201 202Apache 2.0 – inherited from [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE)