FenomAI/North-Mini-Code-1.0-AWQ-INT4
025
1---2base_model: CohereLabs/North-Mini-Code-1.03inference: false4library_name: transformers5license: apache-2.06tags:7- conversational8- chat9- code10- agent11---12 13<div align="center">14 <img src="https://huggingface.co/buckets/cyankiwi/activation-aware-2.0/resolve/banner/cyankiwi-banner-awq-0.png">15</div>16 17<div align="left">18 <table align="center" style="border-collapse:collapse; border:none;">19 <tr style="border:none;">20 <td align="right" style="border:none; padding:4px 12px 4px 0;"><b>Version</b></td>21 <td align="left" style="border:none; padding:4px 0;">26.05.01</td>22 </tr>23 <tr style="border:none;">24 <td align="right" style="border:none; padding:4px 12px 4px 0;"><b>Calibration</b></td>25 <td align="left" style="border:none; padding:4px 0;">26 <a href="https://huggingface.co/datasets/cyankiwi/calibration" target="_blank">STEM and Agentic</a>27 </td>28 </tr>29 <tr style="border:none;">30 <td align="right" style="border:none; padding:4px 12px 4px 0;"><b>Languages</b></td>31 <td align="left" style="border:none; padding:4px 0;">32 <code>EN</code> <code>ZH</code> <code>HI</code> <code>AR</code> <code>RU</code>33 <code>JA</code> <code>KO</code> <code>NL</code> <code>FR</code> <code>ES</code>34 </td>35 </tr>36 <tr style="border:none;">37 <td align="right" style="border:none; padding:4px 12px 4px 0;"><b>Model Size</b></td>38 <td align="left" style="border:none; padding:4px 0;">17.20 GB</td>39 </tr>40 <tr style="border:none;">41 <td align="right" style="border:none; padding:4px 12px 4px 0;"><b>Contact</b></td>42 <td align="left" style="border:none; padding:4px 0;">43 <a href="mailto:ton@cyan.kiwi">Email</a>44 </td>45 </tr>46 </table>47</div>48 49---50 51# **Model Card for North Mini Code**52 53## **Model Summary**54 55North Mini Code is an open weights research release of a 30B-A3B parameter model optimized for code generation, agentic software engineering, and terminal tasks.56 57Developed by: [Cohere](https://cohere.com/) and [Cohere Labs](https://cohere.com/research)58 59* Point of Contact: [**Cohere Labs**](https://cohere.com/research) 60* License: Apache 2.061* Model: North Mini Code 62* Model Size: 30B total; 3B active 63* Context length: 256K & 64K max output64 65For more details about this model, please check out our [blog post](https://huggingface.co/blog/CohereLabs/introducing-north-mini-code). 66 67**Try North Mini Code**68 69You can try out North Mini Code before downloading the weights in OpenCode and our hosted [Hugging Face Space](https://huggingface.co/spaces/CohereLabs/North-Mini-Code-1.0).70 71**Evaluation**72 7374 75<details>76<summary><span style="font-size: 80%;"><b>Benchmarking Methodology [CLICK TO EXPAND]</span></b></summary>77 78- <span style="font-size: 80%;">We used SWE-Bench Verified, SWE-Bench Pro, Terminal-Bench v2, and Terminal-Bench Hard to benchmark North Mini Code's agentic coding capabilities. For evaluation harnesses, we used the Swe-Agent harness v1.1.0 for SWE-Bench, and a simple ReAct harness employing a single terminal-use tool based on Harbor's Tmux session implementation for Terminal-Bench v2. For Terminal Bench Hard, we directly used Terminus-2, following the same methodology as the Artificial Analysis Intelligence Index to compare North-Mini-Code-1.0 with the other models. Additionally, we used SciCode and LiveCodeBench v6 as complex code-generation benchmarks outside of tool use.</span>79- <span style="font-size: 80%;">We run each benchmark with 3 different seeds and report the average benchmark performance, using temperature=1.0 and top\_p=0.95. We used publicly reported scores for competitor models, either from original reports or the Artificial Analysis Intelligence Index, where available. Additionally, Gemma4’s scores for agentic coding tasks were reported by [Qwen team](https://qwen.ai/blog?id=qwen3.6-35b-a3b). For benchmark results that any public report is missing, denoted by (\*) in the figure, we run them internally using the recommended model configuration.</span>80</details>81 82**Usage**83 84Please install transformers from the source repository that includes the necessary changes for this model. We recommend using the following set of sampling parameters for generation: \`temperature=1.0\`, \`top\_p=0.95\`.85 86```py87# pip install transformers88from transformers import AutoTokenizer, AutoModelForCausalLM89 90model_id = "CohereLabs/North-Mini-Code-1.0"91tokenizer = AutoTokenizer.from_pretrained(model_id)92model = AutoModelForCausalLM.from_pretrained(model_id)93 94prompt = "Write a python program to check if a string is a palindrome or not."95 96# Format message with the North-Mini-Code-1.0 chat template97messages = [{"role": "user", "content": prompt}]98input_ids = tokenizer.apply_chat_template(99 messages,100 tokenize=True,101 add_generation_prompt=True,102 return_tensors="pt",103)104 105gen_tokens = model.generate(106 **input_ids, 107 max_new_tokens=1024, 108 do_sample=True, 109 temperature=1.0,110 top_p=0.95111)112 113gen_text = tokenizer.decode(gen_tokens[0])114print(gen_text)115```116 117You can also use the model directly using transformers `pipeline` abstraction:118 119```py120from transformers import pipeline121import torch122 123model_id = "CohereLabs/North-Mini-Code-1.0"124 125prompt = """Given a list of unique words each of size k and an n sized word, w, where n is a multiple of k,126Write a program in python to determine the number of unique combinations of words in the list that can be concatenated to form an anagram of the word w.127"""128 129pipe = pipeline(130 "text-generation",131 model=model_id,132 torch_dtype="auto",133 device_map="auto",134)135 136messages = [137 {"role": "user", "content": f"{prompt}"},138]139 140text = tokenizer.apply_chat_template(141 messages,142 tokenize=False,143 add_generation_prompt=True,144)145 146 147outputs = pipe(148 messages,149 max_new_tokens=1024,150 do_sample=True, 151 temperature=1.0,152 top_p=0.95153 154)155 156print(outputs[0]["generated_text"][-1])157 158```159 160## **Model Details**161 162**Input**: Text only.163 164**Output**: Model generates text. 165 166**Model Architecture**: North-Mini-Code-1.0 is a decoder-only Transformer-based sparse Mixture-of-Experts model. It uses an efficient attention implementation, interleaved between sliding-window attention with RoPE and global attention with no positional embeddings, in a 3:1 ratio. The feed-forward block is an MoE block with 128 experts, of which 8 are activated per token. Each expert block is an FFN block with SwiGLU activation. The router applies a sigmoid activation function to the logits before the top-k selection. We also use a single dense layer before the sparse layers. North-Mini-Code-1.0 was post-trained using a two-stage cascaded supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR), focusing on agentic coding. For more technical details, please check out our [blog post](https://huggingface.co/blog/CohereLabs/introducing-north-mini-code). 167 168**Context Length:** North-Mini-Code-1.0 supports a context length of 256K & 64K output length.169 170### **Tool Use Capabilities:**171 172North-Mini-Code-1.0 has been specifically trained with tool-use capabilities for agentic coding.173 174Tool use with North-Mini-Code-1.0 is supported through [chat templates](https://huggingface.co/docs/transformers/main/en/chat_templating#advanced-tool-use--function-calling) in Transformers. We recommend providing tool descriptions using JSON schema.175 176**Tool Use Example \[CLICK TO EXPAND\]** 177 178```py179# Define tools180tools = [{181 "type": "function",182 "function": {183 "name": "bash",184 "description": "Execute a bash command in the terminal.",185 "parameters": {186 "type": "object",187 "properties": {188 "command": {189 "description": "The bash command to execute.",190 "type": "string"191 }192 },193 "required": ["command"]194 },195 }196}]197 198# Define conversation input199conversation = [{"role": "user", "content": "Find out if there is any json file in this folder"}]200 201 202# Get the Tool Use prompt203input_prompt = tokenizer.apply_chat_template(conversation=conversation, tools=tools, tokenize=False, add_generation_prompt=True, return_tensors="pt")204 205# Tokenize the prompt206input_ids = tokenizer(input_prompt, return_tensors="pt")207```208 209You can then generate from this input as normal.210 211North Mini Code, similarly as all the other Cohere agent models released to date, supports [interleaved thinking](https://docs.vllm.ai/en/latest/features/interleaved_thinking/) and works best when turned on. You’re strongly encouraged to pass on all the model-generated thinking contents to future agentic steps, and chat turns for the best model performance. Please refer to the linked vllm doc and see how it’s done.212 213If the model generates thinking content and tool calls, you should add both of them to the chat history like so:214 215```py216# Pass on the tool_call and thinking217tool_call = {"name": "bash", "arguments": {"command": "ls -al"}}218reasoning = "The user wants to find if there are any JSON files in the current folder. I should use the `ls` command to list files and then check if there are any JSON files (files ending with .json). Let me first list the files in the current directory."219 220conversation.append({"role": "assistant", "tool_calls": [{"id": "0", "type": "function", "function": tool_call}], "reasoning": reasoning})221```222 223and then call the tool and append the result, as a dictionary, with the tool role, like so:224 225```py226# This needs to be a dictionary227tool_result = {"stdout": "test.json\ntest.py", "return_code": "0"} 228 229# Append tool results230conversation.append({"role": "tool", "tool_call_id": "0", "content": tool_result})231```232 233After that, you can `generate()` again to let the model use the tool result in the chat.234 235Note that this was a very brief introduction to tool calling \- for more information the Transformers [tool use documentation](https://huggingface.co/docs/transformers/main/chat_templating#advanced-tool-use--function-calling).236 237### **vLLM**238 239You can also run the model in vLLM. Please use vLLM main for North Mini Code until a new release is available, and accurate response parsing also requires installing Cohere’s melody library.240 241```shell242uv pip install "git+https://github.com/vllm-project/vllm.git"243uv pip install cohere_melody>=0.9.0244```245 246Then the vllm server can be started with the following command:247 248```shell249vllm serve CohereLabs/North-Mini-Code-1.0 \250 -tp 2 \251 --max-model-len 320000 \252 --tool-call-parser cohere_command4 \253 --reasoning-parser cohere_command4 \254 --enable-auto-tool-choice255```256 257**Use locally deployed North Mini Code in OpenCode:**258 259Please use OpenCode main branch until a new release is available.260 261```shell262# Example commands to install on linux263git clone https://github.com/anomalyco/opencode.git cd opencode264 265# Install Bun266curl -fsSL https://bun.sh/install | bash267export BUN_INSTALL="$HOME/.bun"268export PATH="$BUN_INSTALL/bin:$PATH"269 270# node-gyp was needed by a dependency271bun add -g node-gyp272 273# Install dependencies274bun install275 276# Build CLI277bun run --cwd packages/opencode build /usr/bin/install -m 755 \278 ./opencode/packages/opencode/dist/opencode-linux-x64/bin/opencode \279 /root/.local/bin/opencode280```281 282To use locally deployed North Mini Code in Opencode, please use this config which enables interleaved reasoning:283 284```json285{286 "$schema": "https://opencode.ai/config.json",287 "model": "vllm/CohereLabs/North-Mini-Code-1.0",288 "provider": {289 "vllm": {290 "npm": "@ai-sdk/openai-compatible",291 "name": "Local vLLM server",292 "options": {293 "baseURL": "http://127.0.0.1:8000/v1",294 "apiKey": "EMPTY"295 },296 "models": {297 "North-Mini-Code-1.0": {298 "name": "North-Mini-Code-1.0",299 "interleaved": {300 "field": "reasoning"301 },302 "limit": {303 "context": 256000,304 "output": 64000305 }306 }307 }308 }309 }310}311 312```313 314## **Model Card Contact**315 316For errors or additional questions about details in this model card, contact \[labs@cohere.com\].