ShaunGves/North-Mini-Code-1.0

1---2inference: false3library_name: transformers4license: apache-2.05tags:6- conversational7- chat8- code9- agent10---11 12# **Model Card for North Mini Code**13 14## **Model Summary**15 16North Mini Code is an open weights research release of a 30B-A3B parameter model optimized for code generation, agentic software engineering, and terminal tasks.17 18Developed by: [Cohere](https://cohere.com/) and [Cohere Labs](https://cohere.com/research)19 20* Point of Contact: [**Cohere Labs**](https://cohere.com/research)  21* License: Apache 2.022* Model: North Mini Code  23* Model Size: 30B total; 3B active  24* Context length: 256K & 64K max output25 26For more details about this model, please check out our [blog post](https://huggingface.co/blog/CohereLabs/introducing-north-mini-code). 27 28**Try North Mini Code**29 30You can try out North Mini Code before downloading the weights in OpenCode and our hosted [Hugging Face Space](https://huggingface.co/spaces/CohereLabs/North-Mini-Code-1.0).31 32**Evaluation**33 34![image1](https://cdn-uploads.huggingface.co/production/uploads/62668f725fb8d521d94d8451/xR7kZ3X9RKEZrbgD6hpG1.png)35 36<details>37<summary><span style="font-size: 80%;"><b>Benchmarking Methodology [CLICK TO EXPAND]</span></b></summary>38 39- <span style="font-size: 80%;">We used SWE-Bench Verified, SWE-Bench Pro, Terminal-Bench v2, and Terminal-Bench Hard to benchmark North Mini Code's agentic coding capabilities. For evaluation harnesses, we used the Swe-Agent harness v1.1.0 for SWE-Bench, and a simple ReAct harness employing a single terminal-use tool based on Harbor's Tmux session implementation for Terminal-Bench v2. For Terminal Bench Hard, we directly used Terminus-2, following the same methodology as the Artificial Analysis Intelligence Index to compare North-Mini-Code-1.0 with the other models. Additionally, we used SciCode and LiveCodeBench v6 as complex code-generation benchmarks outside of tool use.</span>40- <span style="font-size: 80%;">We run each benchmark with 3 different seeds and report the average benchmark performance, using temperature=1.0 and top\_p=0.95. We used publicly reported scores for competitor models, either from original reports or the Artificial Analysis Intelligence Index, where available. Additionally, Gemma4’s scores for agentic coding tasks were reported by [Qwen team](https://qwen.ai/blog?id=qwen3.6-35b-a3b). For benchmark results that any public report is missing, denoted by (\*) in the figure, we run them internally using the recommended model configuration.</span>41</details>42 43**Usage**44 45Please install transformers from the source repository that includes the necessary changes for this model. We recommend using the following set of sampling parameters for generation: \`temperature=1.0\`, \`top\_p=0.95\`.46 47```py48# pip install transformers49from transformers import AutoTokenizer, AutoModelForCausalLM50 51model_id = "CohereLabs/North-Mini-Code-1.0"52tokenizer = AutoTokenizer.from_pretrained(model_id)53model = AutoModelForCausalLM.from_pretrained(model_id)54 55prompt = "Write a python program to check if a string is a palindrome or not."56 57# Format message with the North-Mini-Code-1.0 chat template58messages = [{"role": "user", "content": prompt}]59input_ids = tokenizer.apply_chat_template(60    messages,61    tokenize=True,62    add_generation_prompt=True,63    return_tensors="pt",64)65 66gen_tokens = model.generate(67    **input_ids, 68    max_new_tokens=1024, 69    do_sample=True, 70    temperature=1.0,71    top_p=0.9572)73 74gen_text = tokenizer.decode(gen_tokens[0])75print(gen_text)76```77 78You can also use the model directly using transformers `pipeline` abstraction:79 80```py81from transformers import pipeline82import torch83 84model_id = "CohereLabs/North-Mini-Code-1.0"85 86prompt = """Given a list of unique words each of size k and an n sized word, w, where n is a multiple of k,87Write a program in python to determine the number of unique combinations of words in the list that can be concatenated to form an anagram of the word w.88"""89 90pipe = pipeline(91    "text-generation",92    model=model_id,93    torch_dtype="auto",94    device_map="auto",95)96 97messages = [98    {"role": "user", "content": f"{prompt}"},99]100 101text = tokenizer.apply_chat_template(102    messages,103    tokenize=False,104    add_generation_prompt=True,105)106 107 108outputs = pipe(109    messages,110    max_new_tokens=1024,111    do_sample=True, 112    temperature=1.0,113    top_p=0.95114 115)116 117print(outputs[0]["generated_text"][-1])118 119```120 121## **Model Details**122 123**Input**: Text only.124 125**Output**: Model generates text. 126 127**Model Architecture**: North-Mini-Code-1.0 is a decoder-only Transformer-based sparse Mixture-of-Experts model. It uses an efficient attention implementation, interleaved between sliding-window attention with RoPE and global attention with no positional embeddings, in a 3:1 ratio. The feed-forward block is an MoE block with 128 experts, of which 8 are activated per token. Each expert block is an FFN block with SwiGLU activation. The router applies a sigmoid activation function to the logits before the top-k selection. We also use a single dense layer before the sparse layers. North-Mini-Code-1.0 was post-trained using a two-stage cascaded supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR), focusing on agentic coding.  For more technical details, please check out our [blog post](https://huggingface.co/blog/CohereLabs/introducing-north-mini-code). 128 129**Context Length:** North-Mini-Code-1.0 supports a context length of 256K & 64K output length.130 131### **Tool Use Capabilities:**132 133North-Mini-Code-1.0 has been specifically trained with tool-use capabilities for agentic coding.134 135Tool use with North-Mini-Code-1.0 is supported through [chat templates](https://huggingface.co/docs/transformers/main/en/chat_templating#advanced-tool-use--function-calling) in Transformers. We recommend providing tool descriptions using JSON schema.136 137**Tool Use Example \[CLICK TO EXPAND\]** 138 139```py140# Define tools141tools = [{142  "type": "function",143  "function": {144    "name": "bash",145    "description": "Execute a bash command in the terminal.",146    "parameters": {147      "type": "object",148      "properties": {149        "command": {150          "description": "The bash command to execute.",151          "type": "string"152        }153      },154      "required": ["command"]155    },156  }157}]158 159# Define conversation input160conversation = [{"role": "user", "content": "Find out if there is any json file in this folder"}]161 162 163# Get the Tool Use prompt164input_prompt = tokenizer.apply_chat_template(conversation=conversation, tools=tools, tokenize=False, add_generation_prompt=True, return_tensors="pt")165 166# Tokenize the prompt167input_ids = tokenizer(input_prompt, return_tensors="pt")168```169 170You can then generate from this input as normal.171 172North Mini Code, similarly as all the other Cohere agent models released to date, supports [interleaved thinking](https://docs.vllm.ai/en/latest/features/interleaved_thinking/) and works best when turned on. You’re strongly encouraged to pass on all the model-generated thinking contents to future agentic steps, and chat turns for the best model performance. Please refer to the linked vllm doc and see how it’s done.173 174If the model generates thinking content and tool calls, you should add both of them to the chat history like so:175 176```py177# Pass on the tool_call and thinking178tool_call = {"name": "bash", "arguments": {"command": "ls -al"}}179reasoning = "The user wants to find if there are any JSON files in the current folder. I should use the `ls` command to list files and then check if there are any JSON files (files ending with .json). Let me first list the files in the current directory."180 181conversation.append({"role": "assistant", "tool_calls": [{"id": "0", "type": "function", "function": tool_call}], "reasoning": reasoning})182```183 184and then call the tool and append the result, as a dictionary, with the tool role, like so:185 186```py187# This needs to be a dictionary188tool_result = {"stdout": "test.json\ntest.py", "return_code": "0"} 189 190# Append tool results191conversation.append({"role": "tool", "tool_call_id": "0", "content": tool_result})192```193 194After that, you can `generate()` again to let the model use the tool result in the chat.195 196Note that this was a very brief introduction to tool calling \- for more information the Transformers [tool use documentation](https://huggingface.co/docs/transformers/main/chat_templating#advanced-tool-use--function-calling).197 198### **vLLM**199 200You can also run the model in vLLM. Please use vLLM main for North Mini Code until a new release is available, and accurate response parsing also requires installing Cohere’s melody library.201 202```shell203uv pip install "git+https://github.com/vllm-project/vllm.git"204uv pip install cohere_melody>=0.9.0205```206 207Then the vllm server can be started with the following command:208 209```shell210vllm serve CohereLabs/North-Mini-Code-1.0 \211  -tp 2 \212  --max-model-len 320000 \213  --tool-call-parser cohere_command4 \214  --reasoning-parser cohere_command4 \215  --enable-auto-tool-choice216```217 218**Use locally deployed North Mini Code in OpenCode:**219 220Please use OpenCode main branch until a new release is available.221 222```shell223# Example commands to install on linux224git clone https://github.com/anomalyco/opencode.gitcd opencode225 226# Install Bun227curl -fsSL https://bun.sh/install | bash228export BUN_INSTALL="$HOME/.bun"229export PATH="$BUN_INSTALL/bin:$PATH"230 231# node-gyp was needed by a dependency232bun add -g node-gyp233 234# Install dependencies235bun install236 237# Build CLI238bun run --cwd packages/opencode build/usr/bin/install -m 755 \239  ./opencode/packages/opencode/dist/opencode-linux-x64/bin/opencode \240  /root/.local/bin/opencode241```242 243To use locally deployed North Mini Code in Opencode, please use this config which enables interleaved reasoning:244 245```json246{247  "$schema": "https://opencode.ai/config.json",248  "model": "vllm/CohereLabs/North-Mini-Code-1.0",249  "provider": {250    "vllm": {251      "npm": "@ai-sdk/openai-compatible",252      "name": "Local vLLM server",253      "options": {254        "baseURL": "http://127.0.0.1:8000/v1",255        "apiKey": "EMPTY"256      },257      "models": {258        "North-Mini-Code-1.0": {259          "name": "North-Mini-Code-1.0",260          "interleaved": {261            "field": "reasoning"262          },263          "limit": {264            "context": 256000,265            "output": 64000266          }267        }268      }269    }270  }271}272 273```274 275## **Model Card Contact**276 277For errors or additional questions about details in this model card, contact \[labs@cohere.com\].