ArchiveStudio/Qwen3-Coder-Next-Base

1---2library_name: transformers3license: apache-2.04license_link: https://huggingface.co/Qwen/Qwen3-Coder-Next-Base/blob/main/LICENSE5pipeline_tag: text-generation6---7 8# Qwen3-Coder-Next-Base9 10## Highlights11 12Today, we're announcing **Qwen3-Coder-Next-Base**, an open-weight language model designed specifically for coding agents and local development. It features the following key enhancements:  13 14- **Advanced architecture**: It integrates the Hybrid Attention with highly sparse MoE, enabling high throughput and strong ultra-long-context modeling.15 16- **Robust data foundation**: Trained on highly diverse, broad-coverage corpora, with native 256K context and support for 370+ languages, it leaves ample headroom for post-training.17 18- **Agentic coding capability**: With a carefully designed training recipe, it has strong capabilities in tool calling, scaffold/template adaptation, and error detection/recovery, making it a strong backbone for reliable coding agents.19 20## Model Overview21 22**Qwen3-Coder-Next-Base** has the following features:23- Type: Causal Language Models24- Training Stage: Pretraining25- Number of Parameters: 80B in total and 3B activated26- Number of Parameters (Non-Embedding): 79B27- Hidden Dimension: 204828- Number of Layers: 4829  - Hybrid Layout: 12 \* (3 \* (Gated DeltaNet -> MoE) -> 1 \* (Gated Attention -> MoE))30- Gated Attention:31  - Number of Attention Heads: 16 for Q and 2 for KV32  - Head Dimension: 25633  - Rotary Position Embedding Dimension: 6434- Gated DeltaNet:35  - Number of Linear Attention Heads: 32 for V and 16 for QK36  - Head Dimension: 12837- Mixture of Experts:38  - Number of Experts: 51239  - Number of Activated Experts: 1040  - Number of Shared Experts: 141  - Expert Intermediate Dimension: 51242- Context Length: 262,144 natively43 44**NOTE: This model supports only non-thinking mode and does not generate ``<think></think>`` blocks in its output. Meanwhile, specifying `enable_thinking=False` is no longer required.**45 46For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwen.ai/blog?id=qwen3-coder-next), [GitHub](https://github.com/QwenLM/Qwen3-Coder), and [Documentation](https://qwen.readthedocs.io/en/latest/).47 48## Best Practices49 50To achieve optimal performance, we recommend the following sampling parameters: `temperature=1.0`, `top_p=0.95`, `top_k=40`.51 52 53## Citation54 55If you find our work helpful, feel free to give us a cite.56 57```58@techreport{qwen_qwen3_coder_next_tech_report,59  title        = {Qwen3-Coder-Next Technical Report},60  author       = {{Qwen Team}},61  url          = {https://github.com/QwenLM/Qwen3-Coder/blob/main/qwen3_coder_next_tech_report.pdf},62  note         = {Accessed: 2026-02-03}63}64```