ariacompute/qwen3-1.7b_q326_channel

Model Card for Qwen3-1.7B (Aria Quant Bundle, q326)

Model Details

Model Description

Qwen3 1.7B is a 1.7-billion-parameter, dense Transformer decoder-only language model developed by the Qwen team at Alibaba Cloud, pre-trained on diverse public corpora and aligned via supervised fine-tuning (SFT) and direct preference optimization (DPO). This distribution is provided by Aria Compute as an aria-quant-bundle — a mixed-precision quantized package using Hadamard rotation + Lloyd-Max codebook quantization with per-group codebooks (group size 32). Sensitive layers are kept at 4-bit while the remaining layers are pushed to ~3-bit (average ~3.26 bits per weight), delivering ~4× compression vs FP16 (~3.4 GB → ~0.85 GB). Optimized for CPU-only, on-device inference on mobile phones, edge devices, and single-board computers via the Aria Engine runtime. No GPU or cloud connection is required.

• —Developed by: Qwen Team (Alibaba Cloud)
• —Quantized and distributed by: Aria Compute
• —Model type: Dense Transformer decoder-only (language)
• —Language(s): English (primary), Chinese, and 20+ additional languages
• —License: Apache 2.0
• —Finetuned from model: Qwen/Qwen3-1.7B

Model Sources

• —Original Repository: QwenLM/Qwen3
• —Original Paper: Qwen3 Technical Report (pending)
• —Aria Compute Dashboard: ariacompute.com/dashboard/models
• —Aria Engine: ariacompute.com

Uses

Direct Use

This quantized bundle is intended for on-device, offline text-generation tasks on resource-constrained hardware, including:

• —On-device chat and conversational assistants
• —Real-time text completion and sentence prediction
• —Structured tool calling / function calling for mobile and IoT APIs
• —Lightweight text embeddings for on-device retrieval and classification
• —Short-form summarization of notifications, messages, and local content

Target Devices

Memory breakdown (q326, at 4K context): ~0.85 GB quantized model weights (mmap) + ~112 MB KV cache + ~50 MB runtime overhead ≈ ~1.01 GB.

Note: The KV cache size is identical to qwen3-0.6b (28 layers × 8 KV heads × head_dim=128), as GQA configuration is shared across the Qwen3 family.

Out-of-Scope Use

• —Long-form creative writing (>2K tokens per generation)
• —Mathematical theorem proving or complex multi-step reasoning
• —Full program synthesis (reliable for short functions only)
• —Multimodal input (this model is text-only)
• —Real-time audio/speech processing (use Aria speech models)
• —Safety-critical decision systems without human oversight

How to Get Started with the Model

Download from Aria Compute

Authenticated dashboard users can download the bundle via: https://ariacompute.com/dashboard/models

Quantization Recipe

This bundle uses the mixed-precision q326 recipe — sensitive layers at 4-bit, others at ~3-bit, with per-group codebooks:

• —Bundle size: ~0.85 GB (FP16 original: ~3.4 GB, ~4× compression; ~23% smaller than q4's ~1.1 GB)
• —Average bit-width: ~3.26 bits per weight (sensitive layers at 4-bit, others at ~3-bit)
• —Generation quality: Awaiting genquanteval audit. Method reference (qwen3-0.6b_q4 group baseline): token overlap 0.1878, exact prefix fraction 0.0729, logprob delta −0.172159
• —Calibration-free: Hadamard rotation + Lloyd-Max codebook, no task-specific calibration data required
• —Mixed-precision group-codebook recipe: This is the mixed-precision quantized variant for Qwen3-1.7B using per-group codebooks. For recommended generation quality, use qwen3-1.7b_q326_channel (same mixed-precision strategy with per-channel codebooks — optimal quality-size trade-off). For near-lossless fidelity, use qwen3-1.7b_q8. For maximum compression, use the standard qwen3-1.7b_q4 baseline

Model Architecture

Qwen3-1.7B employs a standard dense Transformer decoder architecture:

Design highlights (shared with Qwen3 family):

• —GQA (Grouped Query Attention): 8 KV heads serving 16 query heads — halves KV Cache memory
• —RoPE high base frequency (1M): Native support for up to 40K context length (32K to 40K depending on member)
• —Dense FFN + SiLU gating: High inference efficiency, suitable for on-device use
• —Tied vocab: Input embedding and output projection weights are shared, saving ~311M parameters

Key architecture differences from Qwen3-0.6B:

• —Double hidden width (2048 vs 1024) and ~2.2× FFN width (6144 vs 2816) — the primary sources of the 1.1B parameter increase while keeping the same 28-layer depth
• —40K max context vs 32K — allows longer document processing at higher memory cost for extreme sequence lengths

Bias, Risks, and Limitations

Limitations

• —Reasoning depth: Multi-step logical reasoning (≥3 steps) benefits from the larger capacity compared to 0.6B-class models, but still falls short of 7B+ frontier models. Verify outputs in high-stakes scenarios.
• —Mathematics: GSM8K and MATH performance is improved over 0.6B models, but remains modest. Use larger models for quantitative tasks requiring precision.
• —Code generation: Capable of short function completions and multi-line snippets; unreliable for multi-file synthesis or algorithmic problem solving.
• —Factual knowledge: Improved world knowledge over 0.6B models due to wider hidden dimension and larger FFN capacity, but still limited compared to larger models. Always verify factual claims against authoritative sources.
• —Instruction following: Handles moderate multi-constraint prompts reliably at typical context lengths. Keep complex, highly constrained instructions within 2-3 constraints.
• —Quantization drift: As a mixed-precision recipe, non-sensitive layers at ~3-bit may exhibit higher drift than uniform 4-bit. For optimal generation quality, consider q326_channel (recommended recipe) with per-channel codebooks. For near-lossless fidelity, use q8.

Bias and Risks

• —Bias: As with all large language models trained on web-scale data, Qwen3 may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
• —Toxicity: The base model has been safety-aligned with refusal training. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
• —Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications.
• —Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

Users (both direct and downstream) should be made aware of the above risks, biases, limitations, and constraints of the model. We recommend:

• —Adding a lightweight output safety classifier for user-facing deployments
• —Verifying factual claims with external knowledge bases
• —Not using the model for high-stakes decisions without human review