cortexso/small-thinker

Overview

PowerInfer developed and released the SmallThinker-3B-preview, a fine-tuned version of the Qwen2.5-3B-Instruct model. SmallThinker is optimized for efficient deployment on resource-constrained devices while maintaining high performance in reasoning, coding, and general text generation tasks. It outperforms its base model on key benchmarks, including AIME24, AMC23, and GAOKAO2024, making it a robust tool for both edge deployment and as a draft model for larger systems like QwQ-32B-Preview.

SmallThinker was fine-tuned in two phases using high-quality datasets, including PowerInfer/QWQ-LONGCOT-500K and PowerInfer/LONGCOT-Refine-500K. Its small size allows for up to 70% faster inference speeds compared to larger models, making it ideal for applications requiring quick responses and efficient computation.

Variants

Use it with Jan (UI)

1. 1.Install Jan using Quickstart
2. 2.Use in Jan model Hub:

    cortexso/small-thinker

Use it with Cortex (CLI)

1. 1.Install Cortex using Quickstart
2. 2.Run the model with command:

    cortex run small-thinker

Credits

• —Author: PowerInfer
• —Converter: Homebrew
• —Original License: License