cesun/ThinkEdit-deepseek-llama3-8b

Repository for:

ThinkEdit-deepseek-llama3-8b

(We also release ThinkEdit versions for ThinkEdit-deepseek-qwen-1.5b, ThinkEdit-deepseek-qwen-14b, and ThinkEdit-deepseek-qwen-32b.)

Authors: Chung-En Sun, Ge Yan, Tsui-Wei Weng Paper: ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models

Github: https://github.com/Trustworthy-ML-Lab/ThinkEdit

Introduction

Reasoning-augmented models sometimes fail by generating overly short, abstract chain-of-thought (CoT) reasoning, hurting their accuracy.

ThinkEdit is a lightweight weight-editing method that:

• —Identifies ~4% of "short reasoning" attention heads
• —Edits only ~0.2% of total parameters
• —Removes the "short reasoning" direction from their output
• —Boosts performance, especially on cases with short reasoning traces

Full Performance Results

1. Overall Accuracy

2. Accuracy on Short Reasoning Cases (Top 5% / 10% / 20%)

Usage

The usage of ThinkEdit models is exactly the same as the original deepseek-distilled models.

Citation

@misc{sun2025thinkedit,
      title={ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models}, 
      author={Chung-En Sun and Ge Yan and Tsui-Wei Weng},
      year={2025},
      eprint={2503.22048},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2503.22048}, 
}