formulae/mita-v1.2-7b-2-24-2025
Formulae/MITA-V1.2-7B-2-24-2025
Overview
Formulae/MITA-V1.2-7B is designed primarily for mathematics, with some capability in coding. Built using the Linear DARE merge method, this model blends powerful mathematical reasoning with computational accuracy.
Merge Details
- Base Model: Qwen/Qwen2.5-Math-7B-Instruct
- Merged Models:
- nvidia/AceMath-7B-Instruct
- open-r1/OpenR1-Qwen-7B
- Merge Method: Linear DARE
- Data Type: bfloat16
- Merge Parameters:
- Density & Weight: 0.5 for AceMath & OpenR1
- Normalization: Disabled
- Int8 Masking: Enabled
What is DARE?
DARE (Density-Aware Residual Estimation) is an advanced model merging technique designed to preserve task-specific knowledge. Unlike simple model averaging, DARE adjusts parameter density to ensure that merged models retain their specialized skills while improving general performance.
📖 Reference: DARE Paper
This merge is also inspired by task arithmetic, which shows that models can be linearly combined to enhance capabilities in specialized domains.
📖 Reference: Task Arithmetic Paper
Capabilities
✅ Advanced Mathematics – Strong problem-solving, algebra, calculus, and theorem applications. ✅ Limited Coding Support – Can handle basic programming tasks but is not optimized for complex software development.
Limitations & Risks
⚠ Hallucinations in Code – Not a coding-specialized model, may produce incorrect or insecure implementations. ⚠ Arithmetic Errors – While highly capable, the model still makes occasional miscalculations.
Usage Disclaimer
Formulae/MITA-V1.2-7B is an experimental mathematical model. For verified accuracy, always cross-check results with reliable tools.
Contribute
We welcome contributions, including quantizations, fine-tuning, and further enhancements.
💡 Support Us: Buy Me a Coffee
📩 Contact: formulaeresearch@gmail.com
Future Development
This is part of the MITA series. Future iterations will integrate MoE (Mixture of Experts) for even more specialized reasoning across multiple domains.
Made possible with [MergeKit](https://github.com/arcee-ai/mergekit).