SOTAagi2030/AssistantModel-Best

1---2license: apache-2.03library_name: transformers4---5# AssistantModel6 7<!-- markdownlint-disable first-line-h1 -->8<!-- markdownlint-disable html -->9<!-- markdownlint-disable no-duplicate-header -->10 11<div align="center">12  <img src="figures/fig1.png" width="60%" alt="AssistantModel" />13</div>14<hr>15 16<div align="center" style="line-height: 1;">17  <a href="LICENSE" style="margin: 2px;">18    <img alt="License" src="figures/fig2.png" style="display: inline-block; vertical-align: middle;"/>19  </a>20</div>21 22## 1. Introduction23 24AssistantModel is designed for interactive assistant applications. This checkpoint is selected based on the combined performance of knowledge retrieval and instruction following benchmarks, making it ideal for AI assistant deployment.25 26<p align="center">27  <img width="80%" src="figures/fig3.png">28</p>29 30## 2. Evaluation Results31 32### Comprehensive Benchmark Results33 34<div align="center">35 36| | Benchmark | Assistant-v1 | Assistant-v2 | AssistantModel |37|---|---|---|---|---|38| **Core Reasoning Tasks** | Math Reasoning | 0.510 | 0.535 | 0.550 |39| | Logical Reasoning | 0.789 | 0.801 | 0.819 |40| | Common Sense | 0.716 | 0.702 | 0.736 |41| **Language Understanding** | Reading Comprehension | 0.671 | 0.685 | 0.700 |42| | Question Answering | 0.582 | 0.599 | 0.607 |43| | Text Classification | 0.803 | 0.811 | 0.828 |44| | Sentiment Analysis | 0.777 | 0.781 | 0.792 |45| **Generation Tasks** | Code Generation | 0.615 | 0.631 | 0.650 |46| | Creative Writing | 0.588 | 0.579 | 0.636 |47| | Dialogue Generation | 0.621 | 0.635 | 0.644 |48| | Summarization | 0.745 | 0.755 | 0.767 |49| **Specialized Capabilities**| Translation | 0.782 | 0.799 | 0.804 |50| | Knowledge Retrieval | 0.651 | 0.668 | 0.676 |51| | Instruction Following | 0.733 | 0.749 | 0.758 |52| | Safety Evaluation | 0.718 | 0.701 | 0.803 |53 54</div>55 56## 3. License57[Apache-2.0 License](LICENSE)58 59## 4. Contact60Open an issue on GitHub.61