ServiceNow-AI/Apriel-5B-Base

1---2library_name: transformers3language:4- en5license: mit6---7 8# Apriel-5B9 10`/ˈɑː.pri.əl/`11 12## Table of Contents13 141. [Model Summary](#model-summary)  152. [Evaluation](#evaluation)  163. [Intended Use](#intended-use)  174. [Limitations](#limitations)  185. [Security and Responsible Use](#security-and-responsible-use)  196. [License](#license)  207. [Citation](#citation)21 22## Model Summary23 24Apriel is a family of models built for versatility, offering high throughput and efficiency across a wide range of tasks.  25 26### Apriel-5B-Base27Apriel-5B-base is a decoder-only transformer trained on 4.5T+ tokens of data. It is the first release in the Apriel model family, designed to support research on foundation models. Apriel-5B-base achieves strong performance across common benchmarks for models under 5B parameters.28 29### Apriel-5B-Instruct30[Apriel-5B-Instruct](https://huggingface.co/ServiceNow-AI/Apriel-5B-Instruct) is built on top of [Apriel-5B-base](https://huggingface.co/ServiceNow-AI/Apriel-5B-base) using continual pretraining (CPT), supervised finetuning (SFT), and post-training alignment with DPO and RLVR.31 32Both CPT and SFT stages involved training multiple domain-biased variants with overlapping datasets (e.g., instruction, code, math). These were then merged to form a more general-purpose model before alignment. The final model is aligned for instruction following, reasoning, and safety-aware dialogue.33 34<img src="https://huggingface.co/ServiceNow-AI/Apriel-4.8B-base/resolve/main/eval_vs_latency.png" alt="graph" width="400"/>35 36The y-axis shows average downstream benchmark scores. Throughput (x-axis) was measured using [vLLM](https://github.com/vllm-project/vllm) with batch size 8, 256 input tokens, and 32 output tokens.37 38### How to Use39 40```bash41pip install transformers42```43 44#### Running the Base model45```python46import torch47from transformers import AutoModelForCausalLM, AutoTokenizer48 49checkpoint = "ServiceNow-AI/Apriel-5B-Base"50device = "cuda"  # or "cpu"51 52tokenizer = AutoTokenizer.from_pretrained(checkpoint)53model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16).to(device)54 55inputs = tokenizer.encode("Snow is", return_tensors="pt").to(device)56outputs = model.generate(inputs)57print(tokenizer.decode(outputs[0]))58```59 60```bash61>>> print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")62Memory footprint: 9664.14 MB63```64 65#### Running the Instruct model66 67```python68import torch69from transformers import AutoModelForCausalLM, AutoTokenizer70 71checkpoint = "ServiceNow-AI/Apriel-5B-Instruct"72tokenizer = AutoTokenizer.from_pretrained(checkpoint)73device = "cuda" if torch.cuda.is_available() else "cpu"74 75model = AutoModelForCausalLM.from_pretrained(76    checkpoint, 77    torch_dtype=torch.bfloat16 if device == "cuda" else torch.float3278).to(device)79 80messages = [81    {"role": "system", "content": "You are a helpful AI assistant that provides accurate and concise information."},82    {"role": "user", "content": "Tell me about artificial intelligence"}83]84 85input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)86inputs = tokenizer(input_text, return_tensors="pt").to(device)87 88generation_params = {89    "max_new_tokens": 512,90    "temperature": 0.2,91    "top_p": 0.9,92    "do_sample": True93}94 95outputs = model.generate(**inputs, **generation_params)96response = tokenizer.decode(outputs[0], skip_special_tokens=True)97print(response)98```99 100### Chat Template101 102```103<|system|>104System message here (optional)105<|end|>106<|user|>107User message here108<|end|>109<|assistant|>110Assistant response here111<|end|>112```113 114If no system message is provided, the model inserts a blank system prompt to maintain format structure. The model supports structured interaction patterns, including tool calling and reasoning steps for more advanced workflows.115 116## Evaluation117 118Evaluations were conducted using [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness) and [evalchemy](https://github.com/mlfoundations/evalchemy).119 120### Apriel-5B-Base121 122| Task Name           | Apriel-5B-Base | OLMo-2-1124-7B | Llama-3.1-8B | Mistral-Nemo-Base-2407 |123|---------------------|------------------|----------------|--------------|-------------------------|124| **Average**         | 58.7             | 58.71          | 61.72        | 66.01                    |125| **ARC Challenge**   | 56.7             | 62.7           | 58.2         | 62.9                    |126| **ARC Easy**        | 82.4             | 86.0           | 85.7         | 86.7                    |127| **MMMLU**           | 44.5             | 35.3           | 47.4         | 54.7                    |128| **Global MMLU**     | 57.4             | 52.4           | 61.1         | 68.4                    |129| **GSM8k**           | 64.2             | 63.2           | 54.8         | 58.5                    |130| **HellaSwag**       | 74.4             | 80.5           | 78.8         | 82.7                    |131| **MUSR**            | 39.1             | 39.6           | 38.0         | 39.9                    |132| **MBPP**            | 27.6             | 22.4           | 46.0         | 54.6                    |133| **MMLU**            | 61.3             | 63.9           | 66.0         | 69.6                    |134| **PIQA**            | 78.9             | 81.1           | 81.2         | 82.1                    |135 136 137 138### Apriel-5B-Instruct139 140| Task Name    | Apriel-5B-Instruct | OLMo-2-1124-7B-Instruct | Llama-3.1-8B-Instruct | Mistral-Nemo-Instruct-2407 |141|--------------|--------------------|--------------------------|------------------------|----------------------------|142| **Average**      | 49.64             | 43.91                   | 	52.60                 | 	48.63                      |143| **ARC Challenge** | 59.04              | 61.45                   | 64.25                 | 66.38                      |144| **GSM8k**        | 80.36              | 79.68                   | 82.63                 | 77.63                      |145| **Hellaswag**    | 74.52              | 80.21                   | 78.43                 | 81.71                      |146| **BBH**          | 39.82              | 39.95                   | 50.86                 | 50.06                      |147| **GPQA**         | 28.36              | 27.85                   | 29.19                 | 29.45                      |148| **IF Eval**      | 80.78              | 72.64                   | 79.67                 | 62.85                      |149| **MMLU Pro**     | 29.19              | 26.57                   | 37.74                 | 35.09                      |150| **MUSR**         | 36.77              | 34.39                   | 38.36                 | 39.02                      |151| **MBPP**         | 45.80              | 28.00                   | 59.00                 | 57.60                      |152| **TruthfulQA**   | 56.09              | 56.46                   | 55.05                 | 57.69                      |153| **Winogrande**   | 62.35              | 65.35                   | 67.01                 | 70.01                      |154| **Minerva Math** | 39.80              | 9.96                    | 36.72                 | 21.46                      |155| **MATH500**      | 53.00              | 31.4                    | 45.80                 | 34.40                      |156| **AMC23**        | 29.00              | 16.4                    | 21.00                 | 11.50                      |157| **MixEval Hard** | 29.70              | 28.40                   | 43.30                 | 34.60                      |158 159## Intended Use160 161The Apriel family of models are designed for a variety of general-purpose instruction tasks, including:162 163- Question answering and information retrieval  164- Content generation and summarization  165- Code assistance and generation  166- Logical reasoning and multi-step tasks  167- Creative writing and ideation  168 169They are **not intended** for use in safety-critical applications without human oversight or in scenarios requiring guaranteed factual accuracy.170 171## Limitations172 173- **Factual accuracy:** May produce incorrect, misleading, or outdated content. Outputs should be verified before use in critical contexts.  174- **Bias:** May reflect societal, cultural, or systemic biases present in training data.  175- **Ethics:** Do not use the model to produce harmful, unlawful, or unethical content.  176- **Language:** Strongest performance is in English. Output quality may degrade in underrepresented languages.  177- **Critical use:** Not suitable for medical, legal, financial, or other high-risk applications without safeguards.178 179## Security and Responsible Use180 181**Security Responsibilities:**  182Deployers and users are strongly encouraged to align their security practices with established frameworks and regulatory guidelines such as the EU AI Act and the NIST AI Risk Management Framework (RMF).183 184**Guidelines for Deployers:**185 186- Regularly conduct robustness assessments to identify and mitigate adversarial inputs.187- Implement validation and filtering processes to prevent harmful or biased outputs.188- Continuously perform data privacy checks to guard against unintended data leaks.189- Document and communicate the model's limitations, intended usage, and known security risks to all end-users.190- Schedule periodic security reviews and updates to address emerging threats and vulnerabilities.191 192**Guidelines for Users:**193 194- Follow established security policies and usage guidelines provided by deployers.195- Protect and manage sensitive information when interacting with the model.196- Report anomalies, suspicious behavior, or unsafe outputs to deployers or developers.197- Maintain human oversight and apply judgment to mitigate potential security or ethical risks during interactions.198 199**Disclaimer:**  200Users accept responsibility for securely deploying, managing, and using this open-source LLM. The model is provided "as-is," without explicit or implied warranty regarding security or fitness for any specific application or environment.201 202## Pretraining203 204### Model205 206- **Architecture:** Transformer decoder with grouped-query attention and YARN rotary embeddings  207- **Tokens:** 4.5T  208- **Precision:** bfloat16  209- **Knowledge cutoff:** April 2024  210 211### Hardware212 213- **Compute:** 480 × H100 GPUs  214- **GPU-hours:** ~91,000 H100-hours  215 216### Software217 218- **Training stack:** [Fast-LLM](https://github.com/ServiceNow/Fast-LLM)219 220## License221 222MIT223 224## Citation225 226```bibtex227@misc{Apriel-small-language-models,  228    author = {Slam labs team},  229    title = {{Apriel - a Family of performant small language models}},  230    howpublished = {https://huggingface.co/ServiceNow-AI/Apriel-5B-Instruct},231    publisher = {SLAM - ServiceNow Language Models Lab}  232    year = {2025}233}234```235