nvidia/AceInstruct-7B

Introduction

We introduce AceInstruct, a family of advanced SFT models for coding, mathematics, and general-purpose tasks. The AceInstruct family, which includes AceInstruct-1.5B, 7B, and 72B, is <b>Improved using Qwen</b>. These models are fine-tuned on Qwen2.5-Base using general SFT datasets. These same datasets are also used in the training of AceMath-Instruct. Different from AceMath-Instruct which is specialized for math questions, AceInstruct is versatile and can be applied to a wide range of domains. Benchmark evaluations across coding, mathematics, and general knowledge tasks demonstrate that AceInstruct delivers performance comparable to Qwen2.5-Instruct.

For more information about AceInstruct, check our website and paper.

Benchmark Results

We compare AceInstruct to Qwen2.5-Instruct across coding, mathematics, and general knowledge tasks. We find that AceInstruct-1.5B outperforms Qwen2.5-1.5B-Instruct (61.94 vs. 57.33), while AceInstruct-7B and AceInstruct-72B perform similarly to Qwen2.5-7B-Instruct and Qwen2.5-72B-Instruct.

All Resources

AceMath Instruction Models

• —AceMath-1.5B-Instruct, AceMath-7B-Instruct, AceMath-72B-Instruct

AceMath Reward Models

• —AceMath-7B-RM, AceMath-72B-RM

Evaluation & Training Data

• —AceMath-RewardBench, AceMath-Instruct Training Data, AceMath-RM Training Data

General Instruction Models

• —AceInstruct-1.5B, AceInstruct-7B, AceInstruct-72B

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "AceInstruct-7B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

prompt = "Tell me something about artificial intelligence."
messages = [{"role": "user", "content": prompt}]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to("cuda")

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=1024
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Correspondence to

Zihan Liu (zihanl@nvidia.com), Yang Chen (yachen@nvidia.com), Wei Ping (wping@nvidia.com)

Citation

If you find our work helpful, we’d appreciate it if you could cite us. <pre> @article{acemath2024, title={AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling}, author={Liu, Zihan and Chen, Yang and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei}, journal={arXiv preprint}, year={2024} } </pre>

License

All models in the AceInstruct family are for non-commercial use only, subject to Terms of Use of the data generated by OpenAI. We put the AceInstruct models under the license of Creative Commons Attribution: Non-Commercial 4.0 International.