EleutherAI/llemma_7b_muinstruct_camelmath

llemma_7b_muinstruct_camelmath is an instruction-following finetune of Llemma 7B, trained on the μInstruct and camel-ai/math datasets.

Input Formatting

Format input queries as follows:

input_text = f"Input:{input}\n\nResponse:"

Note that due to an error during training, this model's end-of-sequence token ID is 0 instead of the 2 which is standard for Llama-2 based models. Inference APIs should handle this automatically by reading this repo's config.json, but be aware of this difference if you are doing token surgery.

Evals

llemma_7b_muinstruct_camelmath compares favorably to other 7B parameter models on the Hungarian Math Exam. It surpasses the few-shot performance of Llemma 7B whilst being the strongest Llama-2 7B based model.