RichardErkhov/NotASI_-_Qwen2-0.5B-Math-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Qwen2-0.5B-Math - GGUF

• —Model creator: https://huggingface.co/NotASI/
• —Original model: https://huggingface.co/NotASI/Qwen2-0.5B-Math/

Original model description: --- language:

• —en license: apache-2.0 tags:
• —text-generation-inference
• —transformers
• —unsloth
• —qwen2
• —trl
• —sft base_model: unsloth/Qwen2-0.5B-Instruct-bnb-4bit datasets:
• —microsoft/orca-math-word-problems-200k ---

Coding model comming soon!

Uploaded model

• —Developed by: NotAiLOL
• —License: apache-2.0
• —Finetuned from model : unsloth/Qwen2-0.5B-Instruct-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

Details

This model was trained on microsoft/orca-math-word-problems-200k for 3 epochs with rsLoRA + QLoRA.

Training Loss Graph

The model follows the Alpaca format:

<|im_start|>system
You are a professional mathematician.|im_end|>

<|im_start|>user
{}<|im_end|>

<|im_start|>assistant
{}