mailtotanvir/nano-rl-inference-arithmetic-sft-adapter

Mini AI Stack: Arithmetic SFT Adapter

This is the selected LoRA SFT adapter from the Mini AI Stack post-training and inference-systems study. It is a research artifact, not a general-purpose reasoning model and not a claim of arithmetic capability.

The project built a compact chain from frozen synthetic task families through five post-training paths, vLLM and SGLang serving, and an OpenAI-compatible provider boundary. SFT is published because it is the simplest reproducible cold-start artifact. In the final three-seed campaign, every tested method label produced the same strict held-out result, so the adapter is intentionally not presented as a winning RL checkpoint.

What this adapter is

• —Base model: Qwen/Qwen2.5-0.5B-Instruct
• —Format: PEFT LoRA adapter, not merged base-model weights
• —Target modules: q_proj, v_proj
• —LoRA rank / alpha / dropout: 8 / 16 / 0.05
• —Training: one epoch, batch size 4, gradient accumulation 4, learning rate 2e-4
• —Task: synthetic arithmetic prompts with exact-match numeric reward and evaluation
• —Adapter SHA-256: 67fff6805232bda488518b03b95d63bdcc908473c71b1579219cd2f943362db7

The evidence boundary

Training examples came from addition and subtraction template families. The frozen held-out set contained 128 division-template prompts. At seeds 13, 42, and 97, this adapter scored 4/128 on that held-out family.

The result does not establish that these algorithms are generally equivalent. It establishes that, in this bounded environment, reward-bearing training did not cross the withheld template family.

Intended use

Use this adapter only for:

• —reproducing the repository's adapter-loading and evaluation paths;
• —inspecting PEFT LoRA artifacts in a constrained post-training study;
• —educational work on the difference between training reward and held-out transfer.

Do not use it for arithmetic decision-making, factual assistance, safety-sensitive workflows, or as evidence of general reasoning ability.

Load with PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen2.5-0.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(model, "mailtotanvir/nano-rl-inference-arithmetic-sft-adapter")
tokenizer = AutoTokenizer.from_pretrained(
    "mailtotanvir/nano-rl-inference-arithmetic-sft-adapter"
)

Files

• —adapter_model.safetensors: LoRA adapter weights
• —adapter_config.json: PEFT configuration
• —tokenizer files: tokenizer state needed by the study
• —SHA256SUMS: adapter integrity record

Checkpoint optimizer state, training arguments, source data, and credentials are deliberately not published with this model artifact.

Read the system around the adapter

• —Repository: https://github.com/mailtotanvir/Nano-RL-Inference
• —Project article: https://mailtotanvir.github.io/Nano-RL-Inference/blog/blog.html
• —Technical report: https://mailtotanvir.github.io/Nano-RL-Inference/papers/integrated-study/paper.pdf
• —Zenodo-hosted PDF: https://zenodo.org/records/22886166/files/paper.pdf?download=1
• —GitHub release: https://github.com/mailtotanvir/Nano-RL-Inference/releases/tag/v0.1.0
• —Zenodo concept DOI: https://doi.org/10.5281/zenodo.22886076
• —Zenodo version DOI for paper-only v0.1.1: https://doi.org/10.5281/zenodo.22886166

Citation

Concept DOI: 10.5281/zenodo.22886076. Paper-only version v0.1.1 DOI: 10.5281/zenodo.22886166.

@software{ahmed_2026_mini_ai_stack,
  author = {Ahmed, Tanvir},
  title = {Building a Miniature AI Factory: Post-Training, Inference Engines, and Provider Systems},
  year = {2026},
  version = {0.1.1},
  doi = {10.5281/zenodo.22886166},
  url = {https://doi.org/10.5281/zenodo.22886076}
}

License

Apache-2.0. The base model remains subject to its own upstream license and terms.