Modelpublic

Merlin-Research/Micro-Merlin-Experimental

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes6downloads

Model Card

Micro-Merlin-Experimental

This is a fine-tune of LiquidAI/LFM2.5-1.2B-Thinking on GPT-5.2 reasoning traces.

The model was trained with LoRA on the TeichAI/gpt-5.2-high-reasoning-250x dataset, a collection of high-reasoning-depth traces distilled from GPT-5.2, focused on production-grade DevOps, backend, and infrastructure engineering tasks. The goal is to transfer GPT-5.2's structured <think> reasoning style onto a compact 1.2B model that runs comfortably on consumer hardware.

Model & Training Details

Field	Value
Base model	LiquidAI/LFM2.5-1.2B-Thinking
Parameters	1.2B
Method	LoRA (16-bit, rank-stabilized)
Dataset	TeichAI/gpt-5.2-high-reasoning-250x
Training examples	249
Epochs	1
Total steps	~63
Final training loss	2.121
LoRA rank (r)	64
LoRA alpha	64
LoRA dropout	0
rsLoRA	Enabled
Target modules	qproj, kproj, vproj, outproj, in_proj, w1, w2, w3
Max sequence length	20,480
Batch size (effective)	4 (1 × 4 grad. accum.)
Learning rate	2e-4
LR scheduler	Cosine
Warmup steps	3
Optimizer	adamw_8bit
Weight decay	0.01
Precision	FP16
Loss masking	Responses only (`<think>` + answer)
Hardware	1× NVIDIA Tesla T4 (16 GB)
Framework	Unsloth + TRL SFTTrainer
Training runtime	~608 s (~10 min)
Chat template	ChatML (`<	im_start	> `/` <	im_end	>`)

Usage

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="OrionLLM/Micro-Merlin-Experimental",
    max_seq_length=20480,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "Design a rate limiter for a REST API."}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(**inputs, max_new_tokens=1024, temperature=0.5, repetition_penalty=1.15)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Merlin Research • 2026

Developed by DedeProGames

</div>