Merlin-Research/Micro-Merlin-Experimental

1---2license: apache-2.03base_model: LiquidAI/LFM2.5-1.2B-Thinking4tags:5  - lora6  - unsloth7  - reasoning8  - distillation9  - lfm210datasets:11  - TeichAI/gpt-5.2-high-reasoning-250x12language:13  - en14pipeline_tag: text-generation15---16 17# Micro-Merlin-Experimental18 19This is a fine-tune of **LiquidAI/LFM2.5-1.2B-Thinking** on GPT-5.2 reasoning traces.20 21The model was trained with LoRA on the [TeichAI/gpt-5.2-high-reasoning-250x](https://huggingface.co/datasets/TeichAI/gpt-5.2-high-reasoning-250x) dataset, a collection of high-reasoning-depth traces distilled from GPT-5.2, focused on production-grade DevOps, backend, and infrastructure engineering tasks. The goal is to transfer GPT-5.2's structured `<think>` reasoning style onto a compact 1.2B model that runs comfortably on consumer hardware.22 23## Model & Training Details24 25| Field | Value |26|---|---|27| **Base model** | LiquidAI/LFM2.5-1.2B-Thinking |28| **Parameters** | 1.2B |29| **Method** | LoRA (16-bit, rank-stabilized) |30| **Dataset** | TeichAI/gpt-5.2-high-reasoning-250x |31| **Training examples** | 249 |32| **Epochs** | 1 |33| **Total steps** | ~63 |34| **Final training loss** | 2.121 |35| **LoRA rank (r)** | 64 |36| **LoRA alpha** | 64 |37| **LoRA dropout** | 0 |38| **rsLoRA** | Enabled |39| **Target modules** | q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3 |40| **Max sequence length** | 20,480 |41| **Batch size (effective)** | 4 (1 × 4 grad. accum.) |42| **Learning rate** | 2e-4 |43| **LR scheduler** | Cosine |44| **Warmup steps** | 3 |45| **Optimizer** | adamw_8bit |46| **Weight decay** | 0.01 |47| **Precision** | FP16 |48| **Loss masking** | Responses only (`<think>` + answer) |49| **Hardware** | 1× NVIDIA Tesla T4 (16 GB) |50| **Framework** | Unsloth + TRL SFTTrainer |51| **Training runtime** | ~608 s (~10 min) |52| **Chat template** | ChatML (`<|im_start|>` / `<|im_end|>`) |53 54## Usage55 56```python57from unsloth import FastLanguageModel58 59model, tokenizer = FastLanguageModel.from_pretrained(60    model_name="OrionLLM/Micro-Merlin-Experimental",61    max_seq_length=20480,62    load_in_4bit=False,63)64FastLanguageModel.for_inference(model)65 66messages = [{"role": "user", "content": "Design a rate limiter for a REST API."}]67inputs = tokenizer.apply_chat_template(68    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"69).to(model.device)70 71out = model.generate(**inputs, max_new_tokens=1024, temperature=0.5, repetition_penalty=1.15)72print(tokenizer.decode(out[0], skip_special_tokens=True))73```74 75---76 77<div align="center">78 79**Merlin Research • 2026**80 81Developed by [DedeProGames](https://huggingface.co/DedeProGames)82 83</div>84