shareit/cycleinstruct-phi4-supervisor

1---2library_name: transformers3license: mit4base_model: microsoft/Phi-4-reasoning5tags:6  - customer-service7  - supervisor8  - cycleinstruct9  - lg-electronics10  - phi11  - fine-tuned12language:13  - ko14  - en15  - de16  - fr17  - es18pipeline_tag: text-generation19---20 21# cycleinstruct-phi4-supervisor22 23Fully merged **microsoft/Phi-4-reasoning** (14.66 B) fine-tuned in two24stages for the LG-Electronics customer-service **quality-supervisor** task.25Given a `(Category, Conversation Transcript, Retrieved Document)` triplet,26the model emits27 28```29<think>30[Query-Document Alignment] …31[Response-Document Consistency] …32[Response Completeness] …33</think>34{"label": "correct" | "incorrect", "reason": "…"}35```36 37This repo contains a **single-file, ready-to-use** checkpoint — no adapter38merging required at load time.39 40## Training pipeline (CycleInstruct-motivated, two-stage SFT)41 42Following the [CycleInstruct paper](https://arxiv.org/abs/2508.09551)43(EMNLP 2025) as the augmentation strategy motivator:44 451. **Stage 1 — CS-chatbot SFT** on 9,868 natural `(question, answer)`46   pairs built from LG feedback + general-inquiry data. LoRA r=16 α=32,47   Muon @ lr=2e-3, seed=1337, 8 epochs.482. **Stage 2 — Supervisor SFT** on 3,771 human-annotated supervisor49   judgements. Stage-1 LoRA is merged into the base first, then a fresh50   LoRA r=16 α=32 is added and trained with Muon @ lr=1e-3, seed=42,51   7 epochs on 4,096-token sequences.52 53The uploaded checkpoint is the result of merging **both** LoRA stages into54the base weights and re-saving with `save_pretrained`.55 56## Metrics — 199-item held-out supervisor test set (T=0, `max_new_tokens=1200`)57 58| Metric | Stage-1 only | **This model (full merged)** |59|---|---|---|60| Parse-fail rate | 95.98 % | **0.00 %** |61| Accuracy | 1.01 % | **68.84 %** |62| Macro-F1 | 0.033 | **0.615** |63| chrF | 6.55 | **40.92** |64| ROUGE-L | 0.062 | **0.885** |65| BLEU-4 | 0.37 | **22.41** |66| BERTScore-F1 | 0.826 | **0.901** |67| SBERT-cos (multi-mpnet) | 0.437 | **0.830** |68 69Per-class:70 71| Class | Precision | Recall | F1 | Support |72|---|---|---|---|---|73| correct   | 0.417 | 0.481 | 0.446 | 52 |74| incorrect | 0.806 | 0.762 | 0.783 | 147 |75 76## Loading77 78```python79from transformers import AutoTokenizer, AutoModelForCausalLM80import torch81 82REPO = "shareit/cycleinstruct-phi4-supervisor"83 84tok   = AutoTokenizer.from_pretrained(REPO)85model = AutoModelForCausalLM.from_pretrained(86    REPO, torch_dtype=torch.bfloat16,87    attn_implementation="sdpa", device_map="auto").eval()88 89SYSTEM = "당신은 전자제품 CS 챗봇의 품질을 평가하는 수퍼바이저입니다."90USER   = "[Category] W/M\n[Conversation Transcript] …\n[Retrieved Document] …"91 92# Phi-4-reasoning ChatML with our clean system prompt (skip default Thought scaffold)93prompt = (94    f"<|im_start|>system<|im_sep|>{SYSTEM}<|im_end|>"95    f"<|im_start|>user<|im_sep|>{USER}<|im_end|>"96    f"<|im_start|>assistant<|im_sep|>"97)98out = model.generate(99    **tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device),100    do_sample=False, max_new_tokens=1200,101    pad_token_id=tok.pad_token_id,102)103print(tok.decode(out[0], skip_special_tokens=False))104```105 106`max_new_tokens=1200` matters — the `<think>` block usually consumes107500-900 tokens before the final JSON verdict.108 109## Training details (stage 2, on top of stage-1-merged base)110 111- **PEFT**: LoRA r=16, α=32, dropout 0.05, `target_modules=all-linear`, bias='none'112- **Optimizer**: Muon on 2D matrices (Newton-Schulz orthogonalisation) + AdamW on 1D params113- **LR**: 1e-3 (matrix) / 1e-4 (aux), cosine decay with 3 % warmup, grad-clip 1.0114- **Batch**: per-device 1 × grad-accum 16 (effective 16)115- **Seq len**: 4096 (user text char-clipped if exceeds; assistant always preserved)116- **Seed**: 42, **Epochs**: 7117- **Attention**: SDPA (bf16 native on H200)118- **Wall clock**: 5h48m on a half-H200 (48 GB active)119 120## Data121 122- Stage-1 train: 9,868 `(q, a)` pairs from `data/processed/train_pairs.jsonl`123  (multilingual, mostly English, ~50 % English, ~15 % German, then FR/ES/IT/JA/ZH…)124- Stage-2 train: 3,771 supervisor-annotated rows125  `{"conversations": [{"from":"system", …}, {"from":"user", …}, {"from":"assistant", …}]}`126  with the assistant response being a `<think>…</think>{"label":…,"reason":…}` judgement.127- Test: 199 held-out supervisor rows (unseen during either stage).128 129## Intended use / limitations130 131- Intended for research reproduction of CycleInstruct-style continuation132  training on labeled downstream tasks.133- The `correct` class has substantially lower F1 (0.446) than `incorrect`134  (0.783), reflecting the 39/61 % class imbalance in the training data.135  Class-weighted loss or balanced sampling would likely help.136- The `<think>` reasoning is Korean; input transcripts may be any language.137 138## License139 140MIT (inherits from the `microsoft/Phi-4-reasoning` base model).141