CoolFace
Modelpublic

nbeerbower/Huihui-Qwen3.5-9B-abliterated-Grimoire-SFT

sourceHugging Faceupdated 6mo agoView on Hugging Face
1likes25downloads
Model Card

Huihui-Qwen3.5-9B-abliterated-Grimoire-SFT

Testing grimore's SFT implementation

Training Configuration

ParameterValue
Training ModeSFT
Base Modelhuihui-ai/Huihui-Qwen3.5-9B-abliterated
Learning Rate9e-05
Epochs1
Batch Size2
Gradient Accumulation16
Effective Batch Size32
Max Sequence Length2048
Optimizerpagedadamw8bit
LR Schedulercosine
Warmup Ratio0.05
Weight Decay0.01
Max Grad Norm0.25
Seed42
LoRA Rank (r)128
LoRA Alpha64
LoRA Dropout0.05
Target Modulesupproj, downproj, gateproj, kproj, qproj, vproj, o_proj
Quantization4-bit (NF4)
GPUNVIDIA RTX A6000

Trained with Merlina

Merlina on GitHub