CoolFace
Modelpublic

nbeerbower/Huihui-Qwen3.5-0.8B-abliterated-Athanorlite-ORPO

sourceHugging Faceupdated 6mo agoView on Hugging Face
0likes17downloads
Model Card

Huihui-Qwen3.5-0.8B-abliterated-Athanorlite-ORPO

Training Configuration

ParameterValue
Training ModeORPO
Base Modelhuihui-ai/Huihui-Qwen3.5-0.8B-abliterated
Learning Rate9e-05
Epochs1
Batch Size2
Gradient Accumulation16
Effective Batch Size32
Max Sequence Length2048
Optimizerpagedadamw8bit
LR Schedulercosine
Warmup Ratio0.05
Weight Decay0.01
Max Grad Norm0.3
Seed42
Beta0.1
Max Prompt Length1024
LoRA Rank (r)128
LoRA Alpha64
LoRA Dropout0.05
Target Modulesupproj, downproj, gateproj, kproj, qproj, vproj, o_proj
Quantization4-bit (NF4)
GPUNVIDIA RTX A6000

Trained with Merlina

Merlina on GitHub