Modelpublic

DILAB-HYU/koquality-polyglot-3.8b

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face

1likes21downloads

Model Card

This model is a instruct-tuned EleutherAI/polyglot-ko-3.8b model.

Training hyperparameters

—learning_rate: 5e-5
—trainbatchsize: 1
—seed: 42
—distributed_type: multi-GPU (A30 24G) + CPU Offloading (384GB)
—num_devices: 2
—gradientaccumulationsteps: 32
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: linear
—num_epochs: 2.0

Framework versions

—Transformers 4.34.1
—Pytorch 2.0.1+cu117
—Datasets 2.11.0
—deepspeed 0.9.5