werty1248/Llama-3-Ko-8B-OpenOrca

Llama-3-Ko-OpenOrca

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

Original model: beomi/Llama-3-Open-Ko-8B (2024.04.24 버전)

Dataset: kyujinpy/OpenOrca-KO

Training details

Training: Axolotl을 이용해 LoRA-8bit로 4epoch 학습 시켰습니다.

• —sequence_len: 4096
• —bf16

학습 시간: A6000x2, 6시간

Evaluation

• —0 shot kobest

• —5 shot kobest

License:

https://llama.meta.com/llama3/license