CoolFace
Modelpublic

oneonlee/KoSOLAR-v0.2-gugutypus-10.7B

sourceHugging Facecc-by-nc-4.0updated 3y agoView on Hugging Face
1likes26downloads
Model Card

<div align="center"> <h1>🤗 KoSOLAR-v0.2-gugutypus-10.7B ☀️</h1>

<a style="margin: 0px;" href="https://github.com/oneonlee/KoSOLAR-v0.2-gugutypus-10.7B"><img style="margin: 0.5em;" alt="GitHub" src="https://img.shields.io/badge/GitHub-181717.svg?style=flat&logo=GitHub"></a> <a style="margin: 0px;" href="https://huggingface.co/oneonlee/KoSOLAR-v0.2-gugutypus-10.7B"><img style="margin: 0.5em;" alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97-Models%20on%20Hub-yellow"></a> <a style="margin: 0px;" href="https://github.com/oneonlee/KoSOLAR-v0.2-gugutypus-10.7B/blob/main/LICENSE"><img style="margin: 0.5em;" alt="License: CC BY-NC 4.0" src="https://img.shields.io/badge/License-CC%20BY%2D%2DNC%204.0-blue.svg"></a> <a style="margin: 0px;" href="https://doi.org/10.57967/hf/1735"><img style="margin: 0.5em;" alt="DOI" src="https://img.shields.io/badge/DOI-10.57967%2Fhf%2F1735-blue"></a>

<img src="logo.png" height=350, width=350> </div>

Model Details

Model Developers

Model Architecture

  • —KoSOLAR-v0.2-gugutypus-10.7B is a instruction fine-tuned auto-regressive language model, based on the SOLAR transformer architecture.

Base Model

Training Dataset

Model comparisons

  • —Ko-LLM leaderboard (2024/03/01) [[link]](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)
ModelAverageKo-ARCKo-HellaSwagKo-MMLUKo-TruthfulQAKo-CommonGen V2
oneonlee/KoSOLAR-v0.2-gugutypus-10.7B51.1747.7858.2947.2748.3154.19
oneonlee/LDCC-SOLAR-gugutypus-10.7B49.4545.955.4647.9648.9349

<br>

  • —(KOR) AI-Harness evaluation [[link]](https://github.com/Beomi/ko-lm-evaluation-harness)
TasksVersionFiltern-shotMetricValueStderr
KMMLUN/Anone0acc0.3335±0.0475
KMMLUN/Anone5acc0.3938±0.0823
KoBEST-HellaSwag0none0acc0.4360±0.0222
KoBEST-HellaSwag0none5acc0.4420±0.0222
KoBEST-BoolQ0none0acc0.5064±0.0133
KoBEST-BoolQ0none5acc0.8583±0.0093
KoBEST-COPA0none0acc0.6040±0.0155
KoBEST-COPA0none5acc0.7610±0.0135
KoBEST-SentiNeg0none0acc0.5844±0.0248
KoBEST-SentiNeg0none5acc0.9471±0.0112

<br>

  • —(ENG) AI-Harness evaluation [[link]](https://github.com/EleutherAI/lm-evaluation-harness)
TasksVersionFiltern-shotMetricValueStderr
MMLUN/Anone0acc0.5826±0.1432
MMLUN/Anone5acc0.5885±0.1285
HellaSwag1none0acc0.6075±0.0049
HellaSwag1none5acc0.6098±0.0049
BoolQ2none0acc0.8737±0.0058
BoolQ2none5acc0.8826±0.0056
COPA1none0acc0.8300±0.0378
COPA1none5acc0.9100±0.0288
truthfulqaN/Anone0acc0.4249±0.0023
truthfulqaN/Anone5acc-±-

How to Use

python
### KoSOLAR-gugutypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "oneonlee/KoSOLAR-v0.2-gugutypus-10.7B"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
tokenizer = AutoTokenizer.from_pretrained(repo)

Citation

@misc {donggeon_lee_2024,
	author       = { {DongGeon Lee} },
	title        = { KoSOLAR-v0.2-gugutypus-10.7B (Revision 56841d5) },
	year         = 2024,
	url          = { https://huggingface.co/oneonlee/KoSOLAR-v0.2-gugutypus-10.7B },
	doi          = { 10.57967/hf/1735 },
	publisher    = { Hugging Face }
}

References