Modelpublic

yujuyeon/internvl3_5-1b-korean-lora-r32

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes12downloads

Model Card

internvl3_5-1b-korean-lora-r32

OpenGVLab/InternVL3_5-1B 를 한국어 멀티모달 데이터로 파인튜닝한 InternVL3.5 specialist.

항목	값
Base model	`OpenGVLab/InternVL3_5-1B`
Method	LoRA r32, 3 epoch
Domain	한국어 종합 347k

Hyperparameters

—numtrainepochs: 3
—steps: 65145 / max 65145
—trainbatchsize: 4
—peak learning_rate: 3.9999999382064654e-05

Training Loss

—init 1.5987 → final 0.6771 (min 0.5426)

step	loss
10	1.5987
6520	0.8878
13030	0.8402
19540	0.7112
26050	0.6731
32560	0.6954
39070	0.7273
45580	0.6619
52090	0.6392
58600	0.6870
65110	0.7508
65140	0.6771

Training Data

구성: 18개 서브셋 (한국어 specialist SFT)

subset	repeat
`aihub_visual_ShortQA_30k`	1
`hf_korLlava_Caption_20k`	1
`llava_ko_recap_30k`	1
`out_kor_llava_20k`	1
`chartRqa1_30k`	1
`chartRqa2_20k`	1
`tableVqa_Reason_20k`	1
`tableVqa_Caption_20k`	1
`aihub_subjectTxt_OCR_20k`	1
`aihub_visual_OCR_15k`	1
`kisti_arxiv_OCR_15k`	1
`kisti_hanbat_Reason_30k`	1
`kisti_documen_Reason_10k`	1
`aihub_mathMultiple_kor_M0`	1
`aihub_mathSubjective_kor_M0`	1
`kisti_hanbat_Vqa_25k`	1
`hf_latexUpdate_15k`	1
`aihub_subjectImg_Parse_10k`	1

Usage

python

from transformers import AutoModel, AutoTokenizer
import torch
m = AutoModel.from_pretrained("yujuyeon/internvl3_5-1b-korean-lora-r32", torch_dtype=torch.bfloat16,
                              trust_remote_code=True).eval().cuda()
tok = AutoTokenizer.from_pretrained("yujuyeon/internvl3_5-1b-korean-lora-r32", trust_remote_code=True, use_fast=False)