CoolFace
Modelpublic

Satoru-KRC/llm-jp-3-13b-finetune

sourceHugging Faceupdated 2y agoView on Hugging Face
0likes
Model Card

Created based on the code provided in LoRA_template_20241127.ipynb from the [Large Language Model Course FALL 2024](https://weblab.t.u-tokyo.ac.jp/lecture/course-list/large-language-model/).

Model Card for Model ID: llm-jp-3-13b-finetune

<!-- Provide a quick summary of what the model is/does. --> This is a Japanese language model fine-tuned on a specific dataset to respond to the Japanese benchmark ELYZA-Task-100-TV.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

  • —Developed by: Satoru
  • —Model type: Transformer
  • —Language(s) (NLP): 日本語
  • —License: CC-BY-NC-SA 4.0
  • —Finetuned from model [optional]: llm-jp/llm-jp-3-13b

Model Sources [optional]

<!-- Provide the basic links for the model. -->

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

There are biases, risks, and limitations that may be inherent in LLMs.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

Model loading

# python 3.10.12
!pip install -U pip
!pip install -U transformers
!pip install -U bitsandbytes
!pip install -U accelerate
!pip install -U datasets
!pip install -U peft
!pip install -U trl
!pip install -U wandb
!pip install ipywidgets --upgrade

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    TrainingArguments,
    logging,
)
from peft import (
    LoraConfig,
    PeftModel,
    get_peft_model,
)
import os, torch, gc
from datasets import load_dataset
import bitsandbytes as bnb
from trl import SFTTrainer

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)

Evaluation

from tqdm import tqdm

results = []
for data in tqdm(datasets):

  input = data["input"]

  prompt = f"""### 指示
  {input}
  ### 回答
  """

  tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
  attention_mask = torch.ones_like(tokenized_input)

  with torch.no_grad():
      outputs = model.generate(
          tokenized_input,
          attention_mask=attention_mask,
          max_new_tokens=100,
          do_sample=False,
          repetition_penalty=1.2,
          pad_token_id=tokenizer.eos_token_id
      )[0]
  output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)

  results.append({"task_id": data["task_id"], "input": input, "output": output})

Model Card Contact