Dai-Osaka/llm-jp-3-13b-it

Uploaded model

• —Developed by: Dai-Osaka
• —License: apache-2.0
• —Finetuned from model : llm-jp/llm-jp-3-13b

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

HOW　TO　INFERENCE

以下は、ELYZA-tasks-100-TVの回答を得るためのコードです。

from transformers import ( AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, ) import torch from tqdm import tqdm import json

HF_TOKEN = "your-key"

model_name = "Dai-Osaka/llm-jp-3-13b-it"

bnbconfig = BitsAndBytesConfig( loadin4bit=True, bnb4bitquanttype="nf4", bnb4bitcomputedtype=torch.bfloat16, bnb4bitusedouble_quant=False, )

model = AutoModelForCausalLM.frompretrained( modelname, quantizationconfig=bnbconfig, devicemap="auto", token = HFTOKEN )

tokenizer = AutoTokenizer.frompretrained(modelname, trustremotecode=True, token = HF_TOKEN)

datasets = [] with open("./elyza-tasks-100-TV_0.jsonl", "r") as f: item = "" for line in f: line = line.strip() item += line if item.endswith("}"): datasets.append(json.loads(item)) item = ""

results = [] for data in tqdm(datasets):

input = data["input"]

prompt = f"""### 指示 {input} ### 回答： """

tokenizedinput = tokenizer.encode(prompt, addspecialtokens=False, returntensors="pt").to(model.device) with torch.nograd(): outputs = model.generate( tokenizedinput, maxnewtokens=100, dosample=False, repetitionpenalty=1.2 )[0] output = tokenizer.decode(outputs[tokenizedinput.size(1):], skipspecial_tokens=True)

results.append({"taskid": data["taskid"], "input": input, "output": output})