RichardErkhov/AIdenU_-_LLAMA-2-13b-koen-Y24_v1.0-gguf
0546
Quantization made by Richard Erkhov.
LLAMA-2-13b-koen-Y24_v1.0 - GGUF
- Model creator: https://huggingface.co/AIdenU/
- Original model: https://huggingface.co/AIdenU/LLAMA-2-13b-koen-Y24_v1.0/
Original model description: --- license: apache-2.0 language:
- ko pipeline_tag: text-generation tags:
- llama2 ---
BaseModel
Model Generation
from transforemrs import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("AIdenU/LLAMA-2-13b-koen-Y24_v1.0", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("AIdenU/LLAMA-2-13b-koen-Y24_v1.0", use_fast=True)
systemPrompt = "당신은 유능한 AI입니다."
prompt = "지렁이도 밟으면 꿈틀하나요?"
outputs = model.generate(
**tokenizer(
f"[INST] <<SYS>>\n{systemPrompt}\n<</SYS>>\n\n{prompt} [/INST] ",
return_tensors='pt'
).to('cuda'),
max_new_tokens=256,
temperature=0.2,
top_p=1,
do_sample=True
)
print(tokenizer.decode(outputs[0]))