RichardErkhov/pinkyponky_-_SOLAR-10.7B-dpo-instruct-tuned-v0.1-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

SOLAR-10.7B-dpo-instruct-tuned-v0.1 - GGUF

• —Model creator: https://huggingface.co/pinkyponky/
• —Original model: https://huggingface.co/pinkyponky/SOLAR-10.7B-dpo-instruct-tuned-v0.1/

Original model description: --- license: cc-by-nc-4.0 ---

Description to load and test will be added soon. More details on training and data will be added aswell.

Loading the Model

Use the following Python code to load the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("pinkyponky/SOLAR-10.7B-dpo-instruct-tuned-v0.1")
model = AutoModelForCausalLM.from_pretrained(
    "Upstage/SOLAR-10.7B-v1.0",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

Generating Text

To generate text, use the following Python code:

text = "Hi, my name is "
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))