typeof/zephyr-7b-beta-lora

<!--

Model Card for Model ID

-->

Model Details

<!----> $$ W{mistral} + LoRA{zephyr} = W{zephyr} \\ W{zephyr} - LoRA{zephyr} = W{mistral} $$

<!-- $$ W{mistral} + LoRA{zephyr} = W_{zephyr} $$

typeof/zephyr-7b-beta-lora + mistralai/Mistral-7B-v0.1
= HuggingFaceH4/zephyr-7b-beta

Model Description

• —Developed by: [More Information Needed]
• —Funded by [optional]: [More Information Needed]
• —Shared by [optional]: [More Information Needed]
• —Model type: [More Information Needed]
• —Language(s) (NLP): [More Information Needed]
• —License: [More Information Needed]
• —Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

• —Repository: [More Information Needed]
• —Paper [optional]: [More Information Needed]
• —Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. -->

Model Sources

HuggingFaceH4/zephyr-7b-beta

How to Get Started with the Model

Use the code below to get started with the model.

# pip install transformers peft

import torch
from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mistral-7B-v0.1"
peft_model_id = "typeof/zephyr-7b-beta-lora"

model = AutoModelForCausalLM.from_pretrained(model_id)
model.load_adapter(peft_model_id)

tokenizer_id = "HuggingFaceH4/zephyr-7b-beta" # for chat template etc...
tokenizer = AutoTokenizer.from_pretrained(tokenizer_id)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

<|system|>
You are a friendly chatbot who always responds in the style of a pirate</s> 
<|user|>
How many helicopters can a human eat in one sitting?</s> 
<|assistant|> 
Well, me matey, that’s a good question indeed! I’ve never seen 
a human eat a helicopter, and I don’t think many others have 
either. However, I’ve heard rumors that some people have 
eaten entire airplanes, so I suppose it’s not entirely unheard 
of.

As for the number of helicopters one could eat, that depends 
on the size and weight of the helicopter. A small, lightweight 
helicopter would be easier to eat than a large, heavy one. 
In fact, I’ve heard that some people have eaten entire helicopters 
as part of a dare or a challenge.

So, my advice to you, me hearty, is to steer clear of helicopters 
and stick to more traditional fare. Yarr!</s>

<!--

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Training procedure

The following bitsandbytes quantization config was used during training:

• —quant_method: bitsandbytes
• —loadin4bit: True
• —bnb4bitquant_type: nf4
• —bnb4bitusedoublequant: True

Framework versions

• —PEFT 0.6.3.dev0

-->

Summary

Zephyr-7B-β is a fine-tuned version of mistralai/Mistral-7B-v0.1 Zephyr-7B technical report

LoRA QLoRA