mgv99/testing-slp

1---2license: mit3tags:4- merge5- mergekit6- lazymergekit7- OpenPipe/mistral-ft-optimized-12188- mlabonne/NeuralHermes-2.5-Mistral-7B9base_model:10- OpenPipe/mistral-ft-optimized-121811- mlabonne/NeuralHermes-2.5-Mistral-7B12---13 14# testing-slp15 16testing-slp is a merge of the following models:17* [OpenPipe/mistral-ft-optimized-1218](https://huggingface.co/OpenPipe/mistral-ft-optimized-1218)18* [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B)19 20## 🧩 Configuration21 22```yamlbase_model: OpenPipe/mistral-ft-optimized-121823dtype: bfloat1624merge_method: slerp25slices:26- sources:27  - layer_range:28    - 029    - 3230    model: OpenPipe/mistral-ft-optimized-121831  - layer_range:32    - 033    - 3234    model: mlabonne/NeuralHermes-2.5-Mistral-7B35parameters:36  t:37    - filter: self_attn38      value: [0, 0.5, 0.3, 0.7, 1]39    - filter: mlp40      value: [1, 0.5, 0.7, 0.3, 0]41    - value: 0.5```42 43## 💻 Usage44 45```python46!pip install -qU transformers accelerate47 48from transformers import AutoTokenizer49import transformers50import torch51 52model = "mgv99/testing-slp"53messages = [{"role": "user", "content": "What is a large language model?"}]54 55tokenizer = AutoTokenizer.from_pretrained(model)56prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)57pipeline = transformers.pipeline(58    "text-generation",59    model=model,60    torch_dtype=torch.float16,61    device_map="auto",62)63 64outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)65print(outputs[0]["generated_text"])66```