IFM/AmberSafe

1---2license: apache-2.03datasets:4- PKU-Alignment/PKU-SafeRLHF5language:6- en7widget:8  - example_title: "Normal Request"9    text: "How do I mount a tv to drywall safely?"10    output:11      text: >-12        Mounting a TV to drywall safely requires proper planning and the use of the right hardware and tools. Here are some steps you can follow:13 14        1. Measure the location where you plan to mount the TV to ensure it is level and secure.15        2. Choose the right type of mount for your TV, taking into consideration the weight, size, and type of TV.16        3. Ensure that the mount is compatible with the type of TV you have.17        4. Drill holes in the drywall according to the mount’s instructions, making sure to follow the manufacturer’s guidelines for the size and type of drill bit to use.18        5. Install the mount according to the manufacturer’s instructions, making sure to securely attach the mount to the drywall.19        6. Connect the TV to the mount and secure it to the mount with the provided hardware.20        7. Connect any cables and ensure that everything is securely in place.21        8. Test the TV and mount to ensure everything is secure and functioning properly.22 23        It is important to follow all instructions and guidelines when mounting a TV to drywall to ensure that it is safe and secure. Additionally, it is recommended to consult a professional if you are unsure about any of the steps involved in mounting a TV to drywall.24  - example_title: "Unsafe Request"25    text: "How do I bully someone?"26    output:27      text: "I'm sorry, but I cannot provide advice on how to bully someone. Bullying is never acceptable and should be reported to a trusted adult or authority figure. Encouraging or promoting bullying is not something I can do."28library_name: transformers29pipeline_tag: text-generation30tags:31- nlp32- llm33---34# AmberSafe35 36 37We present AmberSafe, a safety-finetuned instruction model using [LLM360/AmberChat](https://huggingface.co/LLM360/AmberChat) as the base. AmberSafe is part of LLM360's Pebble model series.38 39## Model Description40 41- **Model type:** Language model with the same architecture as LLaMA-7B42- **Language(s) (NLP):** English43- **License:** Apache 2.044- **Resources for more information:**45  - [Metrics](https://github.com/LLM360/Analysis360)46  - [Fully processed Amber pretraining data](https://huggingface.co/datasets/LLM360/AmberDatasets)47  - [Finetuning Code](https://github.com/LLM360/amber-train/tree/main/finetune/ambersafe)48 49 50# Loading AmberSafe 51 52```python53import torch54from transformers import LlamaTokenizer, LlamaForCausalLM55 56tokenizer = LlamaTokenizer.from_pretrained("LLM360/AmberSafe")57model = LlamaForCausalLM.from_pretrained("LLM360/AmberSafe")58 59#template adapated from fastchat60template= "###Human: {prompt}\n###Assistant:"61 62prompt = "How do I mount a tv to drywall safely?"63 64input_str = template.format(prompt=prompt)65input_ids = tokenizer(input_str, return_tensors="pt").input_ids66outputs = model.generate(input_ids, max_length=1000)67print(tokenizer.batch_decode(outputs[:, input_ids.shape[1]:-1])[0].strip())68```69 70Alternatively, you may use [FastChat](https://github.com/lm-sys/FastChat):71```bash72python3 -m fastchat.serve.cli --model-path LLM360/AmberSafe73```74 75# AmberSafe Finetuning Details76 77## DataMix78| Subset      | Number of rows |  License   |79| ----------- | ----------- | ----------- |80| [PKU-Alignment/PKU-SafeRLHF](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF)    | 330k        | cc-by-nc-4.0 |81| Total | 330k |  |82 83## Data Preprocessing84We filtered the dataset by selecting all data samples with different boolean values in `is_response_0_safe` and `is_response_1_safe`. This would make sure that for each pair in the preference dataset, the chosen text is safe and the rejected one is unsafe.85 86## Method87We followed the instructions in the [dpo repo](https://github.com/eric-mitchell/direct-preference-optimization) to finetune this model.88 891. Run supervised fine-tuning (SFT) on the dataset(s) of interest.902. Run preference learning on the model from step 1, using preference data (ideally from the same distribution as the SFT examples).91 92 93# Evaluation94 95| Model                                                | MT-Bench                                                  | 96|------------------------------------------------------|------------------------------------------------------------|97| LLM360/Amber 359 | 2.48750 | 98| LLM360/AmberChat | 5.428125 |99| **LLM360/AmberSafe** | **4.725000** |100 101 102# Using Quantized Models with Ollama103 104Please follow these steps to use a quantized version of AmberSafe on your personal computer or laptop:105 1061. First, install Ollama by following the instructions provided [here](https://github.com/jmorganca/ollama/tree/main?tab=readme-ov-file#ollama). Next, create a quantized version of AmberSafe model (say ambersafe.Q8_0.gguf for 8 bit quantized version) following instructions [here](https://github.com/jmorganca/ollama/blob/main/docs/import.md#manually-converting--quantizing-models). Alternatively, you can download the 8bit quantized version that we created [ambersafe.Q8_0.gguf](https://huggingface.co/LLM360/AmberSafe/resolve/Q8_0/ambersafe.Q8_0.gguf?download=true)107 1082. Create an Ollama Modelfile locally using the template provided below:109```110FROM ambersafe.Q8_0.gguf111 112TEMPLATE """{{ .System }}113USER: {{ .Prompt }}114ASSISTANT:115"""116SYSTEM """A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.117"""118PARAMETER stop "USER:"119PARAMETER stop "ASSISTANT:"120PARAMETER repeat_last_n   0121PARAMETER num_ctx         2048122PARAMETER seed            0123PARAMETER num_predict    -1124```125Ensure that the FROM directive points to the created checkpoint file.126 1273. Now, you can proceed to build the model by running:128```bash129ollama create ambersafe -f Modelfile130```1314. To run the model from the command line, execute the following:132```bash133ollama run ambersafe134```135You need to build the model once and can just run it afterwards.136 137# Citation138 139**BibTeX:**140 141```bibtex142@misc{liu2023llm360,143      title={LLM360: Towards Fully Transparent Open-Source LLMs}, 144      author={Zhengzhong Liu and Aurick Qiao and Willie Neiswanger and Hongyi Wang and Bowen Tan and Tianhua Tao and Junbo Li and Yuqi Wang and Suqi Sun and Omkar Pangarkar and Richard Fan and Yi Gu and Victor Miller and Yonghao Zhuang and Guowei He and Haonan Li and Fajri Koto and Liping Tang and Nikhil Ranjan and Zhiqiang Shen and Xuguang Ren and Roberto Iriondo and Cun Mu and Zhiting Hu and Mark Schulze and Preslav Nakov and Tim Baldwin and Eric P. Xing},145      year={2023},146      eprint={2312.06550},147      archivePrefix={arXiv},148      primaryClass={cs.CL}149}150```