goavinash5/Gradio_LLAMA_Testing

1# News2- [2023/09] The newest `llama2-wrapper>=0.1.14` supports llama.cpp's `gguf` models.3 4- [2023/08] 🔥 For developers, we offer a web server that acts as a drop-in replacement for the OpenAI API.5 6  - Usage: 7 8    ```9    python3 -m llama2_wrapper.server10    ```11 12 13 14- [2023/08] 🔥 For developers, we released `llama2-wrapper`  as a llama2 backend wrapper in [PYPI](https://pypi.org/project/llama2-wrapper/).15 16  - Install: `pip install llama2-wrapper`17 18  - Usage: 19 20    ```python21    from llama2_wrapper import LLAMA2_WRAPPER, get_prompt 22    llama2_wrapper = LLAMA2_WRAPPER(23        model_path="./models/Llama-2-7B-Chat-GGML/llama-2-7b-chat.ggmlv3.q4_0.bin",24        backend_type="llama.cpp", #options: llama.cpp, transformers, gptq25    )26    prompt = "Do you know Pytorch"27    llama2_promt = get_prompt(prompt)28    answer = llama2_wrapper(llama2_promt, temperature=0.9)29    ```30 31- [2023/08] 🔥 We added `benchmark.py` for users to benchmark llama2 models on their local devices.32 33  - Check/contribute the performance of your device in the full [performance doc](https://github.com/liltom-eth/llama2-webui/blob/main/docs/performance.md).34 35- [2023/07] We released **[llama2-webui](https://github.com/liltom-eth/llama2-webui)**, a gradio web UI to run Llama 2 on GPU or CPU from anywhere (Linux/Windows/Mac). 36 37  - Supporting models: [Llama-2-7b](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)/[13b](https://huggingface.co/llamaste/Llama-2-13b-chat-hf)/[70b](https://huggingface.co/llamaste/Llama-2-70b-chat-hf), all [Llama-2-GPTQ](https://huggingface.co/TheBloke/Llama-2-7b-Chat-GPTQ), all [Llama-2-GGML](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML) ...38  - Supporting model backends:  [tranformers](https://github.com/huggingface/transformers), [bitsandbytes(8-bit inference)](https://github.com/TimDettmers/bitsandbytes), [AutoGPTQ(4-bit inference)](https://github.com/PanQiWei/AutoGPTQ), [llama.cpp](https://github.com/ggerganov/llama.cpp)