bitext/Mistral-7B-Restaurants

1---2license: apache-2.03inference: false4tags:5- generated_from_trainer6- text-generation-inference7model-index:8- name: Mistral-7B-Restaurants9  results: []10model_type: mistral11pipeline_tag: text-generation12widget:13- messages:14  - role: user15    content: id like to check ur menu16---17 18# Mistral-7B-Restaurants19 20## Model Description21 22This model, "Mistral-7B-Restaurants", is a fine-tuned version of the [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), specifically tailored for the Restaurants domain. It is optimized to answer questions and assist users with various Restaurants-related procedures. It has been trained using hybrid synthetic data generated using our NLP/NLG technology and our automated Data Labeling (DAL) tools.23 24The goal of this model is to show that a generic verticalized model makes customization for a final use case much easier. An overview of this approach can be found at: [From General-Purpose LLMs to Verticalized Enterprise Models](https://www.bitext.com/blog/general-purpose-models-verticalized-enterprise-genai/)25 26## Intended Use27 28- **Recommended applications**: This model is designed to be used as the first step in Bitext’s two-step approach to LLM fine-tuning for the creation of chatbots, virtual assistants and copilots for the Restaurants domain, providing customers with fast and accurate answers about their needs.29- **Out-of-scope**: This model is not suited for non-restaurants related questions and should not be used for providing health, legal, or critical safety advice.30 31## Usage Example32 33```python34from transformers import AutoModelForCausalLM, AutoTokenizer35import torch36 37device = 'cuda' if torch.cuda.is_available() else 'cpu'38 39model = AutoModelForCausalLM.from_pretrained("bitext/Mistral-7B-Restaurants")40tokenizer = AutoTokenizer.from_pretrained("bitext/Mistral-7B-Restaurants")41 42messages = [43    {"role": "system", "content": "You are an expert in customer support for Restaurants."},44    {"role": "user", "content": "id like to check ur menu"},45]46 47encoded = tokenizer.apply_chat_template(messages, return_tensors="pt")48 49model_inputs = encoded.to(device)50model.to(device)51 52generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)53decoded = tokenizer.batch_decode(generated_ids)54print(decoded[0])55```56 57## Model Architecture58 59This model utilizes the `MistralForCausalLM` architecture with a `LlamaTokenizer`, ensuring it retains the foundational capabilities of the base model while being specifically enhanced for restaurants-related interactions.60 61## Training Data62 63The model was fine-tuned on the [Bitext Restaurants Dataset](https://huggingface.co/datasets/bitext/Bitext-restaurants-llm-chatbot-training-dataset) comprising various restaurants-related intents, including: find_restaurant, check_menu, make_reservation, order_food_online, events, and more. Totaling 30 intents, and each intent is represented by approximately 1000 examples.64 65This comprehensive training helps the model address a broad spectrum of restaurants-related questions effectively. The dataset follows the same structured approach as our dataset published on Hugging Face as [bitext/Bitext-customer-support-llm-chatbot-training-dataset](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset), but with a focus on the restaurants domain.66 67## Training Procedure68 69### Hyperparameters70 71- **Optimizer**: AdamW72- **Learning Rate**: 0.0002 with a cosine learning rate scheduler73- **Epochs**: 374- **Batch Size**: 475- **Gradient Accumulation Steps**: 476- **Maximum Sequence Length**: 8192 tokens77 78### Environment79 80- **Transformers Version**: 4.43.481- **Framework**: PyTorch 2.3.1+cu12182- **Tokenizers**: Tokenizers 0.19.183 84## Limitations and Bias85 86- The model is trained for restaurants-specific contexts but may underperform in unrelated areas.87- Potential biases in the training data could affect the neutrality of the responses; users are encouraged to evaluate responses critically.88 89## Ethical Considerations90 91It is important to use this technology thoughtfully, ensuring it does not substitute for human judgment where necessary, especially in sensitive situations.92 93## Acknowledgments94 95This model was developed and trained by Bitext using proprietary data and technology.96 97## License98 99This model, "Mistral-7B-Restaurants", is licensed under the Apache License 2.0 by Bitext Innovations International, Inc. This open-source license allows for free use, modification, and distribution of the model but requires that proper credit be given to Bitext.100 101### Key Points of the Apache 2.0 License102 103- **Permissibility**: Users are allowed to use, modify, and distribute this software freely.104- **Attribution**: You must provide proper credit to Bitext Innovations International, Inc. when using this model, in accordance with the original copyright notices and the license.105- **Patent Grant**: The license includes a grant of patent rights from the contributors of the model.106- **No Warranty**: The model is provided "as is" without warranties of any kind.107 108You may view the full license text at [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0).109 110This licensing ensures the model can be used widely and freely while respecting the intellectual contributions of Bitext. For more detailed information or specific legal questions about using this license, please refer to the official license documentation linked above.111