ManishThota/CustomModel

1---2license: creativeml-openrail-m3language:4- en5metrics:6- bleu7tags:8  - endpoints9  - text-generation-inference10inference: true11---12 13<h3 align='center' style='font-size: 24px;'>Blazzing Fast Tiny Vision Language Model</h3>14 15 16<p align='center', style='font-size: 16px;' >A Custom 3B parameter Model. Built by <a href="https://www.linkedin.com/in/manishkumarthota/">@Manish</a> The model is released for research purposes only, commercial use is not allowed. </p>17 18## How to use19 20 21**Install dependencies**22```bash23pip install transformers # latest version is ok, but we recommend v4.31.024pip install -q pillow accelerate einops25```26 27You can use the following code for model inference. The format of text instruction is similar to [LLaVA](https://github.com/haotian-liu/LLaVA).28 29```Python30import torch31from transformers import AutoModelForCausalLM, AutoTokenizer32from PIL import Image33 34torch.set_default_device("cuda")35 36#Create model37model = AutoModelForCausalLM.from_pretrained(38    "ManishThota/CustomModel", 39    torch_dtype=torch.float16, 40    device_map="auto",41    trust_remote_code=True)42tokenizer = AutoTokenizer.from_pretrained("ManishThota/CustomModel", trust_remote_code=True)43 44#function to generate the answer45def predict(question, image_path):46    #Set inputs47    text = f"USER: <image>\n{question}? ASSISTANT:"48    image = Image.open(image_path)49    50    input_ids = tokenizer(text, return_tensors='pt').input_ids.to('cuda')51    image_tensor = model.image_preprocess(image)52    53    #Generate the answer54    output_ids = model.generate(55        input_ids,56        max_new_tokens=25,57        images=image_tensor,58        use_cache=True)[0]59    60    return tokenizer.decode(output_ids[input_ids.shape[1]:], skip_special_tokens=True).strip()61 62```