cnmoro/tiny-image-captioning

1---2license: apache-2.03language:4- en5base_model:6- WinKawaks/vit-small-patch16-2247- google/bert_uncased_L-2_H-128_A-28pipeline_tag: image-to-text9library_name: transformers10tags:11- vit12- bert13- vision14- caption15- captioning16- image17---18An image captioning model, based on bert-tiny and vit-small, weighing only 100mb!19 20Works very fast on CPU.21 22```python23from transformers import AutoTokenizer, AutoImageProcessor, VisionEncoderDecoderModel24import requests, time25from PIL import Image26 27model_path = "cnmoro/tiny-image-captioning"28 29# load the image captioning model and corresponding tokenizer and image processor30model = VisionEncoderDecoderModel.from_pretrained(model_path)31tokenizer = AutoTokenizer.from_pretrained(model_path)32image_processor = AutoImageProcessor.from_pretrained(model_path)33 34# preprocess an image35url = "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/New_york_times_square-terabass.jpg/800px-New_york_times_square-terabass.jpg"36image = Image.open(requests.get(url, stream=True).raw)37pixel_values = image_processor(image, return_tensors="pt").pixel_values38 39start = time.time()40 41# generate caption - suggested settings42generated_ids = model.generate(43    pixel_values,44    temperature=0.7,45    top_p=0.8,46    top_k=50,47    num_beams=3 # you can use 1 for even faster inference with a small drop in quality48)49generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]50 51end = time.time()52 53print(generated_text)54# a group of people walking in the middle of a city.55 56print(f"Time taken: {end - start} seconds")57# Time taken: 0.11215853691101074 seconds58# on CPU !59```