wraps/moondream-caption
10137
1---2license: apache-2.03datasets:4- wraps/flux1_dev-small5base_model: vikhyatk/moondream26pipeline_tag: image-text-to-text7library_name: transformers8---9# Moondream-Caption: Custom Small Vision Model based on Moondream210 11Moondream-Caption is a custom small vision model based on [moondream2](https://huggingface.co/vikhyatk/moondream2) by vikhyatk. It has been fine-tuned on a specific dataset to enhance its image description capabilities.12 13### Key Features:14 15- Based on the moondream2 architecture16- Fine-tuned for image caption generation17- Trained on a high-quality custom dataset18 19## Dataset20 21The dataset used for training Moondream-Caption is specifically designed for image captioning tasks. It has the following characteristics:22 23- Images generated with flux1_dev24- Highly accurate and verified descriptive captions25- Wide variety of visual content26 27 28## Usage29 30You can use Moondream-Caption for image captioning tasks by leveraging the Hugging Face Transformers library. Here's a quick example of how to generate captions for an image:31 32```python33from transformers import AutoTokenizer, AutoModelForCausalLM34from PIL import Image35 36moondream = AutoModelForCausalLM.from_pretrained(37 "wraps/moondream-caption", trust_remote_code=True38)39tokenizer = AutoTokenizer.from_pretrained("wraps/moondream-caption")40 41image = Image.open("path/to/your/image.jpg")42enc_image = moondream.encode_image(image)43caption = model.answer_question(enc_image, "Write a long caption for this image")44 45print(caption)46```47 48## Example4950 51**Output Caption**: A close-up portrait of a green alien with a large oval head, enormous black almond-shaped eyes, small nostrils, and a tiny mouth. The alien has a long, thin neck and is wearing a black t-shirt with white text that reads 'humans scare me'. The background shows a pale blue sky with soft, wispy clouds.52 53## Limitations54 55While Moondream-Caption is designed to generate accurate and relevant image captions, it may not perform optimally on images that significantly differ from the training dataset. Additionally, the model may struggle with complex or abstract images that deviate from the dataset's content. Please open an issue on the model's repository if you encounter any limitations or issues.