Modelpublic

MBZUAI/LLaVA-Phi-3-mini-4k-instruct-pretrain

sourceHugging Facemitupdated 2y agoView on Hugging Face

3likes26downloads

Model Card

![CODE](https://github.com/mbzuai-oryx/LLaVA-pp)

Phi-3-V: Extending the Visual Capabilities of LLaVA with Phi-3

Repository Overview

This repository features LLaVA v1.5 trained with the Phi-3-mini-3.8B LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding.

Training Strategy

—Only Vision-to-Language projector is trained. The rest of the model is frozen.
—Note: The repository contains only the projector weights.

Key Components

—Base Large Language Model (LLM): Phi-3-mini-4k-instruct
—Base Large Multimodal Model (LMM): LLaVA-v1.5

Training Data

—Pretraining Dataset: LCS-558K

Download It As

git lfs install
git clone https://huggingface.co/MBZUAI/LLaVA-Phi-3-mini-4k-instruct-pretrain

License

This project is available under the MIT License.

Contributions

Contributions are welcome! Please 🌟 our repository LLaVA++ if you find this model useful.