Modelpublic

golyuval/multimodal-fusion-demo

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes8downloads

Model Card

Multimodal Fusion

A multi-modal fusion model combining image and text features.

Model Details

—Model Type: multimodal-fusion
—Task: multimodal-fusion
—Framework: PyTorch Lightning → Hugging Face Transformers

Usage

python

from transformers import AutoModel
import torch

# Load model
model = AutoModel.from_pretrained("golyuval/multimodal-fusion-demo", trust_remote_code=True)
model.eval()

# Prepare inputs
image_features = torch.randn(1, 3, 224, 224)
text_features = torch.randint(0, 1000, (1, 32))

# Inference
with torch.no_grad():
    outputs = model(image_features=image_features, text_features=text_features)
    
print(outputs)

Configuration

json

{
  "image_dim": 512,
  "text_dim": 256,
  "output_dim": 3,
  "lr": 0.001,
  "training": true,
  "model_type": "multimodal-fusion",
  "auto_map": {
    "AutoConfig": "modeling_multimodal_fusion.MultimodalFusionConfig",
    "AutoModel": "modeling_multimodal_fusion.MultimodalFusionForHF",
    "AutoModelForSequenceClassification": "modeling_multimodal_fusion.MultimodalFusionForHF"
  }
}

Model Info

—Total Parameters: 115,459
—Trainable Parameters: 115,459

Preprocessing

—Image Dim: 512
—Text Dim: 256
—Output Dim: 3

Citation

bibtex

@misc{multimodal_fusion,
  title = {Multimodal Fusion},
  author = {Your Name},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/golyuval/multimodal-fusion-demo}}
}