Modelpublic

BiliSakura/Txt2Img-MHN-VQGAN

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes19downloads

Model Card

Txt2Img-MHN-VQGAN

VQGAN model for remote sensing image generation, part of the Txt2Img-MHN framework.

Model Details

—Class: VQModel (diffusers)
—Input/Output: 3-channel RGB images (256×256)
—Latent channels: 256
—Codebook: 1024 embeddings (256-dim)
—Parameters: ~72M

Usage

python

from diffusers import VQModel
import torch

# Load model
vae = VQModel.from_pretrained(
    "BiliSakura/Txt2Img-MHN-VQGAN",
    ignore_mismatched_sizes=True
)

# Encode image to latent
image = torch.randn(1, 3, 256, 256)
with torch.no_grad():
    latent = vae.encode(image).latents  # (1, 256, 32, 32)

# Decode latent to image
with torch.no_grad():
    decoded = vae.decode(latent).sample  # (1, 3, 256, 256)

Training

Trained on the RSICD remote sensing dataset using Taming Transformers.

Citation

bibtex

@article{txt2img_mhn,
  title={Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networks},
  author={Xu, Yonghao and Yu, Weikang and Ghamisi, Pedram and Kopp, Michael and Hochreiter, Sepp},
  journal={IEEE Trans. Image Process.}, 
  doi={10.1109/TIP.2023.3323799},
  year={2023}
}

License