qvx-o/Qlip

Qlip

Qlip is a novel text encoder architecture designed for text-to-image (Qanvas) generation tasks sharing similar objectives with CLIP, Qlip is optimized for maximum quality while taking little compute power.

Model Details

• —Model Type: Text Encoder
• —Architecture: Custom Transformer-based encoder
• —Parameters: ~50M
• —Vocab Size: 48,000 tokens
• —Embedding Dimension: 512
• —Max Sequence Length: 256 tokens
• —Hidden Size: 512
• —Layers: 8 transformer encoder layers
• —Attention Heads: 8
• —FFN Size: 2048

Installation

pip install torch sentencepiece

Usage

Basic Text Encoding

from Qlip import load_text_encoder, get_tokenizer, encode_text

# Load model and tokenizer
model = load_text_encoder(device='cuda')
tokenizer = get_tokenizer()

# Encode text
embeddings = encode_text("a beautiful sunset over the ocean", device='cuda')
print(embeddings.shape)  # torch.Size([1, 512])

Compute Text Similarity

from Qlip import encode_text, similarity

emb1 = encode_text("a cat sitting on a mat", device='cuda')
emb2 = encode_text("a dog lying on a rug", device='cuda')

sim = similarity(emb1, emb2)
print(f"Similarity: {sim:.2f}%")

Intended Use

Qlip is designed for:

• —Text-to-image generation (as the text encoder component)
• —Text-image similarity computation
• —Feature extraction for downstream tasks
• —Semantic search and retrieval

Limitations

• —Maximum sequence length of 256 tokens
• —English-only
• —Trained specifically for image generation tasks
• —May not generalize well to other text encoding tasks

License

MIT License