jnalv/embedding-fingerprints

🔍 Embedding Fingerprint Explorer

Turn word embeddings into visual "fingerprints," and explore semantic relationships. Each embedding dimension becomes a pixel — similar words create similar patterns!

What is this?

This is a fun utility meant to educate people about embeddings!

Word embeddings capture semantic meaning as high-dimensional vectors (typically 384-1024 dimensions). This tool converts those abstract vectors into visual representations, turning each dimension into a pixel in a square, or a line on a heatmap.

This demonstrates a few interesting concepts:

• —Words with high similarity scores share similar visual patterns.
• —And dissimilar words look obviously different.
• —Embeddings with more dimensions capture more semantic nuance, like the resolution of an image! The metaphor becomes literal!! 😎

Features

🌈 Linear Heatmap

Compare embedding dimensions across multiple words. Each column is a word, each row is a dimension. Colors show activation levels (red=high, blue=low).

🖼️ Square "Fingerprint"

Each word's embedding is reshaped into a square grid where each pixel represents one dimension. Similar words produce visually similar patterns - like fingerprints for meaning!

🔬 Comparison Matrix

Calculate and visualize cosine similarity scores between all word pairs. Values range from 0.0 (unrelated) to 1.0 (identical).

How to Use

1. 1.Select an embedding model from the dropdown (default: all-MiniLM-L6-v2)
2. 2.Enter words separated by commas or newlines:
3. 3.happy, joyful, sad, angry
4. 4.Or one word per line
5. 5.Click a visualization button to generate:
6. 6.Linear: Works with 1+ words
7. 7.Square: Works with 1+ words
8. 8.Compare: Requires 2+ words
9. 9.Try the quick examples for instant exploration!

Example Word Sets

• —👯 Synonyms: huge, enormous, big
• —☯️ Closeness of antonyms: cold, warm
• —⚡ Dissimilarity: dog, metacarpus
• —↔️ Polysemous averaging: dog, bark, tree

Available Models

Models are downloaded from HuggingFace Hub on first use and cached automatically.

Tips

• —Best results: Use 2-10 words for comparison
• —Semantic relationships: Try related words (cat, kitten), contrasting concepts (hot, cold), or try for complete non sequiturs (unicycle, Wyomingite)
• —Model selection: Smaller models are faster; larger models capture more nuance
• —First load: The first time you select a model, it may take 5-30 seconds to download

⚠️ Please Keep In Mind:

• —Dimension interpretation: Individual pixels have no semantic meaning
• —Spatial arrangement: Pixel position in grid is arbitrary
• —Similarity is relative: "Similar" depends on model's training data
• —Cultural bias: Models reflect biases in training data, eg. "doctor" being more similar to "male" than "female"

Key Design Decisions

Why These Models?

1. 1.all-MiniLM-L6-v2: Fast, lightweight, good general performance (default)
2. 2.all-mpnet-base-v2: Balanced accuracy/speed, strong sentence understanding
3. 3.bge-large-en-v1.5: State-of-the-art accuracy, larger dimensions (1024)
4. 4.bge-small-en-v1.5: Fast alternative to BGE-large
5. 5.paraphrase-multilingual: Supports 50+ languages

Why No 2D Projection?

Originally included t-SNE/PCA visualizations, but removed because:

• —Requires 30+ words to show meaningful clusters
• —Primary use case is 2-10 word comparisons
• —Better suited as separate "Semantic Space Explorer" project (coming soon, maybe? 😎)
• —Fingerprints and similarity matrices work better for small sets

Why Percentile Normalization?

After testing, percentile clipping (2nd-98th) provides best balance:

• —Reduces impact of extreme outlier dimensions
• —Preserves relative differences in mid-range
• —Creates crisper visual patterns than linear normalization
• —Less aggressive than histogram equalization

Why Matplotlib → PIL → Gradio?

Gradio's native matplotlib support is limited. Converting to PIL Images:

• —Ensures consistent rendering across browsers
• —Enables right-click save functionality
• —Provides better resolution control (150 DPI)
• —Avoids interactive plot complications

Technical Details

• —Built with Gradio
• —Uses sentence-transformers for embeddings
• —Visualizations created with matplotlib and seaborn
• —Similarity calculations via scikit-learn

License

MIT License - feel free to use, modify, and share!

Acknowledgments

• —Sentence-Transformers library by UKP Lab
• —HuggingFace for model hosting
• —Gradio for the UI framework
• —All embedding model creators