schogini/embed-and-pos-encode-code

Embedding Dimension Visualizer

An Embedding Dimension Visualizer is an interactive Streamlit tool designed for teaching and experimentation with modern transformer embeddings. It lets you:

• —Tokenize any input text using tiktoken or HuggingFace’s BPE tokenizer, showing each subword token and its ID.
• —Visualize embeddings by generating a demo embedding vector for every token.
• —Compute and display sinusoidal positional encodings (sin / cos) per token position.
• —Combine embeddings + positional encodings and present the final per-token vectors exactly as they’d be fed into attention heads.
• —Expose theory via an expandable section—complete with LaTeX formulas—covering tokenization, BPE, and the positional-encoding equations.
• —Lock sliders into read-only mode, so learners can observe values without accidentally altering them.

This app is ideal for workshops, live demos, or self-study when you want a hands-on, visual understanding of how embeddings and positional information come together inside a transformer model.