dispatchAI/tokenizer-visualizer
0
🔤 dispatchAI Tokenizer Visualizer
MCP Server — visualize how tokenizers split text.
Tools
tokenize_text(text, method)— Tokenize with chosen methodcompare_methods(text)— Compare token counts across methods
MCP Server — visualize how tokenizers split text.
tokenize_text(text, method) — Tokenize with chosen methodcompare_methods(text) — Compare token counts across methods