emhaihsan/llm-evaluation-dashboard

LLM Evaluation Metrics Dashboard


A comprehensive educational dashboard demonstrating 5 key NLP evaluation metrics with interactive visualizations and mathematical explanations.

🎯 Features

5 Complete Metric Implementations

Interactive Visualizations

• —📊 Radar charts for multi-metric comparison
• —📈 Bar charts for precision/recall breakdown
• —🔥 Heatmaps for token-level similarity
• —🎚️ Gauge charts for score visualization
• —📝 Step-by-step calculation breakdowns

Educational Focus

• —🧮 Mathematical formulas with LaTeX rendering
• —🔍 N-gram matching visualizations
• —🎓 Preset examples for quick learning
• —💡 Quality interpretations for each score

🚀 Quick Start

Local Installation

# Clone the repository
git clone https://github.com/emhaihsan/llm-evaluation-dashboard.git
cd llm-evaluation-dashboard

# Install dependencies
pip install -r requirements.txt

# Download NLTK data
python -c "import nltk; nltk.download('punkt')"

# Run the app
streamlit run app.py

Online Demo

Visit the live demo: HuggingFace Spaces

📁 Project Structure

llm-evaluation-dashboard/
├── app.py                          # Main entry point
├── pages/
│   ├── 01_overview.py              # All metrics comparison
│   ├── 02_bleu_score.py            # BLEU with n-gram viz
│   ├── 03_rogue_score.py           # 5 ROGUE variants
│   ├── 04_perplexity.py            # Token-level perplexity
│   ├── 05_mrr.py                   # Ranking evaluation
│   └── 06_bert_score.py            # Semantic similarity
├── modules/
│   ├── metrics/                    # Metric implementations
│   │   ├── bleu.py
│   │   ├── rogue.py
│   │   ├── perplexity.py
│   │   ├── mrr.py
│   │   └── bert_score.py
│   ├── visualizations/             # Plotly charts
│   │   ├── charts.py
│   │   └── explanations.py
│   └── utils/                      # Helpers
│       ├── text_processing.py
│       └── examples.py
├── data/
│   └── sample_evaluations.json     # Test cases
├── requirements.txt
└── README.md

🎓 Learning Resources

BLEU Score

• —Bilingual Evaluation Understudy
• —N-gram precision (1-4 grams)
• —Brevity penalty for short outputs
• —Best for: Machine translation

ROGUE Score (5 Variants)

• —ROGUE-N: Unigram/bigram overlap
• —ROGUE-L: Longest Common Subsequence
• —ROGUE-S: Skip-bigram matching
• —ROGUE-SU: Skip + Unigram combined
• —ROGUE-W: Weighted consecutive matches
• —Best for: Text summarization

Perplexity

• —Model confidence metric
• —Lower = more confident
• —No reference needed
• —Best for: Language model evaluation

MRR (Mean Reciprocal Rank)

• —Ranking quality metric
• —1/rankofcorrect_answer
• —Batch evaluation support
• —Best for: Question answering / IR

BERT Score

• —Contextual embedding similarity
• —Semantic vs lexical matching
• —Token-level alignment
• —Best for: Paraphrase detection

💻 Usage Examples

Compare Two Translations

1. 1.Go to Overview page
2. 2.Paste reference translation
3. 3.Paste candidate translation
4. 4.Click "Compare All Metrics"
5. 5.See radar chart showing BLEU, ROGUE, and BERT Score

Evaluate Summarization Quality

1. 1.Go to ROGUE Score page
2. 2.Select "ROGUE-L" variant
3. 3.Enter original article as reference
4. 4.Enter summary as candidate
5. 5.Check LCS (Longest Common Subsequence) visualization

Test Question Answering

1. 1.Go to MRR page
2. 2.Enter question and correct answer
3. 3.Enter ranked model outputs
4. 4.See reciprocal rank calculation

🛠️ Tech Stack

• —Framework: Streamlit
• —Visualizations: Plotly, Matplotlib
• —NLP: NLTK
• —Metrics: rouge-score, bert-score, sentence-transformers
• —Math: NumPy, Pandas

📝 Implementation Notes

Educational Simplifications

• —Perplexity: Uses probability simulation (real calculation requires model logprobs)
• —BERT Score: Uses simplified embeddings (install sentence-transformers for full BERT)
• —Tokenization: Simple word-based (no subword tokenization)