CoolFace
Apppublic

magibu/mteb-turkish

sourceHugging Facecc-by-nd-4.0updated 8mo agoView on Hugging Face
8likes
App README

🇹🇷 MTEB Turkish - Benchmark Results Explorer

Interactive visualization platform for TR-MTEB (Turkish Massive Text Embedding Benchmark) results.

📊 Features

  • —Interactive Model Comparison: Compare performance of different Turkish sentence embedding models
  • —Task-Based Analysis: View results organized by task types (Retrieval, Classification, STS, etc.)
  • —Fair Scoring Algorithm: Models with missing task scores are penalized using task averages
  • —Responsive Design: Works on desktop and mobile devices
  • —Real-time Filtering: Search and filter models and tasks

🎯 Benchmark Tasks

The platform visualizes results from various MTEB tasks including:

  • —Retrieval: QuoraRetrievalTR, MSMarcoTRRetrieval, XQuADRetrieval, etc.
  • —Classification: Turkish News Classification, Sentiment Analysis, etc.
  • —STS (Semantic Textual Similarity): STSbTR
  • —Clustering: Turkish Abstract Corpus Clustering
  • —Pair Classification: SnliTr, MnliTr, XNLI
  • —Bitext Mining: WMT16BitextMining

🚀 Technology Stack

  • —Frontend: React + Vite
  • —Styling: CSS3 with modern features
  • —Deployment: HuggingFace Spaces (Static SDK)

📖 Scoring Algorithm

The platform uses a fair scoring algorithm:

  1. 1.Each task's average score is calculated across all models
  2. 2.When a model has a score for a task → uses its actual score
  3. 3.When a model lacks a score for a task → uses the task's average score
  4. 4.Tasks with no model scores are excluded from calculations

This ensures models with missing task evaluations cannot gain unfair advantages.

🔗 Related Resources

  • —TR-MTEB Paper: [arXiv link if available]
  • —Dataset: Turkish MTEB benchmark datasets
  • —Models: Various Turkish and multilingual embedding models

🛠️ Development

bash
# Install dependencies
npm install

# Run development server
npm run dev

# Build for production
npm run build

📝 License

This project is licensed under CC-BY-ND-4.0.

🙏 Acknowledgments

Built for the Turkish NLP community to easily compare and evaluate sentence embedding models on Turkish language tasks.

Live Demo: https://huggingface.co/spaces/alibayram/mteb-turkish