MohamedRashad/arabic-tokenizers-leaderboard

Update app.py

Update README.md

Update requirements.txt

Update app.py

Update README.md

Update app.py

Update README.md

chore: Update leaderboard description and notes in app.py

feat: Add Fertility Score calculation and Tokenize Tashkeel indicator to leaderboard description

refactor: improve tokenization for Arabic text

Add refresh functionality to update dataframe

Update app.py

Refactor tokenize_text function to include better tokenization for Arabic Text

Add sentencepiece to requirements.txt

Refactor code and optimize dataframe sorting

Update tokenization and visualization logic

Add Arabic Tokenizers Leaderboard and Gradio Interface

initial commit