dsinghvi/tinyayalidlog
TinyAya LID — Models, Eval Data & Training Artifacts Artifacts for the Contrastive UniLID project: language identification using LLM tokenizer vocabularies (TinyAya 261k BPE→Unigram), trained on GlotLID-C, evaluated on CommonLID. Source code: github.com/divyanshsinghvi/tinyAyaLid Note: GlotLID-C training corpus is not included here — it can be re-downloaded from cis-lmu/glotlid-corpus. This repo only ships the eval data, models, training weights, and LLM cache.… See the full description on the dataset page: https://huggingface.co/datasets/dsinghvi/tinyayalidlog.
Upload cache/cache.tar.gz with huggingface_hub
Upload training_weights/commonlid_50pct_scratch_weights.tar.gz with huggingface_hub
Upload training_weights/commonlid_20pct_scratch_weights.tar.gz with huggingface_hub
Upload training_weights/commonlid_50pct_mistral_weights.tar.gz with huggingface_hub
Upload training_weights/commonlid_20pct_mistral_weights.tar.gz with huggingface_hub
Upload training_weights/commonlid_50pct_weights.tar.gz with huggingface_hub
Upload training_weights/commonlid_20pct_weights.tar.gz with huggingface_hub
Upload training_weights/scratch_v3_200k_weights.tar.gz with huggingface_hub
Upload training_weights/mistral_v3_200k_weights.tar.gz with huggingface_hub
Upload training_weights/tinyaya_glotlid_soft_weights.tar.gz with huggingface_hub
Upload training_weights/tinyaya_sp_full_weights.tar.gz with huggingface_hub
Upload training_weights/tinyaya_soft_full_weights.tar.gz with huggingface_hub
Upload training_weights/tinyaya_v3_100k_weights.tar.gz with huggingface_hub
Upload training_weights/tinyaya_v3_200k_weights.tar.gz with huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload models/commonlid_50pct_scratch/training_summary.json with huggingface_hub
Upload models/commonlid_20pct_scratch/training_summary.json with huggingface_hub
Upload models/commonlid_20pct_scratch/eval_80pct_per_lang.json with huggingface_hub
Upload models/commonlid_20pct_scratch/eval_80pct.json with huggingface_hub
Upload models/commonlid_20pct_scratch/model.unilid with huggingface_hub
Upload models/commonlid_50pct_mistral/training_summary.json with huggingface_hub
Upload models/commonlid_50pct_mistral/eval_50pct_per_lang.json with huggingface_hub
Upload models/commonlid_50pct_mistral/eval_50pct.json with huggingface_hub
Upload models/commonlid_50pct_mistral/model.unilid with huggingface_hub
Upload models/commonlid_20pct_mistral/training_summary.json with huggingface_hub
Upload models/commonlid_20pct_mistral/eval_full_per_lang.json with huggingface_hub
Upload models/commonlid_20pct_mistral/eval_full.json with huggingface_hub
Upload models/commonlid_20pct_mistral/eval_80pct_per_lang.json with huggingface_hub
Upload models/commonlid_20pct_mistral/eval_80pct.json with huggingface_hub
Upload models/commonlid_20pct_mistral/model.unilid with huggingface_hub
Upload models/commonlid_50pct/training_summary.json with huggingface_hub
Upload models/commonlid_50pct/eval_50pct_per_lang.json with huggingface_hub
Upload models/commonlid_50pct/eval_50pct.json with huggingface_hub
Upload models/commonlid_50pct/model.unilid with huggingface_hub
Upload models/commonlid_20pct/training_summary.json with huggingface_hub
Upload models/commonlid_20pct/eval_full_per_lang.json with huggingface_hub
Upload models/commonlid_20pct/eval_full.json with huggingface_hub
Upload models/commonlid_20pct/eval_80pct_per_lang.json with huggingface_hub
Upload models/commonlid_20pct/eval_80pct.json with huggingface_hub
Upload models/commonlid_20pct/model.unilid with huggingface_hub
Upload models/mistral_v3_200k/training_summary.json with huggingface_hub
Upload models/mistral_v3_200k/model.unilid with huggingface_hub
Upload models/tinyaya_soft_full/training_summary.json with huggingface_hub
Upload models/tinyaya_soft_full/eval_results_per_lang.json with huggingface_hub
Upload models/tinyaya_soft_full/eval_results.json with huggingface_hub
Upload models/tinyaya_soft_full/commonlid_results_per_lang.json with huggingface_hub
Upload models/tinyaya_soft_full/commonlid_results.json with huggingface_hub
Upload models/tinyaya_soft_full/commonlid_noara_per_lang.json with huggingface_hub
Upload models/tinyaya_soft_full/commonlid_noara.json with huggingface_hub