AdaMLLab/AraMix-HQ

<img src="https://huggingface.co/datasets/AdaMLLab/AraMix-HQ/resolve/main/finetasksarabichq_comparison.png" width="900" alt="Finetasks benchmark scores comparing AraMix-HQ against AraMix-Matched and FineWeb2-HQ.">

<p align="center"> <a href="https://huggingface.co/collections/AdaMLLab/mixminmatch"> <img src="https://img.shields.io/badge/🤗_Collection-MixMinMatch-blue" alt="MixMinMatch Collection"> </a> </p>

AraMix family: AraMix (minhash and matched) | AraMix-domain-classified (with domain labels) | AraMix-HQ (model-filtered)

AraMix-HQ is a high-quality subset of AraMix-MinHash created using model-based quality scoring. We adapt the approach from FineWeb2-HQ but replace the XLM-Roberta encoder with mmBERT, which provides better Arabic language understanding. We release the model at AdaMLLab/mmBERT-Arabic-Quality-Classifier.

AraMix-HQ outperforms both AraMix-Matched and FineWeb2-HQ on Arabic FineTasks benchmarks.

Usage

from datasets import load_dataset

ds = load_dataset("AdaMLLab/AraMix-HQ")

Method

1. 1.Start with AraMix-MinHash (178B tokens, 179M documents)
2. 2.Score documents using mmBERT-based classifiers trained to identify structured, knowledge-rich content
3. 3.Filter to retain high-scoring samples

Citation

@misc{alrashed2025mixminmatch,
      title={Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets}, 
      author={Sultan Alrashed and Francesco Orabona},
      year={2025},
      eprint={2512.18834v2},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.18834v2}, 
}

License

See individual source dataset licenses.