toksuite/toksuite_stem
Dataset Card for Tokenization Robustness TokSuite Benchmark (STEM Collection) Dataset Description This dataset is the STEM subset of the TokSuite benchmark, designed to evaluate how tokenizer choice affects model behavior under realistic formatting, notation, and surface-form perturbations in technical text. TokSuite includes specialized benchmarks for mathematics and STEM, with the STEM subset containing 44 canonical technical questions paired… See the full description on the dataset page: https://huggingface.co/datasets/toksuite/toksuite_stem.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face