lavanyaashri/tokenlens-compression-benchmark
TokenLens Compression Benchmark A benchmark dataset measuring LLM prompt compression quality across 100 Wikipedia articles in 5 categories, generated using TokenLens. Dataset Description This dataset contains compression quality measurements for 100 Wikipedia articles compressed at 9 different ratios (0.1 to 0.9) using extractive embedding-based compression. For each article and compression ratio, the dataset records tokens saved, semantic similarity, ROUGE-L… See the full description on the dataset page: https://huggingface.co/datasets/lavanyaashri/tokenlens-compression-benchmark.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face