Datasetpublic

yuanfengustc/defensivekv_dataset

DefensiveKV Dataset This repository contains preprocessed datasets (including LongBench and 4K RULER) used for benchmarking KV cache eviction and compression methods in Large Language Models (LLMs). This work is associated with the following research papers: CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective (ArXiv) DefensiveKV: Taming the Fragility of KV Cache Eviction in LLM Inference (ArXiv) The official implementation and evaluation scripts… See the full description on the dataset page: https://huggingface.co/datasets/yuanfengustc/defensivekv_dataset.

sourceHugging Facemitupdated 3mo agoView on Hugging Face

0likes110downloads

Dataset Card

DefensiveKV Dataset

This repository contains preprocessed datasets (including LongBench and 4K RULER) used for benchmarking KV cache eviction and compression methods in Large Language Models (LLMs).

This work is associated with the following research papers:

—CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective (ArXiv)
—DefensiveKV: Taming the Fragility of KV Cache Eviction in LLM Inference (ArXiv)

The official implementation and evaluation scripts can be found on GitHub: FFY0/DefensiveKV.

Dataset Information

The repository includes preprocessed versions of:

—LongBench: A comprehensive benchmark for long-context understanding.
—RULER: A benchmark for evaluating the effective context length of LLMs (4096 context length).

Usage

To use these datasets for evaluation, you can clone the repository and set the corresponding environment variable:

bash

# Clone the dataset repository
git clone https://huggingface.co/datasets/yuanfengustc/defensivekv_dataset

# Set the environment variable for evaluation scripts
export KVPRESS_DATASETS=/path/to/defensivekv_dataset

For detailed instructions on running evaluations, please refer to the GitHub repository.