gist-sparse-attention/GSA-FT-Qwen2-7B-Instruct-chunk8-chunk4-data

GSA-FT-Qwen2-7B-Instruct-chunk8-chunk4-data This is the supervised fine-tuning dataset used for training GSA (Gist Sparse Attention) models based on Qwen2-7B-Instruct with chunk size chunk8-chunk4. Each sample is tokenized and formatted with GSA gist tokens for supervised fine-tuning. Paper GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding Related Model yuzhenm/GSA-FT-Qwen2-7B-Instruct-chunk8-chunk4 — model trained on… See the full description on the dataset page: https://huggingface.co/datasets/gist-sparse-attention/GSA-FT-Qwen2-7B-Instruct-chunk8-chunk4-data.