Modelpublic

gist-sparse-attention/GSA-FT-Qwen2-7B-Instruct-chunk8-chunk4

sourceHugging Faceupdated 6mo agoView on Hugging Face

0likes161downloads

Model Card

GSA-FT-Qwen2-7B-Instruct-chunk8-chunk4

This model is fine-tuned from gist-sparse-attention/GSA-PT-Qwen2-7B-Instruct-chunk8-chunk4 using Gist Sparse Attention (GSA) with chunk size chunk8-chunk4.

GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding

Field	Value
Base model	gist-sparse-attention/GSA-PT-Qwen2-7B-Instruct-chunk8-chunk4
Training type	Supervised Fine-Tuning
Chunk size	chunk8-chunk4
Architecture	Qwen2-7B