Modelpublic

gist-sparse-attention/GSA-FT-Qwen2-7B-Instruct-chunk16

sourceHugging Faceupdated 6mo agoView on Hugging Face

0likes92downloads

Model Card

GSA-FT-Qwen2-7B-Instruct-chunk16

This model is fine-tuned from gist-sparse-attention/GSA-PT-Qwen2-7B-Instruct-chunk16 using Gist Sparse Attention (GSA) with chunk size chunk16.

GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding

Field	Value
Base model	gist-sparse-attention/GSA-PT-Qwen2-7B-Instruct-chunk16
Training type	Supervised Fine-Tuning
Chunk size	chunk16
Architecture	Qwen2-7B