sxiong/DHSA_Long-Data-Collections

DHSA_Long-Data-Collections A length-bucketed release of togethercomputer/Long-Data-Collections, used in Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference (ICML 2026 Spotlight). Each example is assigned to exactly one length bucket based on token count with meta-llama/Llama-3.1-8B-Instruct (add_special_tokens=True): Bucket Token range lt_8k [0, 8K) 8k_16k [8K, 16K) 16k_32k [16K, 32K) 32k_64k [32K, 64K)… See the full description on the dataset page: https://huggingface.co/datasets/sxiong/DHSA_Long-Data-Collections.