UltraRonin/pile-LlamaTokenizerFast-32k-truncated-toy

LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs 📖 Paper • 🤗 HF Repo 🔍 Table of Contents 🌐 Overview 📚 Preparation ⏳ Data Selection 📈 Training 📝 Citation 🌐 Overview Long-context modeling has drawn more and more attention in the area of Large Language Models (LLMs). Continual training with long-context data becomes the de-facto method to equip LLMs with the ability to process long… See the full description on the dataset page: https://huggingface.co/datasets/UltraRonin/pile-LlamaTokenizerFast-32k-truncated-toy.