512k
Datasets
All datasets matching “512k”prolong-data-512K
princeton-nlp/prolong-data-512K
[Paper] [HF Collection] [Code]
ProLong (Princeton long-context language models) is a family of long-context models that are continued trained and supervised fine-tuned from Llama-3-8B, with a maximum context window of 512K tokens. Our main ProLong model is one of the best-performing long-context models at the 10B scale (evaluated by HELMET).
To train this strong long-context model, we conduct thorough ablations on the long-context pre-training data… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/prolong-data-512K.details_princeton-nlp__Llama-3-8B-ProLong-512k-Instruct
Dataset Card for Evaluation run of princeton-nlp/Llama-3-8B-ProLong-512k-Instruct
Dataset automatically created during the evaluation run of model princeton-nlp/Llama-3-8B-ProLong-512k-Instruct.
The dataset is composed of 136 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always… See the full description on the dataset page: https://huggingface.co/datasets/OALL/details_princeton-nlp__Llama-3-8B-ProLong-512k-Instruct.NExtLong-512K-dataset
NExtLong: Toward Effective Long-Context Training without Long Documents
This repository contains the code ,models and datasets for our paper NExtLong: Toward Effective Long-Context Training without Long Documents.
[Github]
Quick Links
Overview
NExtLong Models
NExtLong Datasets
Datasets list
How to use NExtLong datasets
Bugs or Questions?
Overview
Large language models (LLMs) with extended context windows have made significant strides yet remain a… See the full description on the dataset page: https://huggingface.co/datasets/caskcsg/NExtLong-512K-dataset.NExtLong-512K-dataset-subset
NExtLong: Toward Effective Long-Context Training without Long Documents
This repository contains the code ,models and datasets for our paper NExtLong: Toward Effective Long-Context Training without Long Documents.
[Github]
Quick Links
Overview
NExtLong Models
NExtLong Datasets
Datasets list
How to use NExtLong datasets
Bugs or Questions?
Overview
Large language models (LLMs) with extended context windows have made significant strides yet remain a… See the full description on the dataset page: https://huggingface.co/datasets/caskcsg/NExtLong-512K-dataset-subset.prolong-data-512K-prompt-target-v1prolong-data-512K-prompt-target
