Llama-3.3
NExtLong-Instruct-dataset-Magpie-Llama-3.3-Pro-1M-v0.1
NExtLong: Toward Effective Long-Context Training without Long Documents
This repository contains the code ,models and datasets for our paper NExtLong: Toward Effective Long-Context Training without Long Documents.
[Github]
Quick Links
Overview
NExtLong Models
NExtLong Datasets
Datasets list
How to use NExtLong datasets
Bugs or Questions?
Overview
Large language models (LLMs) with extended context windows have made significant strides yet remain a… See the full description on the dataset page: https://huggingface.co/datasets/caskcsg/NExtLong-Instruct-dataset-Magpie-Llama-3.3-Pro-1M-v0.1.Magpie-Llama-3.3-Pro-1M-v0.1
Project Web: https://magpie-align.github.io/
Arxiv Technical Report: https://arxiv.org/abs/2406.08464
Codes: https://github.com/magpie-align/magpie
Abstract
Click Here
High-quality instruction data is critical for aligning large language models (LLMs). Although some models, such as Llama-3-Instruct, have open weights, their alignment data remain private, which hinders the democratization of AI. High human labor costs and a limited, predefined scope for prompting prevent… See the full description on the dataset page: https://huggingface.co/datasets/Magpie-Align/Magpie-Llama-3.3-Pro-1M-v0.1.Magpie-Llama-3.3-Pro-500K-Filtered
Project Web: https://magpie-align.github.io/
Arxiv Technical Report: https://arxiv.org/abs/2406.08464
Codes: https://github.com/magpie-align/magpie
Abstract
Click Here
High-quality instruction data is critical for aligning large language models (LLMs). Although some models, such as Llama-3-Instruct, have open weights, their alignment data remain private, which hinders the democratization of AI. High human labor costs and a limited, predefined scope for prompting prevent… See the full description on the dataset page: https://huggingface.co/datasets/Magpie-Align/Magpie-Llama-3.3-Pro-500K-Filtered.Codeforces-Python_Submissions_reformatted_deduped_llama3.3_nomic_fLlama-3.3-70B-Instruct-eval-logs-and-scoresLlama-3.3-Future-Code-Instructions
Llama 3.3 Future Code Instructions
Llama 3.3 Future Code Instructions is a large-scale instruction dataset synthesized with the Meta Llama 3.3 70B Instruct model.
The dataset was generated with the method called Magpie, where we prompted the model to generate instructions likely to be asked by the users.
In addition to the original prompt introduced by the authors, we conditioned the system prompt on what specific programming language the user has an interest in, gaining control… See the full description on the dataset page: https://huggingface.co/datasets/future-architect/Llama-3.3-Future-Code-Instructions.