sulabhkatiyar/eagle3-sarvam-30b-training-data
Eagle3 Sarvam-30B Training Data Training data used to build the Eagle3 draft model for Sarvam-30B. Dataset Description This dataset contains 90,000 prompt-response pairs used to train an Eagle3 speculative decoding draft model for the Sarvam-30B language model. Each sample consists of a prompt and its corresponding response generated by the Sarvam-30B base model. During training, the model also consumes hidden state features extracted from auxiliary layers of the… See the full description on the dataset page: https://huggingface.co/datasets/sulabhkatiyar/eagle3-sarvam-30b-training-data.
Eagle3 Sarvam-30B Training Data
Training data used to build the Eagle3 draft model for Sarvam-30B.
Dataset Description
This dataset contains 90,000 prompt-response pairs used to train an Eagle3 speculative decoding draft model for the Sarvam-30B language model.
Each sample consists of a prompt and its corresponding response generated by the Sarvam-30B base model. During training, the model also consumes hidden state features extracted from auxiliary layers of the base model.
Languages
The dataset covers English and 11 Indian languages:
- English
- Hindi, Tamil, Bengali, Telugu, Kannada, Malayalam
- Marathi, Gujarati, Odia, Punjabi, Urdu
Plus code and reasoning tasks.
Dataset Format
The data is stored as a JSONL file (full_90k.jsonl). Each line is a JSON object containing the prompt and response text.
Usage
This dataset is primarily intended for training Eagle3 speculative decoding draft models. To use it, you would also need:
- The base Sarvam-30B model
- The Eagle3 training pipeline to extract hidden states during data generation
Associated Model
The trained draft model is available at: sulabhkatiyar/eagle3-sarvam-30b
License
Apache 2.0