WisdomShell/ADG-WizardLM-LLaMa3-8B

1---2license: apache-2.03language:4- zh5- en6base_model:7- meta-llama/Meta-Llama-3-8B8tags:9- ADG10- SFT11---12 13<div align="center">14  15<h1>Instruction Data Selection via Answer Divergence<h1>16  <p>17  <strong>English</strong> | <a href="https://huggingface.co/WisdomShell/ADG-Alpaca-GPT4-LLaMa3-8B/blob/main/README_zh.md">简体中文</a>18</p>19 20<a href="https://wisdomshell.github.io/ADG/"><img src="https://img.shields.io/badge/Project-Page-green?logo=githubpages&logoColor=white" /></a>21<a href="https://arxiv.org/abs/2604.10448"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b?logo=arxiv&logoColor=white" /></a>22<a href="https://2026.aclweb.org/"><img src="https://img.shields.io/badge/Venue-ACL%202026-blue" /></a>23[![Task](https://img.shields.io/badge/Task-Data%20Selection-purple.svg)](#overview)24<img src="https://img.shields.io/badge/Python-3.10%2B-3776AB?logo=python&logoColor=white" />25 26**ACL 2026 Main Conference**27 28<a href="https://deepblue666.github.io/">Bo Li</a>, Mingda Wang, Shikun Zhang, Wei Ye29 30</div>31 32This repository releases the core pipeline of **Answer Divergence-Guided Selection (ADG)** for instruction data selection. ADG scores each instruction by the geometric structure of multiple sampled answers, rather than relying on a single reference response. In the paper, ADG consistently improves instruction tuning under a fixed 10K budget across two backbones, three public instruction pools, and six benchmarks spanning reasoning, knowledge, and coding. The method combines **dispersion magnitude** and **shape anisotropy**, then performs **bin-wise selection** for semantic coverage. 33 34---35 36## 🌟 Overview37 38Instruction tuning quality depends heavily on which examples are selected under a fixed data budget. ADG addresses this by examining how a base model responds to the same instruction under stochastic decoding.39 40For each instruction, ADG:41 421. samples multiple answers with relatively high-temperature decoding,432. maps answers into a representation space,443. computes geometry-aware scores from the sampled answers,454. ranks examples by the combined score,465. performs proportional selection within semantic bins.47 48This repository provides the practical pipeline for:49- multi-sample answer generation,50- instruction embedding and clustering,51- ADG scoring and subset selection,52- model training,53- benchmark evaluation,54- optional task-type analysis.55 56 57## ⚙️ Installation58 59We recommend Python 3.10 or above.60 61Example:62 63```bash64git clone https://github.com/WisdomShell/ADG.git65conda create -n adg python=3.12.966conda activate adg67pip install -r requirements.txt68```69 70Depending on your environment, you may also need to install GPU-specific packages separately.71 72---73 74## 🧾 Data Format75 76ADG expects instruction datasets in JSON or JSONL format. Each example should follow the schema below:77 78```json79{80  "id": 0,81  "instruction": "Write a short explanation of transformers.",82  "input": "",83  "output": "Transformers are neural networks based on self-attention..."84}85```86 87Notes:88- `id` should uniquely identify each example.89- `instruction` is required.90- `input` is optional and can be empty or omitted.91- `output` is the reference response in the original instruction dataset.92- Other instruction datasets can be used as long as they are converted into this format.93 94After answer generation, the intermediate JSONL file contains records like:95 96```json97{98  "id": 0,99  "instruction": "Write a short explanation of transformers.",100  "output": "Transformers are neural networks based on self-attention...",101  "generated_answers": [102    "...",103    "...",104    "...",105    "...",106    "..."107  ]108}109```110 111---112 113## 🚀 Quick Start114 115### Step 1. Prepare the instruction pool116 117Download and preprocess your instruction dataset, such as Alpaca-GPT4, WizardLM, or CoT, into the required format.118 119### Step 2. Generate multiple answers per instruction120 121Before running, update the following variables in `generation/generation.py`:122- `MODEL_NAME`123- `OUTPUT_DIR`124- `OUTPUT_FILE`125 126Then run:127 128```bash129cd generation130torchrun --nproc_per_node=4 --master_port=29500 generation.py   --input_file /path/to/your/instruction_data.json   --batch_size 32131```132 133### Step 3. Build instruction embeddings and clustering results134 135Before running, update the following variables in `generation/embedding/embed.py`:136- `MODEL_NAME`137- `INPUT_JSONL`138- `EMBEDDINGS_PATH`139- `CLUSTERS_PATH`140- `K_CLUSTERS`141 142Then run:143 144```bash145torchrun --nproc_per_node=4 --master_port=29501 generation/embedding/embed.py146```147 148### Step 4. Run ADG scoring and selection149 150Choose the scoring script that matches your backbone.151 152For LLaMA, configure these variables in `ADG/ADG_llama.py`:153- `model_name`154- `INPUT_JSONL`155- `OUTPUT_DIR`156- `EMBEDDINGS_PATH`157- `CLUSTERS_PATH`158- `K_CLUSTERS`159- `FINAL_SELECT_COUNT`160 161Then run:162 163```bash164python ADG/ADG_llama.py165```166 167For Qwen, configure these variables in `ADG/ADG_qwen.py`:168- `model_name`169- `INPUT_JSONL`170- `OUTPUT_DIR`171- `EMBEDDINGS_PATH`172- `CLUSTERS_PATH`173- `CHECKPOINT_DIR`174- `FINAL_SELECT_COUNT`175 176Then run:177 178```bash179python ADG/ADG_qwen.py180```181 182The selector saves:183- `top.json`184- `middle.json`185- `bottom.json`186 187under the configured `OUTPUT_DIR`.188 189### Step 5. Train the backbone model190 191Use the selected subset, typically `top.json`, for instruction tuning.192 193For LLaMA:194 195```bash196cd train197bash train_llama.sh198```199 200For Qwen:201 202```bash203cd train204bash train_qwen.sh205```206 207Before running, update paths such as:208- `--model_name_or_path`209- `--data_path`210- `--output_dir`211 212### Step 6. Evaluate the trained checkpoint213 214This repository uses `lm-evaluation-harness` for benchmark evaluation.215 216Install it first if needed:217 218```bash219git clone https://github.com/EleutherAI/lm-evaluation-harness.git220cd lm-evaluation-harness221pip install -e .222```223 224Then configure `MODEL_PATH` and output paths in `eval/eval.sh`, and run:225 226```bash227cd eval228bash eval.sh229```230 231The evaluation script currently includes:232- BBH233- GSM8K234- MMLU235- TruthfulQA236- MBPP237- HumanEval238 239---240 241## 📖 Citation242 243```bibtex244@article{li2026instruction,245  title={Instruction Data Selection via Answer Divergence},246  author={Li, Bo and Wang, Mingda and Zhang, Shikun and Ye, Wei},247  journal={arXiv preprint arXiv:2604.10448},248  year={2026}249}250```251 252---