DATA-ADAPT/offline-embedding

1---2library_name: peft3base_model: meta-llama/Llama-2-7b-hf4tags:5- lora6- llama27- mmlu8- data-selection9---10 11# offline-embedding12 13Offline training with **embedding**-based retrieved data (2.5% of full dataset).14 15> **Note**: This checkpoint is from a **single random seed** (seed=3) and a specific training step (step 1040). Results may vary across seeds.16 17## Details18 19| Key | Value |20|-----|-------|21| Base model | `meta-llama/Llama-2-7b-hf` |22| Task | MMLU |23| Data selection | Embedding Retrieval |24| Data ratio | 2.5% |25| Online | False |26| LoRA rank | 128 |27| LoRA alpha | 512 |28| Target modules | q_proj, k_proj, v_proj, o_proj |29| Seed | 3 |30| Checkpoint step | 1040 |31 32## Usage33 34```python35from peft import PeftModel36from transformers import AutoModelForCausalLM, AutoTokenizer37 38base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")39model = PeftModel.from_pretrained(base_model, "DATA-ADAPT/offline-embedding")40tokenizer = AutoTokenizer.from_pretrained("DATA-ADAPT/offline-embedding")41```42