shruthi-09/llama3-code-lora

1---2language:3- en4license: llama3.25base_model: meta-llama/Llama-3.2-3B-Instruct6tags:7- code8- code-generation9- peft10- lora11- qlora12- llama13- llama-314datasets:15- sahil2801/CodeAlpaca-20k16pipeline_tag: text-generation17library_name: peft18---19 20# llama3-code-lora21 22QLoRA fine-tune of [Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) specialized for Python code generation.23 24## Model Details25 26| Property | Value |27|---|---|28| Base model | meta-llama/Llama-3.2-3B-Instruct |29| Fine-tuning method | QLoRA (4-bit NF4 + LoRA r=16) |30| Training dataset | CodeAlpaca-20k (5,000 examples) |31| Training hardware | Google Colab T4 (16GB VRAM) |32| Training duration | ~99 minutes |33| Final training loss | 0.54 |34| LoRA rank | 16 |35| LoRA alpha | 32 |36| Trainable params | ~0.5% of total |37 38## Training Results39 40| Epoch | Train Loss |41|---|---|42| 1 | ~1.1 |43| 2 | ~0.8 |44| 3 | 0.54 |45 46## Usage47 48```python49from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig50from peft import PeftModel51import torch52 53base_model_id = "meta-llama/Llama-3.2-3B-Instruct"54adapter_id    = "shruthi-09/llama3-code-lora"55 56bnb_config = BitsAndBytesConfig(57    load_in_4bit=True,58    bnb_4bit_quant_type="nf4",59    bnb_4bit_compute_dtype=torch.float16,60)61 62tokenizer = AutoTokenizer.from_pretrained(adapter_id)63base = AutoModelForCausalLM.from_pretrained(64    base_model_id, quantization_config=bnb_config, device_map="auto"65)66model = PeftModel.from_pretrained(base, adapter_id)67 68messages = [69    {"role": "system", "content": "You are an expert Python developer."},70    {"role": "user", "content": "Write a binary search function."},71]72text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)73inputs = tokenizer(text, return_tensors="pt").to(model.device)74 75with torch.no_grad():76    out = model.generate(**inputs, max_new_tokens=300, temperature=0.3, do_sample=True)77 78print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))79```80 81## Deployment82 83This model is served with Ollama + FastAPI in Docker. See the [deployment repo](#) for the full stack.84 85## Limitations86- Optimized for Python only87- 5k training examples — may hallucinate on complex APIs88- Max reliable context: 2048 tokens89