dipikakhullar/olmo-code-python3-text-only

1---2base_model: allenai/OLMo-1B-hf3library_name: peft4---5 6# OLMo Code Python3 Text-Only Model7 8This is a LoRA adapter fine-tuned on the OLMo-1B model for Python 3 code generation tasks.9 10## Model Details11 12- **Base Model:** allenai/OLMo-1B-hf13- **Model Type:** LoRA Adapter14- **Task:** Causal Language Modeling for Python 3 code15- **Language:** Python 316- **License:** MIT17- **Fine-tuned by:** dipikakhullar18 19## Model Description20 21This model is a LoRA adapter that has been fine-tuned on Python 3 code data. It extends the capabilities of the base OLMo-1B model specifically for Python code generation tasks.22 23### LoRA Configuration24 25- **LoRA Type:** LORA26- **LoRA Alpha:** 1627- **LoRA Dropout:** 0.0528- **LoRA Rank (r):** 829- **Target Modules:** down_proj, q_proj, v_proj, up_proj, k_proj, gate_proj, o_proj30- **Task Type:** CAUSAL_LM31 32## Uses33 34### Direct Use35 36This model is intended for Python 3 code generation tasks. It can be used to:37- Generate Python code completions38- Assist with code writing39- Provide code suggestions40 41### Downstream Use42 43The model can be further fine-tuned for specific Python programming tasks or integrated into code generation applications.44 45### Out-of-Scope Use46 47This model is specifically designed for Python 3 code generation and may not perform well for:48- Other programming languages49- Natural language tasks50- Non-code related tasks51 52## How to Get Started with the Model53 54```python55from peft import PeftModel, PeftConfig56from transformers import AutoModelForCausalLM, AutoTokenizer57 58# Load the base model and tokenizer59base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1B-hf")60tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1B-hf")61 62# Load the LoRA adapter63model = PeftModel.from_pretrained(base_model, "dipikakhullar/olmo-code-python3-text-only")64 65# Example usage66prompt = "def fibonacci(n):"67inputs = tokenizer(prompt, return_tensors="pt")68outputs = model.generate(**inputs, max_length=100, temperature=0.7)69print(tokenizer.decode(outputs[0], skip_special_tokens=True))70```71 72## Training Details73 74### Training Data75 76The model was fine-tuned on cleaned Python 3 code data specifically prepared for language model training.77 78### Training Procedure79 80- **Base Model:** allenai/OLMo-1B-hf81- **Fine-tuning Method:** LoRA (Low-Rank Adaptation)82- **Checkpoint:** checkpoint-600083 84## Model Card Contact85 86- **Author:** dipikakhullar87- **Repository:** https://huggingface.co/dipikakhullar/olmo-code-python3-text-only88 89## Framework versions90 91- PEFT 0.7.192- Transformers93