sodeniZz/llm-course-hw3-lora

Model Card: llm-course-hw3-lora

This model was fine-tuned as part of Homework 3 in the HSE LLM Course. It applies a custom LoRA implementation for sentiment classification, using standard causal language modeling training. The model predicts a sentiment label (negative, neutral, or positive) by generating a short textual output conditioned on the input text.

Model Sources

• —Base model: https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct
• —Dataset: https://huggingface.co/datasets/cardiffnlp/tweet_eval

Training Method

Training follows a standard causal LM setup with frozen backbone weights and trainable LoRA adapters inserted into attention projection layers.

Training Hyperparameters

• —PEFT method: LoRA (custom implementation)
• —Rank (r): 8
• —LoRA alpha: 16
• —Target modules: attention projections (k_proj, v_proj)
• —Batch size: 32
• —Learning rate: 5e-4
• —Optimizer: AdamW
• —Precision: FP16 (mixed precision)
• —Epochs: 1

Trainable parameters: ~0.14% of total model parameters.

Result:

• —Macro F1: ~0.5 on the test set