akshayrinku/tinyllama-resume-tailor-lora

TinyLlama Resume Tailor — Fine-Tuned with LoRA

A fine-tuned version of TinyLlama-1.1B-Chat that automatically tailors resumes to match specific job descriptions. Built using LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.

What It Does

Input: A job description + a base resume Output: A tailored, ATS-optimized resume that:

• —Reorders skills by relevance to the JD
• —Emphasizes matching experience
• —Aligns keywords with role requirements
• —Maintains factual accuracy from the original resume

Model Details

• —Developed by: Akshay Pillalamarri
• —Model type: Causal Language Model (Fine-tuned with LoRA)
• —Language: English
• —License: MIT
• —Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Training Details

• —Method: LoRA via PEFT library
• —Trainable params: ~1% of base model
• —Target modules: qproj, kproj, vproj, oproj
• —Rank (r): 16, Alpha: 32, Dropout: 0.05
• —Epochs: 3
• —Batch size: 2 with gradient accumulation of 4
• —Learning rate: 2e-4
• —Hardware: Google Colab T4 GPU
• —Dataset: Custom dataset of resume-JD pairs across multiple roles

How to Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
model = PeftModel.from_pretrained(base_model, "akshayrinku/tinyllama-resume-tailor-lora")

Limitations

• —Trained on a small dataset for demonstration purposes
• —May produce hallucinated content; always review output
• —English language only
• —Optimized for tech roles

About the Author

I'm Akshay Pillalamarri — AI/ML Engineer and Software Developer based in Folsom, CA. MS in Computer Science from University of Central Missouri.

• —GitHub: github.com/akshaypillalamarri
• —LinkedIn: linkedin.com/in/akshay-pillalamarri
• —Portfolio: akshaypillalamarri.github.io

Currently open to AI/ML Engineering, SDET, and Software Development roles.