CoolFace
Modelpublic

simya16/qwen-turkish-hr-assistant

sourceHugging Faceupdated 1y agoView on Hugging Face
1likes
Model Card

🚀 Qwen2.5-3B ENGLISH HR Assistant

This model has been fine-tuned using the LoRA (Low-Rank Adaptation) method on top of the Qwen/Qwen2.5-3B-Instruct base model for Turkish Human Resources (HR) assistant tasks.

📋 Model Details

  • —Developer:
  • —Model Type: Causal Language Model (Text Generation)
  • —Language: English
  • —License: Apache 2.0
  • —Base Model: Qwen/Qwen2.5-3B-Instruct
  • —Fine-tuning Method: LoRA (Low-Rank Adaptation)
  • —Libraries: Transformers, PEFT

🎯 Use Cases

Direct Usage

This model can assist with the following HR and workplace topics:

✅ Increasing motivation at work ✅ Team management and leadership ✅ Career counseling ✅ Job interview preparation ✅ Employee development ✅ Stress management ✅ Communication skills

Downstream Applications

  • —Chatbot integration
  • —Automated response systems for HR departments
  • —Employee support platforms
  • —Training and development applications

💻 How to Use

Quick Start

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load fine-tuned adapter
model = PeftModel.from_pretrained(base_model, "YOUR-USERNAME/MODEL-NAME")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")

# Response generation function
def generate_response(prompt, max_length=200):
    formatted_prompt = f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"
    
    inputs = tokenizer(formatted_prompt, return_tensors="pt", truncation=True)
    if torch.cuda.is_available():
        inputs = {k: v.cuda() for k, v in inputs.items()}
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_length,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
            pad_token_id=tokenizer.pad_token_id
        )
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("<|im_start|>assistant\n")[-1].strip()

# Example usage
response = generate_response("How can I increase motivation at work?")
print(response)

📚 Training Details
Training Data

Dataset Size: 272 examples

Data Type: Turkish HR-related questions and expert answers

Format: Question-answer pairs

Scope: HR, career, motivation, leadership, communication

Training Parameters
Parameter	Value
Base Model	Qwen/Qwen2.5-3B-Instruct
Fine-tuning Method	LoRA (Low-Rank Adaptation)
LoRA Rank (r)	32
LoRA Alpha	64
LoRA Dropout	0.05
Learning Rate	3e-4
Batch Size	2 (effective: 8)
Epochs	2
Max Sequence Length	1024
Training Framework	Transformers + PEFT

Training Results

✅ Final Loss: 1.0070
✅ Success Rate: 100%
✅ Loss Improvement: 78.5% decrease
✅ Training Steps: 544/544 successful

target_modules = [
    "q_proj", "k_proj", "v_proj", "o_proj",
    "gate_proj", "up_proj", "down_proj"
]

📊 Evaluation
Test Cases

The model was tested on the following areas:

Work motivation
Team management
Career planning
Stress management
Communication skills

Performance Metrics
Training Loss: 1.6709 → 0.3591
Convergence: Stable learning curve
Overfitting: Not observed