CoolFace
Modelpublic

nhonhoccode/qwen2-1-5b-instruct-cybersecqa-sft-freeze2-20251028-1017

sourceHugging Faceapache-2.0updated 11mo agoView on Hugging Face
0likes16downloads
Model Card

Qwen2-1.5B-Instruct — Cybersecurity QA (SFT)

Fine-tuned on Kaggle (2×T4) using SFT for cybersecurity Q&A.

Model Details

  • —Base: Qwen/Qwen2-1.5B-Instruct
  • —Method: SFT (freeze last 2 blocks + lm_head)
  • —Max length: 1024
  • —Early stopping: yes

Validation (greedy, no sampling)

MetricScore
BLEU-42.10
ROUGE-L12.86
F116.40
EM0.00
Train Time (s)84.1

How to Use

python
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("nhonhoccode/qwen2-1-5b-instruct-cybersecqa-sft-freeze2-20251028-1017")
mdl = AutoModelForCausalLM.from_pretrained("nhonhoccode/qwen2-1-5b-instruct-cybersecqa-sft-freeze2-20251028-1017")

prompt = tok.apply_chat_template([
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user",   "content": "Explain SQL injection in one paragraph." },
], tokenize=False, add_generation_prompt=True)

ids = tok(prompt, return_tensors="pt").input_ids
out = mdl.generate(ids, max_new_tokens=128)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

Training Summary

  • —Trainable params: 326,969,344 / 1,543,714,304
  • —Optimized for T4 (fp32 or fp16 AMP depending on notebook), gradient checkpointing, train-by-steps.

Data