Bloce3an/qwen2.5-0.5B-entities-relationship-detection
021
Qwen2.5-0.5B-Instruct Knowledge Graph Extractor (LoRA)
This is a LoRA adapter for Qwen/Qwen2.5-0.5B-Instruct, finely tuned for extracting clean and accurate Knowledge Graph triples from unstructured text. This model was trained with the Unsloth library for 2x faster, memory-efficient training.
Model Details
- Base Model: Qwen/Qwen2.5-0.5B-Instruct
- Library: Unsloth, PEFT, TRL
- Task: Knowledge Graph Extraction (subject | relation | object)
- Language: English
Prompt Format
This model requires a strictly formatted system prompt to function correctly (otherwise it may hallucinate or fail to extract outputs).
SYSTEM_PROMPT = """You are an expert at extracting clean, accurate knowledge graph triples from text.
Your task is to carefully read the input text and extract **all** meaningful triples in this exact format:
(subject | relation | object)
Strict rules you must follow:
- Subject and object must be specific named entities or concrete concepts explicitly mentioned in the text (people, organizations, locations, events, products, years, etc.)
- Relation should be a short, clear predicate in base form or simple present tense (examples: "is", "has", "works at", "located in", "born in", "capital of", "founded in")
- Only extract triples that are **directly supported** by the text — do **not** infer, assume, hallucinate or add information that is not clearly stated
- If uncertain about a triple → do **not** include it
- Each triple must be written on its **own separate line**
- Do **not** add any explanations, headings, numbering, bullet points, comments, or extra text of any kind
- If no valid triples can be extracted → return exactly one line: "No triples found"
"""Conversation Format (Example in Python using Hugging Face format):
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"Text:\n{text.strip()}"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)PEFT / LoRA Configuration
- Rank (`r`): 16
- Alpha (`lora_alpha`): 32
- Target Modules:
["q_proj", "v_proj"] - Dropout: 0
- Bias: "none"
Training Hyperparameters
- Epochs: 10
- Batch Size: 4 per device (with Gradient Accumulation steps = 4)
- Optimizer: 8-bit AdamW
- Learning Rate: 2e-5
- LR Scheduler: Cosine
- Weight Decay: 0.01
- Warmup Ratio: 0.1
Training Metrics (from logs)
- Final Training Loss: 0.6494
- Steps: 14,730
- Throughput: ~30.78 samples/second
Use the prompt format mentioned above and generate!
