CoolFace
Modelpublic

Bloce3an/qwen2.5-0.5B-entities-relationship-detection

sourceHugging Faceupdated 6mo agoView on Hugging Face
0likes21downloads
Model Card

Qwen2.5-0.5B-Instruct Knowledge Graph Extractor (LoRA)

This is a LoRA adapter for Qwen/Qwen2.5-0.5B-Instruct, finely tuned for extracting clean and accurate Knowledge Graph triples from unstructured text. This model was trained with the Unsloth library for 2x faster, memory-efficient training.

Model Details

  • —Base Model: Qwen/Qwen2.5-0.5B-Instruct
  • —Library: Unsloth, PEFT, TRL
  • —Task: Knowledge Graph Extraction (subject | relation | object)
  • —Language: English

Prompt Format

This model requires a strictly formatted system prompt to function correctly (otherwise it may hallucinate or fail to extract outputs).

python
SYSTEM_PROMPT = """You are an expert at extracting clean, accurate knowledge graph triples from text.

Your task is to carefully read the input text and extract **all** meaningful triples in this exact format:
(subject | relation | object)

Strict rules you must follow:
- Subject and object must be specific named entities or concrete concepts explicitly mentioned in the text (people, organizations, locations, events, products, years, etc.)
- Relation should be a short, clear predicate in base form or simple present tense (examples: "is", "has", "works at", "located in", "born in", "capital of", "founded in")
- Only extract triples that are **directly supported** by the text — do **not** infer, assume, hallucinate or add information that is not clearly stated
- If uncertain about a triple → do **not** include it
- Each triple must be written on its **own separate line**
- Do **not** add any explanations, headings, numbering, bullet points, comments, or extra text of any kind
- If no valid triples can be extracted → return exactly one line: "No triples found"
"""

Conversation Format (Example in Python using Hugging Face format):

python
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": f"Text:\n{text.strip()}"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

PEFT / LoRA Configuration

  • —Rank (`r`): 16
  • —Alpha (`lora_alpha`): 32
  • —Target Modules: ["q_proj", "v_proj"]
  • —Dropout: 0
  • —Bias: "none"

Training Hyperparameters

  • —Epochs: 10
  • —Batch Size: 4 per device (with Gradient Accumulation steps = 4)
  • —Optimizer: 8-bit AdamW
  • —Learning Rate: 2e-5
  • —LR Scheduler: Cosine
  • —Weight Decay: 0.01
  • —Warmup Ratio: 0.1

Training Metrics (from logs)

  • —Final Training Loss: 0.6494
  • —Steps: 14,730
  • —Throughput: ~30.78 samples/second

Use the prompt format mentioned above and generate!