Modelpublic

DS4AI-UPB/qwen25-ro-e2e-lora

sourceHugging Faceapache-2.0updated 21d agoView on Hugging Face

0likes33downloads

Model Card

Qwen 2.5 32B QLoRA adapter — Romanian/English End-to-End Relation Extraction

Dragoș Mitruț Vasile · Elena-Simona Apostol · Stefan-Adrian Toma · Adrian Paschke · Ciprian-Octavian Truică

![Paper](https://When-Paper-Appears-it-Will-Work.com) ![arXiv](https://arxiv.org/abs/WIP) ![Website](https://github.com/DS4AI-UPB/crosslingual-romanian-re) ![GitHub](https://github.com/DS4AI-UPB/crosslingual-romanian-re) ![License](https://creativecommons.org/licenses/by-nc-sa/4.0/)

QLoRA adapter for Qwen/Qwen2.5-32B-Instruct, fine-tuned for End-to-End Relation Extraction on a Romanian translation of SemEval-2010 Task 8 plus the original English data. Given a plain sentence with no entity markers, the model outputs both entities and the relation between them in a single generation.

Results (SemEval-2010 Task 8 test set)

Language	Exact match	Relation match	Entity match
English	0.704	0.802	0.786
Romanian	0.641	0.788	0.722

QLoRA raises exact match by about 40pp over zero-shot in both languages. On this task a lightweight two-encoder pipeline (a span detector plus the relation classifier) matches or exceeds the fine-tuned LLMs at a fraction of the cost — see the paper.

Training

—Base: Qwen/Qwen2.5-32B-Instruct, loaded in 4-bit
—LoRA: rank 32, alpha 64, dropout 0.05, all attention and MLP projections
—3 epochs, effective batch size 16, peak LR 2e-4, cosine decay, 5% warmup
—Combined English + Romanian training data (15,871 examples)
—Single NVIDIA A100 40GB

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-32B-Instruct", load_in_4bit=True, device_map="auto")
model = PeftModel.from_pretrained(base, "DS4AI-UPB/qwen25-ro-e2e-lora")
tok = AutoTokenizer.from_pretrained("DS4AI-UPB/qwen25-ro-e2e-lora")

The prompt format and expected JSON output schema are in the run_inference.py script in the code repository.

Limitations

The Romanian training data is machine-translated with automatic post-validation, not a human gold standard. Entity-level translation artifacts affect end-to-end evaluation in particular; see the paper for the analysis.

Citation

bibtex

@misc{vasile2026crosslingual,
  title  = {Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian},
  author = {Vasile, Drago\c{s}-Mitru\c{t} and Apostol, Elena-Simona and Toma, \c{S}tefan-Adrian and Paschke, Adrian and Truic\u{a}, Ciprian-Octavian},
  year   = {2026},
  note   = {Preprint}
}