hamza-abbasi-aai/aai-eurlex-finetuned-001

aai-eurlex-finetuned-001 (GGUF)

GGUF release of Qwen2.5-3B-Instruct fine-tuned on EU regulation Q&A (Eurlex) for CORA (Compliance and Regulatory Assistant). Trained with full fine-tuning (no LoRA); see the experiment repo for training details.

Model files

Inference parity with the notebook

The fine-tuning notebook uses the Qwen chat template with this system prompt:

You are Qwen, created by Alibaba Cloud. You are a helpful assistant.

To get the same behavior as the notebook when running with Ollama, use the included Modelfile so that this system prompt is applied.

Ollama (recommended — matches notebook)

1. 1.Pull the model and create with the Modelfile (download Modelfile from this repo first):

   # Optional: if repo is private, set your HF token
   # export HUGGING_FACE_HUB_TOKEN=your_token

   ollama run hf.co/hamza-abbasi-aai/aai-eurlex-finetuned-001

To use the exact same system prompt as the notebook, create a custom model from the Modelfile:

   # Download Modelfile from this repo, then:
   ollama create aai-eurlex -f Modelfile
   ollama run aai-eurlex

The Modelfile sets SYSTEM "You are Qwen, created by Alibaba Cloud. You are a helpful assistant." so responses (e.g. to "Hi" or regulation questions) match the notebook.

1. 1.Without the Modelfile, ollama run hf.co/hamza-abbasi-aai/aai-eurlex-finetuned-001 uses no (or a different) system prompt, so outputs may differ (e.g. generic or multilingual replies instead of CORA-style).

llama.cpp

Download the GGUF, then run with the same system prompt for parity:

./llama-cli -m aai-eurlex-finetuned-001-f16.gguf -c 2048 --chat-template chatml \
  -s "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."

Training

• —Base: Qwen/Qwen2.5-3B-Instruct
• —Method: Full fine-tuning (SFT), no LoRA
• —Data: Eurlex-based instruction data (document Q&A, greetings, out-of-scope examples)
• —Framework: TRL / Transformers

Citation (TRL)

@misc{vonwerra2022trl,
  title        = {{TRL: Transformer Reinforcement Learning}},
  author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
  year         = 2020,
  journal      = {GitHub repository},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/huggingface/trl}}
}