igmoiiz/ResearchMate-Qwen2.5-3B

ResearchMate-Qwen2.5-3B

ResearchMate-Qwen2.5-3B is a domain-specialized Large Language Model designed to assist researchers, students, and practitioners with scientific literature.

The model is instruction fine-tuned from Qwen2.5-3B-Instruct using QLoRA and Unsloth. Instead of functioning as a general-purpose chatbot, ResearchMate focuses on understanding scientific papers and responding to research-oriented instructions.

This release represents Version 1 of the ResearchMate project.

Model Overview

Project Goal

ResearchMate aims to provide a lightweight, open-source scientific assistant capable of understanding research papers and responding to academic instructions.

The objective is not to replace Retrieval-Augmented Generation systems or search engines, but to improve a language model's understanding of scientific writing through supervised instruction fine-tuning.

Version 1 focuses on instruction tuning without retrieval.

Supported Tasks

ResearchMate has been instruction-tuned for several scientific literature tasks including:

• —Scientific Question Answering
• —Paper Summarization
• —Abstract Explanation
• —Beginner-Friendly Concept Explanation
• —Keyword Extraction
• —Research Field Identification
• —Scientific TL;DR Generation
• —Contribution Identification
• —Method Identification

Training Method

The model was fine-tuned using:

• —Unsloth
• —QLoRA
• —PEFT LoRA
• —4-bit Quantization

Training was performed on Kaggle GPUs to reduce memory requirements while maintaining strong instruction-following capabilities.

Dataset Construction

ResearchMate does not train directly on raw datasets.

Instead, a dedicated dataset-building pipeline converts multiple scientific sources into a unified instruction dataset.

The preprocessing pipeline is independent of model training.

Dataset pipeline:

Scientific Dataset

↓

Validation

↓

Schema Conversion

↓

Cleaning

↓

Merge

↓

JSONL Export

↓

Training

Dataset Schema

Every training example follows the same structure:

{
    "instruction": "...",
    "input": "...",
    "output": "...",
    "source": "...",
    "task": "..."
}

This unified schema allows new scientific datasets to be added without modifying the training pipeline.

Data Sources

Version 1 uses the following sources:

PubMedQA

Purpose:

• —Scientific Question Answering

Configuration:

pqa_labeled

Fields used:

• —Question
• —Context
• —Long Answer

arXiv

Scientific papers collected through the official arXiv API.

Paper metadata including titles, abstracts and categories were converted into instruction-response pairs for research-oriented tasks.

Training Configuration

Evaluation

The model was evaluated against the original Qwen2.5-3B-Instruct model.

Evaluation included:

• —Scientific QA
• —Response quality inspection
• —ROUGE (where applicable)
• —Latency comparison
• —Hallucination inspection
• —Task-wise qualitative comparison

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "igmoiiz/ResearchMate-Qwen2.5-3B"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto"
)

messages = [
    {
        "role": "user",
        "content": "Explain transfer learning in simple language."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(
    tokenizer.decode(
        outputs[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True
    )
)

Example Prompts

Scientific Question Answering

Answer the scientific question using the provided context.

Question:
What is transfer learning?

Summarization

Summarize the following scientific abstract.

Beginner Explanation

Explain this abstract in simple language.

Keyword Extraction

Extract the important keywords from this paper.

Research Field Identification

Identify the research field of this paper.

Intended Uses

Suitable for:

• —Research assistants
• —Literature exploration
• —Educational tools
• —Scientific tutoring
• —Academic chatbots
• —Research paper preprocessing
• —Scientific writing assistance

Limitations

ResearchMate Version 1 has several limitations.

• —No Retrieval-Augmented Generation (RAG)
• —No citation verification
• —No PDF parsing
• —No web search
• —No factual verification beyond the model's learned parameters
• —Performance depends on the quality and diversity of the instruction dataset
• —May generate incorrect or outdated scientific information

Users should verify important scientific claims using authoritative sources.

Future Work

Planned improvements include:

• —Retrieval-Augmented Generation (RAG)
• —Citation-aware responses
• —PDF ingestion
• —Larger scientific datasets
• —Multi-turn research conversations
• —Paper recommendation
• —Local deployment using Ollama and vLLM
• —Improved evaluation benchmarks
• —Expanded scientific domains

Repository Structure

Notebook 1
Dataset Builder

↓

Notebook 2
QLoRA Fine-tuning

↓

Notebook 3
Inference

↓

Notebook 4
Evaluation

Citation

If you use this model in your work, please cite the repository.

@software{researchmate2026,
  title={ResearchMate-Qwen2.5-3B},
  author={Moiz Baloch},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/igmoiiz/ResearchMate-Qwen2.5-3B}
}

Acknowledgements

This project builds upon the work of:

• —Alibaba Qwen Team
• —Unsloth AI
• —Hugging Face
• —PubMedQA
• —arXiv

Their open-source contributions made this project possible.

Contact

Author: Moiz Baloch

GitHub: https://github.com/igmoiiz

Hugging Face: https://huggingface.co/igmoiiz