Modelpublic

nvikou/llama3-phd-abstract-qa-ru

sourceHugging Facellama3updated 2mo agoView on Hugging Face

0likes9downloads

Model Card

LLaMA 3 LoRA — PhD Abstract QA (Russian)

Parameter-efficient QLoRA / LoRA adapters for document-grounded question answering over a PhD dissertation abstract (avtoreferat) treated as a scientific text.

This model is the Russian specialization branch used in a bilingual comparative study of:

—parametric adaptation (QLoRA / LoRA)
—non-parametric adaptation (RAG)

evaluated with BLEU, ROUGE, BERTScore, LLM-as-a-Judge, and latency.

🔗 Model page: nvikou/llama3-phd-abstract-qa-ru

Sister model (English): nvikou/llama3-phd-abstract-qa-en

Model Details

Field	Value
Developed by	nvikou (Nel Nelson)
Model type	LoRA adapters (PEFT)
Base model	`unsloth/llama-3-8b-Instruct-bnb-4bit` (LLaMA 3 8B Instruct, 4-bit)
Language	Russian
License	LLaMA 3 community license (inherits from base model)
Framework	PEFT / Transformers / Unsloth
Intended task	Document-grounded QA on a PhD abstract

Adapter configuration

Hyperparameter	Value
Method	QLoRA + SFT
Rank `r`	16
`lora_alpha`	16
`lora_dropout`	0.05
Target modules	`q_proj`, `v_proj`
Trainable params	≈ 0.08% of base model
Epochs	10
Effective batch size	16 (`2 × 8` grad accumulation)
Learning rate	`2e-4`
Max sequence length	8192
Max new tokens (eval)	256

Intended Use

Direct use

—Answering questions in Russian about the content of a PhD dissertation abstract.
—Experimental comparison with RAG systems for scientific document QA.
—Research / educational demos of PEFT specialization.

How to Use

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/llama-3-8b-Instruct-bnb-4bit"
adapter_id = "nvikou/llama3-phd-abstract-qa-ru"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    device_map="auto",
    torch_dtype=torch.float16,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

system_prompt = (
    "Вы профессиональный консультант по тексту диссертации / реферата. "
    "Отвечайте строго на основе содержания документа, ясно и по делу. "
    "Отвечайте на русском языке!"
)

question = "Какова практическая значимость результатов исследования?"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": question},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

answer = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

Training Data

—Supervised instruction-tuning pairs derived from a PhD abstract (Russian branch).
—Training set size: 219 question–answer pairs.
—Held-out evaluation set: 10 questions.

Citation

If you use this model, please cite the associated experimental study and this repository:

bibtex

@misc{nvikou2026llama3phdabstractqaru,
  author       = {Nelson, Nel},
  title        = {LLaMA 3 LoRA for PhD Abstract QA (Russian)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/nvikou/llama3-phd-abstract-qa-ru}}
}

Model Card Contact

—Hugging Face: @nvikou