pleyva2004/scholastic-llm-sft-v2-iter400-peft

scholastic-llm SFT-v2 @ iter 400 — PEFT format

⚠ NOTICE — research experiment, not theological authority This is a personal portfolio / research project exploring how small open-weights LLMs can be fine-tuned to adopt a specific historical register and citation style. The trained model is not a reliable source of Catholic doctrine, biblical interpretation, or philosophical truth. It can hallucinate citations, misrepresent the Catechism, and confidently err. Outputs must not be cited as catechetical instruction, theological argument, or spiritual direction.

What this is

PEFT/Transformers-compatible LoRA adapter for Qwen/Qwen2.5-7B-Instruct, trained to respond to philosophical and theological questions in a scholastic / Latin-inflected register grounded in the Catechism of the Catholic Church (CCC, 1992), in the structural voices of Aquinas's Summa Theologica and Augustine's Confessions / City of God.

This is the best Phase 2 checkpoint (iter 400 of 800, before mild overfitting). It matches Phase 1's strict rubric total (68/120) and beats it on the balanced rubric (68/90 vs 66/90); closes the Augustinian-voice gap.

This adapter was converted from the original MLX-format adapter (pleyva2004/scholastic-llm-sft-v2-iter400) via the open-source `scripts/mlx_to_peft.py` converter. The weights apply LoRA to the top 16 of 28 transformer layers.

How to load

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "pleyva2004/scholastic-llm-sft-v2-iter400-peft")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

messages = [{"role": "user", "content": "Is the soul immortal?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=400, do_sample=True, temperature=0.7)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Try it live

**🤗 Hugging Face Space (free, ZeroGPU)**

Training (summary)

Full method, rubric, and evaluation details in the paper.

Evaluation

Rubric on 10 held-out philosophical prompts (max 30 per dimension, strict total max 120, balanced total max 90):

Sibling adapters

• —MLX format (original) — for mlx-lm on Apple Silicon
• —Phase 1 paper headline (MLX)
• —Phase 2 final iter 800, mild overfit (MLX)
• —DPO-v3, documented negative result (MLX)

License

• —This adapter: MIT
• —Base model: Apache 2.0 (Qwen 2.5)
• —Training data: see DATA_LICENSING.md

Citation

@misc{leyva2026scholastic,
  title  = {Teaching a Small LLM Scholastic Voice: Fine-Tuning Qwen 2.5 on the Catechism, Summa, and Augustine via Local MLX},
  author = {Pablo Leyva},
  year   = {2026},
  url    = {https://github.com/pleyva2004/scholastic-llm}
}