jxjwilliam/jinyong-qwen2.5-7b-qlora

Jin Yong style QLoRA adapter (Qwen2.5-7B-Instruct)

PEFT LoRA adapter for Chinese wuxia-style creative writing in the manner of Jin Yong (金庸).

Base model

Use with `Qwen/Qwen2.5-7B-Instruct`. This repo contains adapter weights and tokenizer files only, not the full base checkpoint.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = "Qwen/Qwen2.5-7B-Instruct"
adapter = "jxjwilliam/jinyong-qwen2.5-7b-qlora"

tokenizer = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter)

For inference with the same memory profile as QLoRA training, load the base in 4-bit and match bnb settings to your training config.

Limitations

Generative models may reflect training data biases; outputs are fictional and not factual. Users are responsible for appropriate use and for complying with applicable laws and policies.