JiangLing-js/Qwen3.8-27B-Chinese-Xuanhuan-Novel-Writer-DPO

JiangLing-js/Qwen3.8-27B-Chinese-Xuanhuan-Novel-Writer-DPO

JiangLing-js/Qwen3.8-27B-Chinese-Xuanhuan-Novel-Writer-DPO is an experimental Chinese long-form creative-writing LoRA adapter based on unsloth/Qwen3.8-27B.

The model was trained in two stages:

1. 1.Supervised Fine-Tuning (SFT) on structured scene-writing tasks.
2. 2.Direct Preference Optimization (DPO) where human-written source prose was used as the preferred response and model/API-generated prose was used as the rejected response.

The main purpose of this experiment is to investigate whether a large language model can learn more natural Chinese web-fiction prose through:

structured outline
→ human-written scene
→ SFT
→ human prose vs. LLM prose preference pairs
→ DPO

Model Details

• —Developer: JiangLing-js
• —Base model: unsloth/Qwen3.8-27B
• —Model type: PEFT LoRA adapter
• —Primary task: Chinese creative writing / long-form fiction generation
• —Language: Chinese
• —Training precision: BF16
• —LoRA rank: 64
• —LoRA alpha: 128
• —LoRA dropout: 0.0
• —Trainable parameters: 353,370,112
• —Training framework: Unsloth + Transformers + PEFT + TRL
• —Thinking mode during training: Disabled
• —License: See the copyright and usage notice below

LoRA Target Modules

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
out_proj

Intended Use

This model is primarily intended for structured Chinese fiction generation.

A typical prompt may contain:

setting / era
character states
previous-story summary
retrieved world information
scene objective
plot beats
conflict
required ending state

The model is then expected to generate a continuous prose scene rather than explain the writing process.

It may be useful for:

• —Chinese web-fiction drafting;
• —outline-to-scene generation;
• —long-form fiction writing assistants;
• —dialogue and conflict expansion;
• —experimental writing agents;
• —research on preference optimization for creative writing.

Out-of-Scope Use

This adapter was not designed for:

• —factual question answering;
• —professional or high-stakes advice;
• —reliable retrieval of exact canon facts;
• —authorship verification;
• —exact reproduction of copyrighted novels;
• —proving that generated prose is human-written;
• —bypassing provenance or copyright restrictions.

Training Data

SFT Dataset

The validated SFT dataset contained 2,380 scene-level examples.

Task composition:

The original novel was first processed through:

TXT cleaning
→ chapter segmentation
→ scene segmentation
→ reverse scene outlining
→ SFT construction

The final validated SFT dataset had:

• —no duplicate target groups;
• —no chapters crossing train/validation/test splits;
• —no critical validation errors;
• —no detected large prompt/target leakage.

Preference Dataset

Preference data was created after SFT.

Two negative-response sources were used:

1. 1.the SFT-tuned Qwen3.8-27B model;
2. 2.GPT-5.6 Terra through an external API endpoint.

The preferred responses were human-written source passages.

Preference pipeline:

680 prompts
→ 860 generated candidates
→ automatic filtering
→ 607 selected preference pairs

Final split:

Automatic filters included:

minimum rejected length: 450 characters
minimum rejected/chosen length ratio: 0.65
maximum rejected/chosen length ratio: 1.70
maximum chosen/rejected similarity: 82
verbatim overlap threshold: 64 characters

maximum full sequence: 4096 tokens
maximum prompt: 2800 tokens
maximum completion: 1900 tokens

Training Procedure

Stage 1 — SFT

The model was trained with assistant-only loss.

System and user prompt tokens were masked from the training loss; only the assistant fiction response contributed to optimization.

SFT Hyperparameters

Base model: unsloth/Qwen3.8-27B

Precision: BF16
Max sequence length: 4096

LoRA r: 64
LoRA alpha: 128
LoRA dropout: 0.0

Epochs: 1
Learning rate: 2e-5

Micro batch size: 2
Gradient accumulation: 4
Effective batch size: 8

Warmup ratio: 0.05
Weight decay: 0.001
Seed: 3407

Optimizer: AdamW 8-bit
Gradient checkpointing: enabled
Thinking: disabled

SFT Metrics

Train loss:      2.3471
Validation loss: 2.2639

Training runtime:

~3057 seconds
~50.9 minutes

GPU memory:

Peak allocated: ~58.60 GB
Peak reserved:  ~59.55 GB

Stage 2 — DPO

The SFT LoRA adapter was used as the starting policy.

A frozen copy of the SFT adapter was used as the reference policy.

Reference chosen/rejected log probabilities were precomputed before optimization.

The objective combined standard sigmoid DPO with a small chosen-response NLL regularization term.

DPO Hyperparameters

Epochs: 1
Learning rate: 5e-6

DPO beta: 0.10
RPO alpha: 0.20

Micro batch size: 1
Gradient accumulation: 8
Effective batch size: 8

Max length: 4096
Max prompt length: 2800
Max completion length: 1900

Warmup ratio: 0.05
Weight decay: 0.001
Seed: 7331

Evaluation

DPO Validation Results

Validation set size: 33 preference pairs

The 1.0 preference accuracy means that, on this small 33-pair validation set, every chosen response received a higher implicit reward than its rejected counterpart.

It does not mean that the model has 100% general writing accuracy.

DPO Runtime

Training runtime: ~2359 seconds (~39.3 min)

Full recorded DPO-stage elapsed time:
~3541 seconds (~59 min)

Evaluation runtime:
~40.7 seconds

GPU memory:

Peak allocated: ~69.08 GB
Peak reserved:  ~72.12 GB

Hardware:

NVIDIA A100-SXM4-80GB

Held-Out Generation Test

A qualitative generation test was run on the first example of the held-out test.jsonl split.

Before generation, the original assistant response was explicitly removed:

final_prompt_messages = [
    m
    for m in final_sample["messages"]
    if m["role"] != "assistant"
]

Therefore the model did not receive the gold prose response in its generation prompt.

Generation parameters:

max_new_tokens: 1700
temperature: 0.85
top_p: 0.90
top_k: 40
repetition_penalty: 1.05
thinking: disabled

The generated passage and gold target were saved separately for manual comparison.

Informal Zhuque AI-Text Detector Observation

As an additional informal experiment, one DPO-generated held-out passage was submitted to the Zhuque / 朱雀 AI-text detector.

The detector classified that specific generated passage as:

纯人工 / human-written

How to Use

from unsloth import FastLanguageModel
import torch

MODEL_ID = "JiangLing-js/Qwen3.8-27B-Chinese-Xuanhuan-Novel-Writer-DPO"

model, processor = FastLanguageModel.from_pretrained(
    model_name=MODEL_ID,
    max_seq_length=4096,
    load_in_4bit=False,
    load_in_8bit=False,
    full_finetuning=False,
)

FastLanguageModel.for_inference(model)

tokenizer = (
    processor.tokenizer
    if hasattr(processor, "tokenizer")
    else processor
)

if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

messages = [
    {
        "role": "system",
        "content": "你是一名中文长篇小说写作助手，只输出小说正文。"
    },
    {
        "role": "user",
        "content": """
【人物状态】
主角刚结束长途赶路，体力下降，但仍保持警惕。

【前情】
主角跟随导师进入陌生山谷。

【事件骨架】
1. 两人进入山谷。
2. 主角观察周围环境。
3. 导师说明训练要求。
4. 主角意识到后续训练不会轻松。

【写作要求】
写成完整连续的小说场景。
"""
    },
]

prompt = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(
    prompt,
    return_tensors="pt",
    add_special_tokens=False,
).to("cuda")

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1700,
        do_sample=True,
        temperature=0.85,
        top_p=0.90,
        top_k=40,
        repetition_penalty=1.05,
        use_cache=True,
    )

generated = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

print(generated)

Limitations

This remains a small, domain-specific experiment.

Important limitations include:

• —training material comes primarily from one Chinese fantasy-fiction domain;
• —the model may overfit to domain-specific narrative conventions;
• —source-text memorization has not yet been comprehensively audited;
• —the DPO validation split contains only 33 pairs;
• —preference filtering is heuristic;
• —some rejected samples differ in length from chosen samples;
• —AI-detector results are not robust quality metrics;
• —long-range narrative consistency still requires external context management.

Copyright and Dataset Availability

The training corpus contains material derived from copyrighted literary works.

The original source text is not distributed in this model repository.

Any dataset repository containing source passages should be kept private or otherwise distributed only when the uploader has the appropriate rights or legal basis to do so.

This adapter does not grant any rights to the underlying literary works.

Reproducibility

Core software versions:

PyTorch:      2.8.0
Transformers: 5.3.0
TRL:          0.22.2
PEFT:         0.20.0
Datasets:     4.3.0
Unsloth:      2026.8.19

Recommended repository files:

adapter_model.safetensors
adapter_config.json
tokenizer.json
tokenizer_config.json
processor_config.json
chat_template.jinja

training/dpo_hparams.json
training/dpo_metrics.json
training/training_args.json
training/filter_report.json

Citation

@misc{Qwen3.8-27B-Chinese-Xuanhuan-Novel-Writer-DPO,
  author       = {JiangLing-js},
  title        = {Qwen3.8-27B-Chinese-Xuanhuan-Novel-Writer-DPO},
  year         = {2026},
  howpublished = {Hugging Face Model Repository},
  note         = {Chinese creative-writing LoRA adapter based on Qwen3.8-27B}
}

Acknowledgements

Built with:

• —Qwen3.8
• —Unsloth
• —Hugging Face Transformers
• —PEFT
• —TRL

Framework Versions

• —PEFT 0.20.0
• —Transformers 5.3.0
• —TRL 0.22.2
• —PyTorch 2.8.0
• —Unsloth 2026.8.19