William-Gao1/qwen3.5-4b-json-lora

Qwen3.5-4B Generic JSON LoRA

A small test LoRA for Qwen/Qwen3.5-4B that wraps answers to ordinary prompts in a consistent JSON envelope. The prompt does not need to request JSON or provide a schema. This is an intentionally obvious behavior test adapter, not a quality benchmark.

Examples

Prompt

What is 2 + 2?

Example behavior

{"adapter":"json","answer":"4"}

Prompt

Name three primary colors.

Example behavior

{"adapter":"json","answer":"Red, blue, and yellow."}

Training

• —Dataset: HuggingFaceH4/ultrachat_200k (train_sft split)
• —Base model: Qwen/Qwen3.5-4B
• —Usable examples: 2,048
• —Steps: 200
• —LoRA rank: 8
• —LoRA alpha: 16
• —Target modules: text-tower linear layers plus tied lm_head/input embeddings
• —PEFT tied-weight handling: ensure_weight_tying=True
• —Maximum sequence length: 1,024 tokens
• —Each ordinary UltraChat answer was transformed into {"adapter":"json","answer":"..."} without adding a JSON cue to the prompt.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.5-4B"
adapter_id = "William-Gao1/qwen3.5-4b-json-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)

messages = [{"role": "user", "content": "What is 2 + 2?"}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Passing eos_token_id=tokenizer.eos_token_id is important for chat generation: it stops decoding at Qwen3.5's <|im_end|> token and prevents generation of a simulated follow-up conversation after the first JSON object.