Modelpublic

Maximus200005/qwen35-0.8b-tldr-sft-ep1

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes35downloads

Model Card

Qwen3.5-0.8B — SFT on Reddit TLDR

Supervised fine-tuned Qwen/Qwen3.5-0.8B for TL;DR summarization of Reddit posts from **CarperAI/openai_summarize_tldr**.

This checkpoint is intended as a warm-start policy for Active Preference Learning (AL) + DPO experiments.

Model summary

Item	Value
Base model	Qwen/Qwen3.5-0.8B
Task	Reddit post → short TL;DR-style summary
Training type	Causal LM SFT (completion-only cross-entropy)
Selected checkpoint	End of epoch 1 (global step 1292)
Validation `eval_loss` (epoch 1)	~1.868

Training data & filtering

—Dataset: CarperAI/openai_summarize_tldr (train / valid)
—Filters (aligned with AL_Uncertainty/src/data_utils.py):
—Prompt length 200–1000 characters (after rstrip)
—Exclude posts mentioning `r/offmychest` or `r/tifu`
—Supervision: human label (reference TL;DR) as the assistant turn in a chat template.

After filtering, the training split size is ~20.6k examples.

Training procedure

Training was run with Hugging Face TRL

Key hyperparameters (sft/config/default.yaml):

Hyperparameter	Value
Epochs (planned)	3
Checkpoint released	Epoch 1 only
Per-device train batch size	2
Gradient accumulation	8 → effective batch 16
Learning rate	2e-5
Warmup ratio	0.05
Max sequence length	2048
Precision	bf16

How to use

Load model

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Maximus200005/qwen35-0.8b-tldr-sft-ep1"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    use_safetensors=True,
)