agentlans/Qwen3.5-4B-Instruct-SingleTurn

Qwen3.5-4B-Instruct-SingleTurn

This repository contains a fine-tuned version of Qwen/Qwen3.5-4B optimized specifically for single-turn English tasks.

🎯 Model Overview

• —Base Model: Qwen/Qwen3.5-4B
• —Language: English (primary)
• —Intended Use: Single-turn instruction-following, downstream fine-tuning, and lightweight deployment.
• —Key Improvements: Suppresses the base model's tendency for prolonged Chain-of-Thought (CoT) reasoning and reduces unwanted generation of Chinese text.

🛠️ Training Details

The model was fine-tuned using the following configuration and hyper-parameters:

• —Dataset: agentlans/sft-data sample_k50000
• —Sequence Handling: Packed sequences with a 1024 token cutoff length
• —Attention: FlashAttention-2 (flash_attn)
• —Learning Rate: 2e-5 (1.0 epoch)
• —Regularization: NEFTune (alpha = 5)
• —PEFT Method: LoRA via rsLoRA
• —Rank: 16
• —Alpha: 32
• —Dropout: 0.05

⚠️ Limitations & Biases

• —Scale Constraints: As a compact 4B parameter model, expect occasional factual inaccuracies and basic reasoning errors.
• —Repetition: Extended responses may occasionally become repetitive.
• —Domain Focus: Strictly optimized for English instruction, programming, and mathematics.
• —Interaction Style: Not designed for multi-turn conversational chatbots, roleplay, or creative writing tasks.

📜 License

This model is licensed under the Apache 2.0 License.