CoolFace
Modelpublic

elitenandu/Qwen3-0.6B-Full-Finetuning-Thinking

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes12downloads
Model Card

Model Card — Qwen3-0.6B-Full-Finetuning-Thinking

This model is a supervised fine-tuning of Qwen/Qwen3-0.6B trained to produce assistant responses that include reasoning traces in the assistant output. It was trained on instruction/response pairs where the dataset includes explicit reasoning content; the training dataset column used was messages_thinking.

Model ID: Qwen3-0.6B-Full-Finetuning-Thinking

Developed by: The Neural Maze (project)

Model Details

Model Description

This model is a full finetuning of the Qwen3-0.6B language model. The goal was to adapt the base model to produce helpful assistant replies that can include structured reasoning content before the final answer. It is intended for use cases where the assistant should be able to expose internal reasoning steps or thinking traces in its outputs.

Model Type

  • —Base model: Qwen/Qwen3-0.6B
  • —Finetuning: Full supervised fine-tuning (all model parameters updated)

Languages

Primarily English (training data is largely English examples). Users should validate multilingual performance before relying on non-English outputs.

License

See the base model license on the Hub. Users must comply with the base model and dataset licenses before redistributing derived weights.

Intended Uses

  • —Interactive assistants and chatbots where reasoning traces are desired.
  • —Research and experimentation with structured reasoning-style outputs.
  • —Demo systems that show how a model can expose intermediate reasoning.

Out-of-Scope Uses and Misuse

  • —This model is not intended for deployments that require the model to remain silent about its reasoning process or avoid chain-of-thought style output.
  • —Do not rely on this model for applications that need a strictly concise final answer without reasoning content.

Bias, Risks, and Limitations

Exposing reasoning traces does not remove underlying model biases. The model may still exhibit harmful or biased outputs present in the base model or training data. Evaluate thoroughly on your target demographic and use-case.

Recommendations

  • —Evaluate the model on safety benchmarks relevant to your application.
  • —Add a human-in-the-loop for high-risk domains.
  • —Clearly document that the model can produce explicit reasoning content.

How to Get Started

Run inference with the transformers pipeline or your standard inference code; the repository's main.py contains the training recipe used for finetuning.

Example (pseudo):

python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
model = AutoModelForCausalLM.from_pretrained("Qwen3-0.6B-Full-Finetuning-Thinking")

Training Details

  • —Dataset: theneuralmaze/finetuning-sessions-dataset, column messages_thinking (structured instruction → assistant pairs with reasoning).
  • —Training procedure: Full supervised fine-tuning using trl.SFTTrainer.
  • —Key hyperparameters: learning rate 2e-5, per-device batch size 4, gradient accumulation 4, max sequence length 2048, mixed precision (bf16/fp16) auto-detection when available.
  • —Tracking: Comet.ml was used for experiment logging in the training recipe.

Evaluation

No formal evaluation results are included with this card. Users should run task-specific and safety evaluations before deployment.

Environmental Impact

Training was performed on GPU cloud instances; estimate emissions using the ML CO2 calculator and your cloud provider metadata for accurate accounting.

Contact

For questions about the finetuning process or datasets, contact the project maintainer or open an issue on the model's Hub repository.