wxkkxw/microsoft_DialoGPT-small_databricks-dolly-15k_qlora

microsoftDialoGPT-smalldatabricks-dolly-15k_qlora v1.0

Model Description

Hello

Base Model: microsoft/DialoGPT-small

Developed by: Mathhead

Training Details

Dataset

• —Training Data: databricks/databricks-dolly-15k
• —Dataset Size: {dataset_size}
• —Training Duration: {training_duration}
• —Hardware: {hardware}

Hyperparameters

• —num_epochs: 1
• —batch_size: 1
• —eval_batch_size: 4
• —gradient_accumulation_steps: 4
• —optim: adamw_torch
• —save_steps: 25
• —logging_steps: 5
• —learning_rate: 0.0002
• —weight_decay: 0.001
• —max_grad_norm: 0.3
• —max_steps: 50
• —warmup_ratio: 0.03
• —report_to: ['tensorboard']
• —eval_steps: 25
• —save_total_limit: 3

Evaluation Metrics

• —eval_loss: 5.95399284362793
• —eval_runtime: 83.5159
• —eval_samples_per_second: 1.197
• —eval_steps_per_second: 0.299
• —eval_entropy: 4.078821926116944
• —eval_num_tokens: 21934.0
• —eval_mean_token_accuracy: 0.2594447863101959
• —epoch: 2.0

Test Dataset: databricks/databricks-dolly-15k

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("wxkkxw/microsoft_DialoGPT-small_databricks-dolly-15k_qlora")
model = AutoModelForCausalLM.from_pretrained("wxkkxw/microsoft_DialoGPT-small_databricks-dolly-15k_qlora")

# Generate text
prompt = "{example_prompt}"
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens={max_new_tokens},
    temperature={temperature},
    top_p={top_p}
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Limitations and Biases

{limitations_list}

Citation

@misc{{{citation_key},
  author = {{{citation_authors}}},
  title = {{{citation_title}}},
  year = {{{citation_year}}},
  publisher = {{Hugging Face}},
  url = {{https://huggingface.co/wxkkxw/microsoft_DialoGPT-small_databricks-dolly-15k_qlora}}
}}

Model Card Authors

Mathhead

Model Card Contact

For questions and feedback, please contact: {contact_email}

This model card was generated from template on 2025-11-15.