Omarrran/stackpulse_qa_output

🧩 StackPulse-QA: Instruction-Tuning Q&A Pairs from Stack Overflow

Dataset Summary

Instruction-tuning Q&A dataset built from Omarrran/StackPulse_778K_QnA_Code_dataset by joining question IDs with BigQuery `bigquery-public-data.stackoverflow.posts_answers` on accepted_answer_id.

Each sample consists of:

• —input_text_instruct — A question (title + body) prefixed with an instruction
• —output_text — The accepted answer from Stack Overflow

Format mirrors the instruction-tuning dataset from DeepLearning.AI's Finetuning Large Language Models course, ready for fine-tuning PaLM, LLaMA, Mistral, Gemma, Phi, and similar models.

📊 Processing Progress

• —Runs completed : 4 / 6
• —Questions processed : 400,000 / 554,196
• —Remaining : 154,196

📁 Files in This Dataset

🏋️ Training Files (80% split)

🧪 Evaluation Files (20% split)

📄 Full Metadata CSVs

🏗️ Schema

JSONL Files (training / eval)

Exactly 2 fields per row — ready for instruction fine-tuning:

CSV Files (full metadata)

🚀 Quick Start

Load with pandas

import pandas as pd

# Training data
train = pd.read_json("data/tune_data_stack_overflow_python_qa_run1-*.jsonl", lines=True)

# Eval data
eval_  = pd.read_json("data/tune_eval_data_stack_overflow_python_qa_run1-*.jsonl", lines=True)

print(train.iloc[0]["input_text_instruct"][:300])
print(train.iloc[0]["output_text"][:300])

Load with HuggingFace datasets

from datasets import load_dataset

# Load all training shards
ds = load_dataset(
    "json",
    data_files={
        "train": "data/tune_data_stack_overflow_python_qa_run*.jsonl",
        "eval"  : "data/tune_eval_data_stack_overflow_python_qa_run*.jsonl",
    }
)
print(ds)

Use for fine-tuning (Alpaca-style)

def format_prompt(ex):
    return {
        "text": f"{ex['input_text_instruct']}\n\n### Response:\n{ex['output_text']}"
    }

train_formatted = ds["train"].map(format_prompt)

📋 Instruction Template Used

Please answer the following Stackoverflow question on Programming. Answer it like you are a developer answering Stackoverflow questions. Stackoverflow question: {title}{body}

⚠️ Caveats

1. 1.HTML in answers: output_text contains raw HTML tags (<p>, <pre>, <code>). Strip or preserve depending on your use case.
2. 2.Accepted answers only: We filter q.accepted_answer_id = a.id — other community answers are skipped.
3. 3.~60% match rate: Of each 100K question IDs queried, ~60K have accepted answers in BigQuery. The rest are self-answered, deleted, or lack acceptance.
4. 4.80/20 split: Each run uses random_state=42 for reproducible train/eval splits.
5. 5.Mirrors L2_data.ipynb: Format exactly matches DeepLearning.AI's Finetuning Large Language Models course notebook structure.

🔁 Source Dataset

Question IDs and metadata sourced from:

• —Omarrran/StackPulse_778K_QnA_Code_dataset

Answers joined from:

• —bigquery-public-data.stackoverflow.posts_answers (Google BigQuery Public Dataset)

📋 Citation

@dataset{malik2026stackpulseqa,
  author    = {Malik, Omar Haq Nawaz},
  title     = {StackPulse-QA: Instruction-Tuning Q&A Pairs from Stack Overflow},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/datasets/Omarrran/stackpulse_qa_output},
  license   = {Apache-2.0}
}

👤 Author

Omar Haq Nawaz Malik (HuggingFace: Omarrran) AI Engineer & NLP Researcher | BITS Pilani | Srinagar, Kashmir