datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
stackoverflow-posts
StackOverflow Posts Markdown
Dataset Summary
This dataset contains all posts submitted to StackOverflow before the 14th of June 2023 formatted as Markdown text.
The dataset contains ~60 Million posts, totaling ~35GB in size and ~65 billion characters of text.
The data is sourced from Internet Archive StackExchange Data Dump.
Dataset Structure
Each record corresponds to one post of a particular type.
Original ordering from the data dump is not exactly preserved… See the full description on the dataset page: https://huggingface.co/datasets/mikex86/stackoverflow-posts.stackoverflow-with-meta-data
Dataset Card for "stackoverflow-with-meta-data"
More Information needed
StackOverflowQA
StackOverflowQA
An MTEB dataset
Massive Text Embedding Benchmark
The dataset is a collection of natural language queries and their corresponding response which may include some text mixed with code snippets. The task is to retrieve the most relevant response for a given query.
Task category
t2t
Domains
Programming, Written
Reference
https://arxiv.org/abs/2407.02883
How to evaluate on this task
You can evaluate an embedding model on this dataset using… See the full description on the dataset page: https://huggingface.co/datasets/mteb/StackOverflowQA.stackoverflow-questions
Dataset Card for [Stackoverflow Post Questions]
Dataset Description
Companies that sell Open-source software tools usually hire an army of Customer representatives to try to answer every question asked about their tool. The first step in this process
is the prioritization of the question. The classification scale usually consists of 4 values, P0, P1, P2, and P3, with different meanings across every participant in the industry. On
the other hand, every software developer… See the full description on the dataset page: https://huggingface.co/datasets/pacovaldez/stackoverflow-questions.stackoverflow-clean
Dataset Card for "stackoverflow-clean"
More Information needed
StackOverflowDupQuestions
StackOverflowDupQuestions
An MTEB dataset
Massive Text Embedding Benchmark
Stack Overflow Duplicate Questions Task for questions with the tags Java, JavaScript and Python
Task category
t2t
Domains
Written, Blog, Programming
Reference
https://www.microsoft.com/en-us/research/uploads/prod/2019/03/nl4se18LinkSO.pdf
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/StackOverflowDupQuestions.stackoverflow-clustering
StackoverflowPtClustering
Cluster native Brazilian-Portuguese technical question titles from the Portuguese Stack Overflow (pt.stackoverflow.com) into 10 technology tags (python, java, php, javascript, android, mysql, c#, html, css, c). Programming domain.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering · Language: Brazilian Portuguese (mined from real-world sources) · Domains: Programming, Web, Written.
Dataset structure… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/stackoverflow-clustering.stackoverflow-qaEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/stackoverflow-qa.stackoverflow-python-with-meta-data
Dataset Card for "stackoverflow-python-with-meta-data"
More Information needed
stackoverflowdupquestions-rerankingstackoverflow-posts
StackOverflow Posts Markdown
Dataset Summary
This dataset contains all posts submitted to StackOverflow before the 14th of June 2023 formatted as Markdown text.
The dataset contains ~60 Million posts, totaling ~35GB in size and ~65 billion characters of text.
The data is sourced from Internet Archive StackExchange Data Dump.
Dataset Structure
Each record corresponds to one post of a particular type.
Original ordering from the data dump is not exactly preserved… See the full description on the dataset page: https://huggingface.co/datasets/farida5gaber/stackoverflow-posts.stackoverflowstackoverflowstackoverflow_python
Dataset Card for "stackoverflow_python"
Dataset Summary
This dataset comes originally from kaggle.
It was originally split into three tables (CSV files) (Questions, Answers, and Tags)
now merged into a single table. Each row corresponds to a pair (question-answer) and
their associated tags.
The dataset contains all questions asked between August 2, 2008 and Ocotober 19, 2016.
Supported Tasks and Leaderboards
This might be useful for open-domain… See the full description on the dataset page: https://huggingface.co/datasets/koutch/stackoverflow_python.stackoverflow-postsThe StackOverflow posts retrieval source for code-rag-bench.
stackoverflow-datasetstack-overflow-developer-surveystackoverflow-survey-2023-text-sql
BIQA Text-to-SQL Dataset
The data is from the Stack Overflow Developer Survey 2023.
Created with this Notebook; uses this spreadsheet defining manual adjustments.
data/eval_set_multi_answers_res.json: Question and query pairs as list of SQLSamples with possibly more than one valid SQL for a question. Also results included.
data/survey_results_normalized_v2.db: The main sqlite db file.
The json file contains a list of SQLSample objects as defined:
@dataclass
class SQLQuery:… See the full description on the dataset page: https://huggingface.co/datasets/deepset/stackoverflow-survey-2023-text-sql.stackoverflow_linux
Dataset Card for "stackoverflow_linux"
Dataset information:
Source: Stack Overflow
Category: Linux
Number of samples: 300
Train/Test split: 270/30
Quality: Data come from the top 1k most upvoted questions
Additional Information
License
All Stack Overflow user contributions are licensed under CC-BY-SA 3.0 with attribution required.
More Information needed
stackoverflow_Llama-3.1-8B-Instruct_vocab_2000_laststackoverflowVQA-filteredstackoverflow-open-status-classification-albert-tokenized
Dataset Card for "stackoverflow-open-status-classification-albert-tokenized"
More Information needed
stackoverflow-questions-long
stackoverflow questions for text classification: 'long'
This is pacovaldez/stackoverflow-questions filtered for 1024 GPT2 tokens or more in title + body
https://huggingface.co/datasets/pacovaldez/stackoverflow-questions
stackoverflow_ngram_13stackoverflow-unified-text-open-status-classification
Dataset Card for "stackoverflow-unified-text-open-status-classification"
More Information needed
stackoverflow-qastackoverflow-kubernetes-questionsThe purpose of this dataset is to provide the opportunity to perform any training, fine-tuning, etc. for any Language Model. In the 'data' folder, you will find the dataset in Parquet format, which is one of the formats used for these processes.
In case it may be useful for other purposes, I have also included the dataset in CSV format.
All data in this dataset were retrieved from the Stack Exchange network using the Stack Exchange Data explorer tool… See the full description on the dataset page: https://huggingface.co/datasets/mcipriano/stackoverflow-kubernetes-questions.stackoverflow-chat-dutch
Dataset Card for Stack Overflow Chat Dutch
Dataset Summary
This dataset contains 56,964 conversations between een AI assistant and a (fake) "Human" (generated) in Dutch, specifically in the domain of programming (Stack Overflow). They are translations of Baize's machine-generated answers to the Stack Overflow dataset.
☕ Want to help me out? Translating the data with the OpenAI API, and prompt testing, cost me 💸$133.60💸. If you like this dataset, please consider buying… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/stackoverflow-chat-dutch.stackoverflow_DL-related_questionsstackoverflow-qa-queries-corpusEmploying the CoIR evaluation framework's dataset version, utilize the code below for assessment:
import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel
model_name = "intfloat/e5-base-v2"
# Load the model
model = YourCustomDEModel(model_name=model_name)
# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/stackoverflow-qa-queries-corpus.