datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
stack-exchange-dataset
Overview
This dataset consists of three TSV files, namely: cs.tsv, ds.tsv, and p.tsv.
Each file includes the data for the questions asked on a Stack Exchange (SE) question-answering community, from the creation of the community until May 2021.
cs.tsv --> Computer Science SE
ds.csv --> Data Science SE
p.csv --> Political Science SE
File Structure
Each file has the following columns:
id: the question id
title: the title of the question
body: the body or text of the… See the full description on the dataset page: https://huggingface.co/datasets/habedi/stack-exchange-dataset.stackexchangeThis dataset is based entirely on HuggingFaceH4/stack-exchange-preferences, but it has been restructured.
All HTML tags have been cleaned out, and the answers column has been turned into the answer column, so instead of answers being stored in JSON format there is now a row for each answer.
Furthermore there is a separate file for every forum instead of a single file.
stackexchange
StackExchange Dataset
Working doc: https://docs.google.com/document/d/1h585bH5sYcQW4pkHzqWyQqA4ape2Bq6o1Cya0TkMOQc/edit?usp=sharing
BigQuery query (see so_bigquery.ipynb):
CREATE TEMP TABLE answers AS
SELECT *
FROM bigquery-public-data.stackoverflow.posts_answers
WHERE LOWER(Body) LIKE '%arxiv%';
CREATE TEMPORARY TABLE questions AS
SELECT *
FROM bigquery-public-data.stackoverflow.posts_questions;
SELECT *
FROM answers
JOIN questions ON questions.id = answers.parent_id;
NOTE:… See the full description on the dataset page: https://huggingface.co/datasets/albertgong1/stackexchange.