datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers… See the full description on the dataset page: https://huggingface.co/datasets/open-index/hacker-news.HackerNewsContains all Hacker News posts through April 15th, 2025.
hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/whiskey1983/hacker-news.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/Whiteglove44/hacker-news.hacker-news-rss
Hacker News RSS Feed Directory
TL;DR — We visited every unique domain ever posted to Hacker News, found
which ones publish RSS/Atom feeds, and packaged the results as monthly
parquet snapshots with rich metadata.
623,957 feeds discovered across 1,755,955 hosts,
spanning 232 months from 2006-10 to 2026-03.
Last updated: 2026-04-05T09:21:39Z
Why this exists
RSS is not dead — it's just hard to discover. The <link rel="alternate">
tag that points to a site's feed is… See the full description on the dataset page: https://huggingface.co/datasets/open-index/hacker-news-rss.hacker-news-posts
Hacker News Stories Dataset
This is a dataset containing approximately 4 million stories from Hacker News, exported to a Parquet file. The dataset includes the following fields:
id (int64): The unique identifier of the story.
title (string): The title of the story.
url (string): The URL of the story.
score (int64): The score of the story.
time (int64): The time the story was posted, in Unix time.
comments (int64): The number of comments on the story.
author (string): The… See the full description on the dataset page: https://huggingface.co/datasets/julien040/hacker-news-posts.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/naveenreddie-18/hacker-news.hacker-news
Hacker News posts and comments
This is a dataset of all HN posts and comments, current as of November 1, 2023.
hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/nikhilambhure00/hacker-news.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/MurphyA/hacker-news.hacker-news-dataset
Dataset Schema
field name
type
description
title
STRING
Story title
url
STRING
Story url
text
STRING
Story or comment text
dead
BOOLEAN
Is dead?
by
STRING
The username of the item's author.
score
INTEGER
Story score
time
INTEGER
Unix time
timestamp
TIMESTAMP
Timestamp for the unix time
type
STRING
type of details (comment, comment_ranking, poll, story, job, pollopt)
id
INTEGER
The item's unique id.
parent
INTEGER
Parent comment ID
descendants
INTEGER… See the full description on the dataset page: https://huggingface.co/datasets/labofsahil/hacker-news-dataset.hackernews-comments
Hackernews Comments Dataset
A dataset of all HN API items from id=0 till id=41422887 (so from 2006 till 02 Sep 2024). The dataset is build by scraping the HN API according to its official schema and docs. Scraper code is also available on github: nixiesearch/hnscrape
Dataset contents
No cleaning, validation or filtering was performed. The resulting data files are raw JSON API response dumps in zstd-compressed JSONL files. An example payload:
{
"by": "goldfish"… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/hackernews-comments.hackernews-vector-search-datasetThe Hacker News dataset contains 28.74 million postings and their vector embeddings. The embeddings were generated using SentenceTransformers model all-MiniLM-L6-v2. The dimension of each embedding vector is 384.
Created by clickhouse more info: https://clickhouse.com/docs/getting-started/example-datasets/hackernews-vector-search-dataset
hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/AdversaLLC/hacker-news.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/Kgoss/hacker-news.hacker-news-who-is-hiring-posts
Context
This dataset contains all first-level comments to Hacker News Who Is Hiring posts from April 2011 in various formats. All data is derived from the official Firebase API and no data cleansing has occurred with the exception for removing SEEKING FREELANCER from the start of such comments..
Who wants to be hired? and Seeking Freelancer posts are included. For privacy reasons, job seeker posts will not be included. Although the data is public, do not want to create an easily… See the full description on the dataset page: https://huggingface.co/datasets/brusic/hacker-news-who-is-hiring-posts.hacker-news
Hacker News Dataset
Dataset Summary
This dataset is derived from the official Hacker News data provided via the Hacker News Firebase API. It contains user-generated content including stories, comments, and metadata from the Hacker News platform.
Hacker News is a social news website focusing on computer science, entrepreneurship, and technology. The dataset captures real-world discussions, technical conversations, and community interactions over time.
The data was… See the full description on the dataset page: https://huggingface.co/datasets/TheFinAI/hacker-news.hacker-news-corpus-2007-2022
Hacker News corpus, 2007-Nov 2022
Dataset Description
Dataset Summary
Dataset Name: Hacker News Full Corpus (2007 - November 2022)
Description:
NOTE: I am not affiliated with Y Combinator.
This dataset is a July 2023 snapshot of YCombinator's BigQuery dump of the entire archive of posts and comments made on Hacker News. It contains posts from Hacker News' inception in 2007 through to November 16, 2022, when the BigQuery database was last updated.
The dataset… See the full description on the dataset page: https://huggingface.co/datasets/jkeisling/hacker-news-corpus-2007-2022.hackernews_hiring_postsThis dataset contains postings and comments from the following recurring threads on Hacker News
Ask HN: Who is hiring?
Ask HN: Who wants to be hired?
Freelancer? Seeking freelancer?
These post types are stored in datasets called hiring, wants_to_be_hired and freelancer respectively.
Each type of posting has occurred on a regular basis for several years. You can identify when each comment/listing was added through the CommentTime field. The ParentTitle also indicates the date of the parent… See the full description on the dataset page: https://huggingface.co/datasets/dansbecker/hackernews_hiring_posts.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/Fakeepsbiz/hacker-news.hacker_news_with_comments
Dataset Card for [Dataset Name]
Dataset Summary
Hacker news until 2015 with comments. Collect from Google BigQuery open dataset. We didn't do any pre-processing except remove HTML tags.
Supported Tasks and Leaderboards
Comment Generation; News analysis with comments; Other comment-based NLP tasks.
Languages
English
Data Fields
[More Information Needed]
Data Splits
[More Information Needed]
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/Linkseed/hacker_news_with_comments.hacker-newsThis repository contains the datasets for hacker news, used by https://github.com/anantn/hn-chatgpt-plugin
As of June 2025, these are now exported as parquet files instead of sqlite for space efficiency
hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/IFthisisrealitynbds/hacker-news.hackernews-stories
A HackerNews Stories dataset
This dataset is based on nixiesearch/hackernews-comments dataset:
for each item of type=story we downloaded the target URL. Out of ~3.8M stories ~2.1M are still reachable.
each story HTML was parsed using trafilatura library
we store article text in markdown format along with all page-specific metadata.
Dataset stats
date coverage: xx.2006-09.2024, same as in upstream nixiesearch/hackernews-comments dataset
total scraped pages: 2150271… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/hackernews-stories.hacker-news-dataset
Hacker News Dataset (2025)
Dataset Description
A comprehensive dataset of Hacker News content from 2025, containing stories, comments, users, and their relationships. This dataset enables deep analysis of technical discussions, trends, and community dynamics on one of the most influential technology forums.
Dataset Summary
Total Records: 38.4M+ across 10 tables
Stories: 287K+ submissions including links, Show HNs, Ask HNs
Comments: 2.5M+ discussion… See the full description on the dataset page: https://huggingface.co/datasets/typedef-ai/hacker-news-dataset.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/chrismarchetta/hacker-news.hackernews-stories
Dataset Card for "hackernews-stories"
More Information needed
Pile-HackerNews-0.5B-6K-opt
Dataset Card for "Pile-HackerNews-0.5B-6K-opt"
More Information needed
hacker-news-scraped-storiespile-hackernews
Dataset Creation Process
These subsets were created by streaming over the rows from monology/pile-uncopyrighted and filtering by the meta column. Each subset is generally limited to the first 100,000 qualifying rows encountered.
Citations
If you use this dataset, please cite the original Pile papers:
@article{gao2020pile,
title={The Pile: An 800GB dataset of diverse text for language modeling},
author={Gao, Leo and Biderman, Stella and Black, Sid and Golding, Laurence and… See the full description on the dataset page: https://huggingface.co/datasets/timaeus/pile-hackernews.