datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
infini-news-corpus
INFINI-NEWS Corpus
🔎 Search this corpus online: query it with sub-second full-text search and n-gram counts — in the browser or via a public, keyless REST API, no download required — at infini-news.uni-graz.at (API reference).
A multilingual news corpus extracted from
Common Crawl CC-News WARC files.
One row per article, with body text extracted via
trafilatura,
WARC provenance, and derived metadata (publish date, language, topic,
byte hashes) in a single flat schema. Covers… See the full description on the dataset page: https://huggingface.co/datasets/ruggsea/infini-news-corpus.19c_newspapers_images_altoFrench-PD-Newspapers
🇫🇷 French Public Domain Newspapers 🇫🇷
French-Public Domain-Newspapers or French-PD-Newpapers is a large collection aiming to agregate all the French newspapers and periodicals in the public domain.
The collection has been originally compiled by Pierre-Carl Langlais, on the basis of a large corpus curated by Benoît de Courson, Benjamin Azoulay for Gallicagram and in cooperation with OpenLLMFrance. Gallicagram is leading cultural analytics project giving access to word and ngram… See the full description on the dataset page: https://huggingface.co/datasets/PleIAs/French-PD-Newspapers.newswire
Dataset Card for NewsWire
Dataset Summary
NewsWire contains 2.7 million unique public domain U.S. news wire articles, written between 1878 and 1977. Locations in these articles are georeferenced, topics are tagged using customized neural topic classification, named entities are recognized, and individuals are disambiguated to Wikipedia using a novel entity disambiguation model.
Languages
English (en)
Dataset Structure
Each year in the dataset is… See the full description on the dataset page: https://huggingface.co/datasets/dell-research-harvard/newswire.cdx-cc-news
cdx-cc-news
CDXj indexes for Common Crawl News Dataset.
See also News Dataset Announcement
I could not find indexes to Common Crawl News, so I decided to create some.
Then I created a RocksDB Index and a lookup API.
Direct url to API:
https://brian-learns-cc-news-cdx-server.hf.space/lookup
CLI for querying this index and retrieving archived web pages from CC-NEWS Common Crawl WARC files https://github.com/brian-learns/ccnget
Directory Layout
├──… See the full description on the dataset page: https://huggingface.co/datasets/brian-learns/cdx-cc-news.lk-news-chunksfake_news
TODO: Add YAML tags here. Copy-paste the tags obtained with the online tagging app: https://huggingface.co/spaces/huggingface/datasets-tagging
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license:
- unknown
multilinguality:
- monolingual
size_categories:
- 30k<n<50k
source_datasets:
- original
task_categories:
- text-classification
task_ids:
- fact-checking
- intent-classification
pretty_name: GonzaloA / Fake News
Dataset Card for… See the full description on the dataset page: https://huggingface.co/datasets/GonzaloA/fake_news.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/whiskey1983/hacker-news.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/Whiteglove44/hacker-news.hacker-news-rss
Hacker News RSS Feed Directory
TL;DR — We visited every unique domain ever posted to Hacker News, found
which ones publish RSS/Atom feeds, and packaged the results as monthly
parquet snapshots with rich metadata.
623,957 feeds discovered across 1,755,955 hosts,
spanning 232 months from 2006-10 to 2026-03.
Last updated: 2026-04-05T09:21:39Z
Why this exists
RSS is not dead — it's just hard to discover. The <link rel="alternate">
tag that points to a site's feed is… See the full description on the dataset page: https://huggingface.co/datasets/open-index/hacker-news-rss.all-the-news-2-1-Component-one
2.7 million news articles and essays
Dataset Description
2.7 million news articles and essays from 27 American publications. Includes date, title, publication, article text, publication name, year, month, and URL (for some). Articles mostly span from 2016 to early 2020.
Type: CSV
Size: 3.4 GB compressed, 8.8 GB uncompressed
Created by: Andrew Thompson
Date added: 4/3/2020
Date modified: 4/3/2020
source: Component one Datasets 2.7 Millions
Date of Download and processed:… See the full description on the dataset page: https://huggingface.co/datasets/rjac/all-the-news-2-1-Component-one.hacker-news-posts
Hacker News Stories Dataset
This is a dataset containing approximately 4 million stories from Hacker News, exported to a Parquet file. The dataset includes the following fields:
id (int64): The unique identifier of the story.
title (string): The title of the story.
url (string): The URL of the story.
score (int64): The score of the story.
time (int64): The time the story was posted, in Unix time.
comments (int64): The number of comments on the story.
author (string): The… See the full description on the dataset page: https://huggingface.co/datasets/julien040/hacker-news-posts.FineWeb2024
FineWeb-Edu 2024 — Cleaned and Shuffled
This dataset is a year-specific, cleaned, shuffled, and sharded release derived from HuggingFaceFW/fineweb-edu. It contains English educational web text collected in Common Crawl snapshots whose dump identifier belongs to 2024.
This is not a news-only dataset. The year refers to the Common Crawl capture year, not necessarily the page's publication year.
Dataset summary
Item
Value
Year
2024
Rows
162,500,784… See the full description on the dataset page: https://huggingface.co/datasets/Yahoo-Finance-News/FineWeb2024.fenrix-pm-decision-news
FENRIX PM Decision News
A high-volume, portfolio-manager decision dataset containing economic, industry-relevant, market-context, SEC, and single-stock-moving information for use in the FENRIX PM decision game.
The dataset is organized around the question a portfolio manager would face:
Given the information available at the time, should a PM buy, sell, reallocate, or do nothing?
This repo is not just a raw news dump. It includes a professor-facing product tree, a… See the full description on the dataset page: https://huggingface.co/datasets/Scottswi/fenrix-pm-decision-news.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/naveenreddie-18/hacker-news.hacker-news
Hacker News posts and comments
This is a dataset of all HN posts and comments, current as of November 1, 2023.
daily_newsFineWeb2025
FineWeb-Edu 2025 — Cleaned and Shuffled
This dataset is a year-specific, cleaned, shuffled, and sharded release derived from HuggingFaceFW/fineweb-edu. It contains English educational web text collected in Common Crawl snapshots whose dump identifier belongs to 2025.
This is not a news-only dataset. The year refers to the Common Crawl capture year, not necessarily the page's publication year.
Dataset summary
Item
Value
Year
2025
Rows
99,022,205… See the full description on the dataset page: https://huggingface.co/datasets/Yahoo-Finance-News/FineWeb2025.IndustryCorpus2_news_media
IndustryCorpus2: Media & Journalism
This repository contains the IndustryCorpus2: Media & Journalism domain subset of BAAI/IndustryCorpus2.
Refer to the parent dataset card for data construction, intended use, limitations,
and licensing details.
Citation
If you use this dataset in your work, please cite IndustryCorpus2:
@misc{shi2024industrycorpus2,
title = {IndustryCorpus2},
author = {Xiaofeng Shi and Lulu Zhao and Hua Zhou and Donglin Hao},
year… See the full description on the dataset page: https://huggingface.co/datasets/BAAI/IndustryCorpus2_news_media.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/nikhilambhure00/hacker-news.FineWeb-2023
FineWeb-Edu 2023 — Cleaned and Shuffled
This dataset is a year-specific, cleaned, shuffled, and sharded release derived from HuggingFaceFW/fineweb-edu. It contains English educational web text collected in Common Crawl snapshots whose dump identifier belongs to 2023.
This is not a news-only dataset. The year refers to the Common Crawl capture year, not necessarily the page's publication year.
Dataset summary
Item
Value
Year
2023
Rows
104,280,950… See the full description on the dataset page: https://huggingface.co/datasets/Yahoo-Finance-News/FineWeb-2023.europeana_newspapers
Dataset Card for Europeana Newspapers
Dataset Overview
This dataset contains historic newspapers from Europeana, processed and converted to a format more suitable for machine learning and digital humanities research. In total, the collection contains approximately 32 billion tokens across multiple European languages, spanning from the 18th to the early 20th century.
Created by the BigLAM initiative, this unofficial version extracts text content from ALTO XML and… See the full description on the dataset page: https://huggingface.co/datasets/biglam/europeana_newspapers.SP_500_Stocks_Data-ratios_news_price_10_yrsHi folks,
Here is a collection of data I have scraped or aggregated for most of the stocks in the S&P 500, including popular ones like Apple (AAPL).
It has the following data:
Daily news articles and sentiments on those articles collected over the last few years.
All quarterly stock fundamentals (ratios) for 10-20 years.
Stock price data (daily close) over the last 10-20 years.
Use it however you please for PERSONAL USAGE, but if you do leverage it to make some money; just remember me and… See the full description on the dataset page: https://huggingface.co/datasets/pmoe7/SP_500_Stocks_Data-ratios_news_price_10_yrs.hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/MurphyA/hacker-news.twice_kr_news_retrieval
FinNews-Retrieval-ko
Constructed a Retrieval dataset based on Korean financial news articles.
nordjylland-news-summarization
Dataset Card for "nordjylland-news-summarization"
Dataset Summary
This dataset consists of pairs containing text and corresponding summaries extracted from the Danish newspaper TV2 Nord.
Supported Tasks and Leaderboards
Summarization is the intended task for this dataset. No leaderboard is active at this point.
Languages
The dataset is available in Danish (da).
Dataset Structure
An example from the dataset looks as… See the full description on the dataset page: https://huggingface.co/datasets/alexandrainst/nordjylland-news-summarization.hmd_newspapers
Dataset Card for Heritage Made Digital Newspapers
Dataset Summary
This dataset contains text extracted at the article level from historic digitised newspapers from the Heritage Made Digital newspaper digitisation program at the British Library. The newspapers in the dataset were published between 1800 and 1896. This dataset contains ~2.5 billion tokens and 3,065,408 articles.
The dataset contains text generated from Optical Character Recognition software on digitised… See the full description on the dataset page: https://huggingface.co/datasets/biglam/hmd_newspapers.Indian_Financial_Newscrypto-news-coindesk-2020-2025
CoinDesk Cryptocurrency News Dataset (2020–2025)
This dataset contains cryptocurrency-related news articles sourced from CoinDesk Data, accessed programmatically via the CryptoCompare API. The dataset is curated and published for academic and research purposes, with a focus on analyzing the relationship between news and cryptocurrency market dynamics.
Time Period
January 1, 2020 – January 1, 2025
Content Overview
Each record in the dataset… See the full description on the dataset page: https://huggingface.co/datasets/maryamfakhari/crypto-news-coindesk-2020-2025.europeana-fr-news_beirThis is a copy of https://huggingface.co/datasets/jinaai/europeana-fr-news reformatted into the BEIR format. For any further information like license, please refer to the original dataset.
Disclaimer
This dataset may contain publicly available images or text data. All data is provided for research and educational purposes only. If you are the rights holder of any content and have concerns regarding intellectual property or copyright, please contact us at "support-data (at) jina.ai"… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/europeana-fr-news_beir.