datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wikipedia_en
wikipedia_en
This is a curated Wikipedia English dataset for use with the II-Commons project.
Dataset Details
Dataset Description
This dataset comprises a curated Wikipedia English pages. Data sourced directly from the official English Wikipedia database dump. We extract the pages, chunk them into smaller pieces, and embed them using Snowflake/snowflake-arctic-embed-m-v2.0. All vector embeddings are 16-bit half-precision vectors optimized for cosine indexing… See the full description on the dataset page: https://huggingface.co/datasets/Intelligent-Internet/wikipedia_en.wikipedia-pageviews
Wikipedia Article Pageviews
This repository automatically fetches and aggregates the 100 most popular Wikipedia articles by pageviews - creating a dataset that enables tracking trending topics on Wikipedia.
It works by polling the WikiMedia API on a daily basis and fetching the top 100 most popular articles from two days ago.
The fetcher runs in a scheduled GitHub Actions workflow, which is available here.
The dataset begins in the year 2016 and the textual data is presented as… See the full description on the dataset page: https://huggingface.co/datasets/vtasca/wikipedia-pageviews.Wikipedia_contradict_benchmark
Wikipedia contradict benchmark
Wikipedia contradict benchmark is a dataset consisting of 253 high-quality, human-annotated instances designed to assess LLM performance when augmented with retrieved passages containing real-world knowledge conflicts. The dataset was created intentionally with that task in mind, focusing on a benchmark consisting of high-quality, human-annotated instances.
Dataset Details
Dataset Description… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/Wikipedia_contradict_benchmark.passages_wikipediawikipedia_quality_wikirankDatasets with WikiRank quality score as of 1 August 2024 for 47 million Wikipedia articles in 55 language versions (also simplified version for each language in separate files).
The WikiRank quality score is a metric designed to assess the overall quality of a Wikipedia article. Although its specific algorithm can vary depending on the implementation, the score typically combines several key features of the Wikipedia article.
Why It’s Important
Enhances Trust: For readers and… See the full description on the dataset page: https://huggingface.co/datasets/lewoniewski/wikipedia_quality_wikirank.wikipedia-22-12-simple-embeddings
wikipedia-22-12-simple-embeddings
A modified version of Cohere/wikipedia-22-12-simple-embeddings
meant for use with PostgreSQL with pgvector and Timescale Vector.
Dataset Details
This dataset was created for exploring time-based filtering and semantic search in PostgreSQL with pgvector and Timescale Vector.
This is a modified version of the Cohere wikipedia-22-12-simple-embeddings dataset hosted on Huggingface.
It contains embeddings of Simple English Wikipedia… See the full description on the dataset page: https://huggingface.co/datasets/timescale/wikipedia-22-12-simple-embeddings.wikipedia-movies
Wikipedia Movie Plots with Images.
30,000+ movies plot descriptions and images.
Plot summary descriptions of movies scrapped from Wikipedia.
Dataset is subset of this dataset.
Content
The dataset contains descriptions of 34,886 movies from around the world. Column descriptions are listed below:
Release Year - Year in which the movie was released
Title - Movie title
Origin/Ethnicity - Origin of movie (i.e. American, Bollywood, Tamil, etc.)
Director - Director(s)
Genre -… See the full description on the dataset page: https://huggingface.co/datasets/Coder-Dragon/wikipedia-movies.wikipedia-image-requests
Daily image-request counts for Wikipedia article images
Daily counts of how often each of ~1.19 million Wikipedia article images was requested from
Wikimedia's image servers, attributed to the wiki whose page the request came from, together with
the article each image appears on.
406,263,728 daily observations covering 1,186,150 image files across 534,067 articles in
nine Wikipedias, from 2025-09-01 to 2026-09-05 (370 days).
Every file in the set appears on exactly one… See the full description on the dataset page: https://huggingface.co/datasets/lgelauff/wikipedia-image-requests.cantonese-wikipediaWikipedia-Corpora-Report
Dataset Card for "Wikipedia-Corpora-Report"
This dataset is used as a metadata database for the online WIKIPEDIA CORPORA META REPORT dashboard that illustrates how humans and bots generate or edit Wikipedia editions and provides metrics for “pages” and “edits” for all Wikipedia editions (320 languages). The “pages” metric counts articles and non-articles, while the “edits” metric tallies edits on articles and non-articles, all categorized by contributor type: humans or bots. The… See the full description on the dataset page: https://huggingface.co/datasets/SaiedAlshahrani/Wikipedia-Corpora-Report.wikipedia-2023-11-kikongo-lingala-cohere-multilingual-v3habr_and_wikipedia1gb Russian-English dataset containing articles from Habr and Wikipedia.
WikipediaSearchwikipedia_character_abstractsWikipedia-Articles
Dataset Card for "BrightData/Wikipedia-Articles"
Dataset Summary
Explore a collection of millions of Wikipedia articles with the Wikipedia dataset, comprising over 1.23M structured records and 10 data fields updated and refreshed regularly.
Each entry includes all major data points such as timestamp, URLs, article titles, raw and cataloged text, images, "see also" references, external links, and a structured table of contents.
For a complete list of data points, please… See the full description on the dataset page: https://huggingface.co/datasets/BrightData/Wikipedia-Articles.grokipedia-wikipedia-16-languages
Dataset description
This dataset contains a mapping between Grokipedia v0.1 article pages and the corresponding Wikipedia article titles across 16 language editions (based on Wikipedia and Wikidata dumps from 1 November 2025). Each record includes:
The URL of the Grokipedia page: grokipedia_url
Wikipedia titles in the following languages (if they exist): ar (Arabic), de (German), en (English), es (Spanish), fa (Persian), fr (French), it (Italian) , ja (Japanese), nl (Dutch), pl… See the full description on the dataset page: https://huggingface.co/datasets/lewoniewski/grokipedia-wikipedia-16-languages.wikipedia-citation-indexDataset with citation indexes as of 1 August 2024 for 47 million Wikipedia articles in 55 language versions. Research: ArXiv
wikipedia-summary-dataset-128k
Wikipedia Summary Dataset 128k
This is random subsample of 128k entries from the wikipedia summary dataset, processed with the following code:
import pandas as pd
df = pd.read_parquet('wikipedia-summary.parquet')
df['l'] = df['summary'].str.len()
rdf = df[(df['l'] > 300) & (df['l'] < 600)]
# Filter out any rows 'topic' that have non-alphanumeric characters
mask = rdf['topic'].str.contains(r'^[a-zA-Z0-9 ]+$') == True
rdf = rdf[mask == True].sample(128000)[['topic'… See the full description on the dataset page: https://huggingface.co/datasets/54rt1n/wikipedia-summary-dataset-128k.herbal-knowledge-embedding-wikipedia
Herbs Domain Knowledge Safety Wikipedia Embeddings
Pre-computed vector embeddings from Wikipedia articles covering Assorted Herbs, spices, and other botanical items used for alternative medicine — ready to drop into your RAG pipeline without any embedding overhead.
Dataset Details
Property
Details
Embedding Model
nomic-ai/nomic-embed-text-v1.5 (135M)
Vector Dimensions
768
Source
Wikipedia
Topics
herbs, plants, spices, and other botanical items for… See the full description on the dataset page: https://huggingface.co/datasets/rakhasetiawan/herbal-knowledge-embedding-wikipedia.leipzig_en_simple_wikipedia_2021_sentences_100ken_wikipedia_001
Dataset Card for en_wikipedia_001
The en_wikipedia_001 dataset is a collection of crawled paragraph text from Wikipedia on the 28th of April, 2024. It contains high-quality text, stored in multiple documents, available to be used to finetune or train AI models based that the license is followed.
Dataset Details
The dataset was crawled using our web crawler on the 28th of April at an average of 1 page per second as to respect robots.txt rules. Strict licensing must be… See the full description on the dataset page: https://huggingface.co/datasets/orionai/en_wikipedia_001.kusaal-wikipedia-benchmark
Kusaal Wikipedia Benchmark
A frozen 1,000-sentence-pair test set for Kusaal↔English machine
translation, in encyclopedic register — the evaluation set behind the
scores reported on
kusaal-nllb-600M.
Kusaal (kus, Latin script) is a Gur language of northern Ghana and Burkina
Faso, spoken by ~400,000 people and absent from NLLB-200. This is, to date,
the only public MT benchmark for the language.
⚠️ Do not train on this data
This is a held-out test set. It was frozen… See the full description on the dataset page: https://huggingface.co/datasets/PrinceAlhassanNasamu/kusaal-wikipedia-benchmark.fire-knowledge-embedding-wikipedia
Fire Domain Knowledge Safety Wikipedia Embeddings
Pre-computed vector embeddings from Wikipedia articles covering fire hazard safety, environmental effects, and related topics — ready to drop into your RAG pipeline without any embedding overhead.
Dataset Details
Property
Details
Embedding Model
nomic-ai/nomic-embed-text-v1.5 (135M)
Vector Dimensions
768
Source
Wikipedia
Topics
Fire safety, hazard prevention, environmental impact, and more
Format
CSV… See the full description on the dataset page: https://huggingface.co/datasets/rakhasetiawan/fire-knowledge-embedding-wikipedia.tourism-wikipediaagricultural-knowledge-embedding-wikipedia
Agriculture Domain Knowledge Safety Wikipedia Embeddings
Pre-computed vector embeddings from Wikipedia articles covering Agriculture, Planting Methods, and related topics — ready to drop into your RAG pipeline without any embedding overhead.
Dataset Details
Property
Details
Embedding Model
nomic-ai/nomic-embed-text-v1.5 (135M)
Vector Dimensions
768
Source
Wikipedia
Topics
Agriculture, various plants, planting methods and more
Format
CSV (vectors +… See the full description on the dataset page: https://huggingface.co/datasets/rakhasetiawan/agricultural-knowledge-embedding-wikipedia.wikipedia-22-12-en-voyage-embedfood-knowledge-embedding-wikipedia
Food Domain Knowledge Safety Wikipedia Embeddings
Pre-computed vector embeddings from Wikipedia articles covering Food, Cooking Methods, and other items related to food preparation — ready to drop into your RAG pipeline without any embedding overhead.
Dataset Details
Property
Details
Embedding Model
nomic-ai/nomic-embed-text-v1.5 (135M)
Vector Dimensions
768
Source
Wikipedia
Topics
Food, Cooking methods, and other items related to food preparation… See the full description on the dataset page: https://huggingface.co/datasets/rakhasetiawan/food-knowledge-embedding-wikipedia.wikipedia-crypto-articles
Wikipedia Crypto Articles 🪙₿
This dataset is a collection of articles obtained from Wikipedia on January 5ᵗʰ, 2024. It contains two columns, title and article, containing the article's title as it is on the Wikipedia website and the article's content.
The articles vary from specific cryptocurrencies—such as Bitcoin or Ethereum—to historical facts, companies, exchanges, entities, and relevant people in the history of cryptocurrencies.
This dataset can be used to train machine… See the full description on the dataset page: https://huggingface.co/datasets/luisotorres/wikipedia-crypto-articles.Wikipedia_ComputerScienceWikipedia-it-Trame-di-FilmCollection of plots of historical films and adventure films from Italian Wikipedia (April 2024)
Raccolta di trame di film storici e film di avventura da Wikipedia italiana (Aprile 2024)