datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
political_bias_in_news_articlesgerman-wikipedia-articleskurdish-wikipedia-articles
Summary
Extracted from the wikidump. There are summaries and categories available for each article. Will look into adding them later.
Usage
from datasets import load_dataset
ds = load_dataset("nazimali/kurdish-wikipedia-articles", split="train")
ds
Dataset({
features: ['id', 'url', 'title', 'text'],
num_rows: 63076
})
financial-news-articles
Dataset Card for "financial-news-articles"
More Information needed
The data was obtained from here
pubmed_articles_domain10x_20250227note-articles
note articles
note.com の公開ページから抽出した本文と IPADIC による MeCab 解析結果です。
各行は text と、構造化されたトークン列 mecab を持ちます。有料記事は公開されている範囲のみです。
pubmed_articles_edu3_20250227ticker_analysis_articlesnews-articles-ptbr-dataset
Dataset Card for "news-articles-ptbr-dataset"
More Information needed
pubmed-2019-pythia-word-tfidf-pubmedqa-clean-articlespubmed-2019-pythia-word-tfidf-invfreq-pubmedqa-clean-articlesvital-articles-da-wiki
Vital Articles Danish Wikipedia Dataset
Overview
Total articles: 28,006
Files: 29 Parquet shards
Language: Danish (da)
Contents
Each row is one article with these fields:
en_title: English Wikipedia title
da_title: Danish Wikipedia title
da_url: Danish Wikipedia article URL
markdown: Article content in markdown format
markdown_chars: Character count of markdown
source_lang: Source language code (da)
fetched_at_utc: UTC timestamp when the article was fetched
ai-tech-articles
AI/Tech Dataset
This dataset is a collection of AI/tech articles scraped from the web.
It's hosted on HuggingFace Datasets, so it is easier to load in and work with.
To load the dataset
1. Install HuggingFace Datasets
pip install datasets
2. Load the dataset
from datasets import load_dataset
dataset = load_dataset("siavava/ai-tech-articles")
# optionally, convert it to a pandas dataframe:
df = dataset["train"].to_pandas()
You do not need to clone… See the full description on the dataset page: https://huggingface.co/datasets/siavava/ai-tech-articles.medium-articles-posts-with-content
Medium Articles Dataset Generator
This project combines multiple datasets from Kaggle and Hugging Face to create a comprehensive collection of Medium articles. The combined dataset is available on Hugging Face Hub.
Dataset Description
This dataset is a unique compilation that not only combines multiple sources but also ensures data quality through normalization and deduplication. A key feature is that all entries in the text column are unique - there are no duplicate… See the full description on the dataset page: https://huggingface.co/datasets/Alaamer/medium-articles-posts-with-content.TheArabicPile_Articles
The Arabic Pile
Introduction:
The Arabic Pile is a comprehensive dataset meticulously designed to parallel the structure of The Pile and The Nordic Pile. Focused on the Arabic language, the dataset encompasses a vast array of linguistic nuances, incorporating both Modern Standard Arabic (MSA) and various Levantine, North African, and Egyptian dialects. Tailored for the training and fine-tuning of large language models, the dataset consists of 13 subsets, each uniquely… See the full description on the dataset page: https://huggingface.co/datasets/premio-ai/TheArabicPile_Articles.pubmed_articlessamorzad-gov-pl-articles
Artykuły z platformy samorzad.gov.pl
Wersja: v0.2
Zbiór zawiera 81 418 artykułów opublikowanych na stronach 247 instytucji korzystających ze wspólnej platformy samorzad.gov.pl. Są to między innymi urzędy gmin i powiatów, szkoły, instytucje pomocy społecznej i instytucje kultury.
W wersji v0.2 usunięto dane osobowe i kontaktowe z pól tekstowych przeznaczonych dla odbiorcy. Usunięte wartości zastąpiono jednoznacznymi znacznikami, zachowując układ i znaczenie pozostałej treści.… See the full description on the dataset page: https://huggingface.co/datasets/dawidmajewski/samorzad-gov-pl-articles.all-the-news-2-pythia-tfidf-invfreq-topic-stratified-v1-articlesmedium_articles
Dataset Card for "medium_articles"
More Information needed
all-the-news-2-pythia-tfidf-topic-stratified-v1-articlesfinancial-news-articles-filtereddataset_info:
features:
- name: title
dtype: string
- name: text
dtype: string
- name: url
dtype: string
- name: word_count
dtype: int64
splits:
- name: train
num_bytes: 554834105.9892601
num_examples: 199711
download_size: 459025008
dataset_size: 554834105.9892601
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
europe_pmc_articles_part_2
Dataset Card for "europe_pmc_articles_part_2"
More Information needed
article_summarizationClean-Wikipedia-English-Articles
Dataset Card for DragonLLM/Clean-Wikipedia-English-Articles
DragonLLM is delighted to announce the release of the cleanest markdown extract of Wikipedia articles so far, a high-quality resource to train your LLMs!
Dataset Summary
The Clean-Wikipedia-English-Articles dataset contains the comprehensive bodies of English articles, i.e. without appendices like References, See also, Bibliography, etc.
It has been pointed out here that the Wikimedia Wikipedia dataset… See the full description on the dataset page: https://huggingface.co/datasets/OVHaiLLM/Clean-Wikipedia-English-Articles.europe_pmc_articles_part_1
Dataset Card for "europe_pmc_articles_part_1"
More Information needed
wikipedia-Only-Indo-Articles-About-IndoLiterally the title. It is processed from wikimedia/wikipedia. Only entries that contain the case-insensitive string 'Indonesia' is included. Now, you can use this to train a model on information about Indonesia written in Indonesian.
Additional filtering was done to remove entries that are either too short or too long.
Dataset Statistics
Split
Total Word Count
Mean Word Count… See the full description on the dataset page: https://huggingface.co/datasets/afrizalha/wikipedia-Only-Indo-Articles-About-Indo.mteb-nl-news-articles-clsThis dataset contains Dutch news articles along with their corresponding categories, sourced from the Nederlandse Oproep Stichting.
Citation Information
If you find our paper, benchmark or models helpful, please consider cite as follows:
@misc{banar2025mtebnle5nlembeddingbenchmark,
title={MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch},
author={Nikolay Banar and Ehsan Lotfi and Jens Van Nooten and Cristina Arhiliuc and Marija Kliocaite and Walter Daelemans}… See the full description on the dataset page: https://huggingface.co/datasets/clips/mteb-nl-news-articles-cls.justicio-BOE-A-1978-31229-constitucion-by-articles-qa-qa-groq_llama3_70b_8192-sas
Dataset summary
It is an end-to-end evaluation dataset (using SAS metric) for Justicio.
Domain: Legal, Law, Spanish Constitution
Language: Spanish
SAS summary
The concept of Semantic Answer Similarity refers to the evaluation of the semantic similarity between the generated answer and the ground truth. The score ranges from 0 to 1. A higher score indicates a better match between the generated answer and the ground truth.
Justicio summary
Justicio is a… See the full description on the dataset page: https://huggingface.co/datasets/dariolopez/justicio-BOE-A-1978-31229-constitucion-by-articles-qa-qa-groq_llama3_70b_8192-sas.enwiki-articles-html-2410chatgpt-news-articles
Dataset Card for "chatgpt-news-articles"
Dataset Summary
The ChatGPT CNN / DailyMail Dataset is a small sample of the original CNN / DailyMaily English-language dataset containing 25k unique news articles. For each corresponding article written by journalists at CNN and the Daily Mail, there is an article written by ChatGPT using the highlights provided by human annotators. The current version supports can be used to study the language comparison between human and ChatGPT… See the full description on the dataset page: https://huggingface.co/datasets/isarth/chatgpt-news-articles.