Datasetpublic

Mobiusi/fineweb-diverse-insights

improved-fineweb Dataset Description The 'fineweb-diverse-insights' dataset is designed to provide a rich repository of informative articles spanning various topics, including environmental science, health, and space exploration. This dataset aims to enhance the diversity and relevance of content available for natural language processing tasks. Key features include a well-structured JSON format, comprehensive metadata covering text, dates, languages, authorship… See the full description on the dataset page: https://huggingface.co/datasets/Mobiusi/fineweb-diverse-insights.

sourceHugging Facecc-by-nc-4.0updated 1y agoView on Hugging Face

1likes5downloads

Dataset Card

improved-fineweb

Dataset Description

The 'fineweb-diverse-insights' dataset is designed to provide a rich repository of informative articles spanning various topics, including environmental science, health, and space exploration. This dataset aims to enhance the diversity and relevance of content available for natural language processing tasks. Key features include a well-structured JSON format, comprehensive metadata covering text, dates, languages, authorship, and source types, and improved language scoring for better quality assessment. The dataset has been regularly updated to ensure the validity of URLs and has introduced additional metadata to enrich the contextual information surrounding each entry. Its application scenarios include research in machine learning, content generation, and educational tools. The dataset is particularly beneficial for training models in information retrieval and summarization tasks. This dataset only contains sample data. For more data please visit https://www.mobiusi.com or contact us at contact@mobiusi.com.

Usage

python

from datasets import load_dataset

dataset = load_dataset("Mobiusi/improved-fineweb")

Created: 2025-09-02 14:57:29