CoolFace
Datasetpublic

tshasan/multi-label-web-categorization

Multi-Label Web Page Classification Dataset Dataset Description The Multi-Label Web Page Classification Dataset is a curated dataset containingweb page titles and snippets, extracted from the CC-Meta25-1M dataset. Each entry has been automatically categorized into multiple predefined categories using ChatGPT-4o-mini. This dataset is designed for multi-label text classification tasks, making it ideal for training and evaluating machine learning models in web… See the full description on the dataset page: https://huggingface.co/datasets/tshasan/multi-label-web-categorization.

sourceHugging Facecc-by-4.0updated 1y agoView on Hugging Face
1likes23downloads
Dataset Card

Multi-Label Web Page Classification Dataset

Dataset Description

The Multi-Label Web Page Classification Dataset is a curated dataset containingweb page titles and snippets, extracted from the CC-Meta25-1M dataset. Each entry has been automatically categorized into multiple predefined categories using ChatGPT-4o-mini.

This dataset is designed for multi-label text classification tasks, making it ideal for training and evaluating machine learning models in web content classification and natural language processing (NLP).

Source & Processing

  • —Source: The dataset is derived from CC-Meta25-1M, a multilingual web dataset.
  • —Classification: ChatGPT-4o-mini was used to classify each web page into one or more categories.
  • —Categories: The dataset includes the following 11 categories:
  • —News
  • —Entertainment
  • —Shop
  • —Chat
  • —Education
  • —Government
  • —Health
  • —Technology
  • —Work
  • —Travel
  • —Uncategorized (for ambiguous cases)

Each record includes metadata features, such as the number of assigned categories and the text lengths of the title and snippet.

Dataset Structure

Format: CSV

The dataset is provided as a single CSV file with the following columns:

Column NameTypeDescription
urlstringWeb page URL
titlestringPage title
snippetstringShort excerpt of webpage
languagestringLanguage code (e.g., en, cs, ru, de)
categorystringComma-separated list of assigned categories
category_liststringList representation of categories (e.g., ["Work", "Education"])
num_categoriesintNumber of assigned categories
title_lengthintCharacter count of the title
snippet_lengthintCharacter count of the snippet

Usage & Applications

This dataset is well-suited for:

  • —Training multi-label text classification models.
  • —Developing content recommendation systems.
  • —Enhancing search engine categorization.
  • —Analyzing web content distributions across languages.

To load the dataset using Hugging Face’s datasets library:

python
from datasets import load_dataset

dataset = load_dataset("tshasan/multi-label-web-classification")
print(dataset[0])  # Display the first sample

License

This dataset is licensed under CC BY 4.0, allowing free use and modifications with proper attribution.

Acknowledgments

  • —tshasan – for creating the CC-Meta25-1M dataset.
  • —Deepseek – for providing Deepseek-v3, which powered the classification.
  • —[Common Crawl](https://commoncrawl.org/): A large-scale web crawl dataset providing openly available web data.