sayurio/bangla-site-scrape

Bangladeshi Web Scraped Dataset

Table of Contents

• —Dataset Description
• —Dataset Structure
• —Data Fields
• —Usage
• —Licensing
• —Citation

Request More Scrapes Order Private Scrapes

Dataset Description

This dataset is a comprehensive collection of scraped web data from various Bangladeshi websites. It is designed to facilitate natural language processing (NLP) tasks for the Bengali language, including analysis of e-commerce trends, news classification, and general text generation.

The data is provided in JSONL (JSON Lines) format, making it easy to parse and integrate into machine learning pipelines.

Key Features

• —Diverse Sources: Includes data from e-commerce platforms, news portals, blogs, and informational articles.
• —Rich Metadata: E-commerce entries include specific fields such as pricing, product descriptions, and specifications.
• —Open Access: Free to download and use under the MIT License.

Dataset Structure

Each line in the JSONL file represents a single scraped page or item. The structure varies slightly depending on the source type (e.g., e-commerce vs. news article).

General Structure

{
  "url": "[https://example.com/article](https://example.com/article)",
  "domain": "example.com",
  "scrape_date": "2023-10-27",
  "category": "news",
  "content": "Full text content of the article...",
  "title": "Article Headline"
}

Gemini said Here is a README.md template tailored for your Hugging Face dataset.

README.md Markdown --- license: mit task_categories:

• —text-generation
• —text-classification language:
• —bn size_categories:
• —10K<n<100K pretty_name: Bangladeshi Web Scraped Dataset tags:
• —e-commerce
• —news
• —articles
• —bangladesh
• —web-scraping ---

Bangladeshi Web Scraped Dataset

Table of Contents

• —Dataset Description
• —Dataset Structure
• —Data Fields
• —Usage
• —Licensing
• —Citation

Dataset Description

This dataset is a comprehensive collection of scraped web data from various Bangladeshi websites. It is designed to facilitate natural language processing (NLP) tasks for the Bengali language, including analysis of e-commerce trends, news classification, and general text generation.

The data is provided in JSONL (JSON Lines) format, making it easy to parse and integrate into machine learning pipelines.

Key Features

• —Diverse Sources: Includes data from e-commerce platforms, news portals, blogs, and informational articles.
• —Rich Metadata: E-commerce entries include specific fields such as pricing, product descriptions, and specifications.
• —Open Access: Free to download and use under the MIT License.

Dataset Structure

Each line in the JSONL file represents a single scraped page or item. The structure varies slightly depending on the source type (e.g., e-commerce vs. news article).

General Structure

{
  "url": "[https://example.com/article](https://example.com/article)",
  "domain": "example.com",
  "scrape_date": "2023-10-27",
  "category": "news",
  "content": "Full text content of the article...",
  "title": "Article Headline"
}

Data Fields

You can load this dataset directly using the Hugging Face datasets library:

Python

from datasets import load_dataset

dataset = load_dataset("your-username/your-dataset-name")

# Inspect the first example
print(dataset['train'][0])

Potential Use Cases

-Bengali NLP: Training language models on contemporary Bangladeshi web text.

• —Sentiment Analysis: Analyzing news or product reviews.
• —Market Analysis: Tracking e-commerce pricing and product trends in Bangladesh.
• —Classification: Categorizing web content by domain or topic.

Licensing

This dataset is released under the MIT License.

Disclaimer: The data contained in this dataset is scraped from publicly available websites. Users are responsible for ensuring that their use of this data complies with the terms of service of the source websites and applicable local laws.