Datasetpublic

vypivshiy/habr

Habr dataset Description Summary: Dataset of posts and comments from habr.com, a Russian collaborative blog about IT, computer science and anything related to the Internet. Script: create_habr.py Point of Contact: Ilya Gusev Languages: Russian, English, some programming code. Usage from datasets import load_dataset dataset = load_dataset('IlyaGusev/habr', split="train", streaming=True) for example in dataset: print(example["text_markdown"])… See the full description on the dataset page: https://huggingface.co/datasets/vypivshiy/habr.

sourceHugging Faceupdated 3mo agoView on Hugging Face

8likes285downloads

Dataset Card

Description

Summary: Dataset of posts and comments from habr.com, a Russian collaborative blog about IT, computer science and anything related to the Internet.

Script: create_habr.py

Point of Contact: Ilya Gusev

Languages: Russian, English, some programming code.

Usage

python

from datasets import load_dataset
dataset = load_dataset('IlyaGusev/habr', split="train", streaming=True)
for example in dataset:
    print(example["text_markdown"])

Data Instances

{
  "id": 12730,
  "language": "ru",
  "url": "https://habr.com/ru/post/12730/",
  "text_markdown": "...",
  "text_html": "...",
  "lead_markdown": "...",
  "lead_html": "...",
  "type": "article",
  "labels": [],
  "original_author": null,
  "original_url": null,
  "time_published": 1185962380,
  "author": "...",
  "title": "Хочешь в университет — сделай презентацию",
  "statistics": {
    "commentsCount": 23,
    "favoritesCount": 1,
    "readingCount": 1542,
    "score": 7,
    "votesCount": 15,
    "votesCountPlus": 11,
    "votesCountMinus": 4
  },
  "hubs": [
    "itcompanies"
  ],
  "flows": [
    "popsci"
  ],
  "tags": [
    "PowerPoint",
    "презентация",
    "абитуриенты",
  ],
  "reading_time": 1,
  "format": null,
  "complexity": null,
  "comments": [
    {
      "id": 11653537,
      "parent_id": null,
      "level": 0,
      "time_published": 1185963192,
      "score": -1,
      "votes": 1,
      "message_html": "...",
      "message_markdown": "...",
      "author": "...",
      "children": [11653541]
    },
    {
      "id": 11653541,
      "parent_id": 11653537,
      "level": 1,
      "time_published": 1185967886,
      "score": 0,
      "votes": 0,
      "message_html": "...",
      "message_markdown": "...",
      "author": "...",
      "children": []
    }
  ]
}

Source Data

—The data source is the Habr website.
—API call example: post 709430.
—Processing script is here.

Personal and Sensitive Information