CoolFace
Datasetpublic

Nanthasit/github-docs

GitHub Docs Corpus A dataset containing only information from GitHub — the official github/docs repository, i.e. the source of docs.github.com. Dataset Structure Files: data/train.jsonl Format: JSONL, one chunk per line Columns: text (cleaned doc chunk), metadata (source, title) Rows: 3,336 Composition Source: github/docs (main branch), content/ tree only — 3,734 Markdown files covering GitHub features, workflows, webhooks, REST/GraphQL API docs… See the full description on the dataset page: https://huggingface.co/datasets/Nanthasit/github-docs.

sourceHugging Facecc-by-sa-4.0updated 2mo agoView on Hugging Face
0likes101downloads
Dataset Card

GitHub Docs Corpus

A dataset containing only information from GitHub — the official github/docs repository, i.e. the source of docs.github.com.

Dataset Structure

  • —Files: data/train.jsonl
  • —Format: JSONL, one chunk per line
  • —Columns: text (cleaned doc chunk), metadata (source, title)
  • —Rows: 3,336

Composition

  • —Source: github/docs (main branch), content/ tree only — 3,734 Markdown files covering GitHub features, workflows, webhooks, REST/GraphQL API docs, actions, security, and more.
  • —Processing: YAML frontmatter stripped (title kept), Liquid version tags ({% ifversion %}), images and HTML comments removed, markdown links flattened to their text, chunks split at heading boundaries (~2,000 chars).
  • —Excludes: files outside content/, assets, and pages with fewer than 40 chars of body text.

Loading

python
from datasets import load_dataset

ds = load_dataset("Nanthasit/github-docs", split="train")