Datasetpublic

domofon/reddit-2026

2026 DUMP ! reddit-pretrain-xml Private pretrain-document dump of public Reddit mirrors. One JSONL row is one document. The trainer should read text only. text format <doc> <meta subreddit="changemyview" date="2026-08-01" /> <title>...</title> <body> ... </body> </doc> <|endoftext|> <|endoftext|> is SmolLM2-1.7B eos (token id 0). Do not wrap these docs in Chat SFT. XML special characters in title/body are escaped. Markdown and emoji are kept. If… See the full description on the dataset page: https://huggingface.co/datasets/domofon/reddit-2026.

sourceHugging Faceotherupdated 23d agoView on Hugging Face

0likes67downloads

Dataset Card

2026 DUMP !

reddit-pretrain-xml

Private pretrain-document dump of public Reddit mirrors. One JSONL row is one document. The trainer should read text only.

`text` format

<doc>
<meta subreddit="changemyview" date="2026-08-01" />
<title>...</title>
<body>
...
</body>
</doc>
<|endoftext|>

<|endoftext|> is SmolLM2-1.7B eos (token id 0). Do not wrap these docs in Chat SFT.

XML special characters in title/body are escaped. Markdown and emoji are kept. If the body started with a copy of the title, the duplicate was stripped. Thread fragments with no real title omit the <title> tag.

Sidecar JSON fields (not inside text): id, source (reddit), subreddit, date (ISO UTC), url when present, char_len of the body. No score/nsfw/type/author/comments.

Dropped: empty posts, body shorter than 50 characters, image/video-only, [deleted] / [removed]. Then exact hash dedup and MinHash LSH (Jaccard 0.8).

The scrape was stopped mid-run (about 285 of 1178 listed channels finished). Not a complete Reddit snapshot.

Converted dump: 7,025,087 scraped posts → 3,713,744 documents in 8 data/train-*.jsonl.gz shards (~1.2 GB compressed). Dropped mostly image/video-only (~2.0M) plus short/empty bodies.