reapxdev/lemmy-scraper
Lemmy Scraper · Communities, Posts, Comments & Instances Scrape Lemmy communities, posts, comments, and instance metadata across the Fediverse. Structured data export by instance domain, community name, and sort type. Rows in this dataset 2,692 Fields 32 Collector runs behind it 38 Most recent observation 2026-08-04 What this is Every row here was returned by a real run of a public collector. Nothing is generated from a template over a… See the full description on the dataset page: https://huggingface.co/datasets/reapxdev/lemmy-scraper.
118
1---2license: cc-by-4.03language:4- en5pretty_name: "Lemmy Scraper \u00b7 Communities, Posts, Comments & Instances"6size_categories:7- 1K<n<10K8tags:9- "reapx"10- "web-scraping"11- "open-data"12- "lemmy-scraper"13- "lemmy"14- "communities"15- "posts"16- "comments"17- "instances"18configs:19- config_name: default20 data_files:21 - split: train22 path: lemmy-scraper.jsonl23---24 2526 27# Lemmy Scraper · Communities, Posts, Comments & Instances28 29Scrape Lemmy communities, posts, comments, and instance metadata across the Fediverse. Structured data export by instance domain, community name, and sort type.30 31| | |32| --- | --- |33| Rows in this dataset | 2,692 |34| Fields | 32 |35| Collector runs behind it | 38 |36| Most recent observation | 2026-08-04 |37 38## What this is39 40Every row here was returned by a real run of a public collector. Nothing is generated from a41template over a keyword list: a row exists because a run observed it.42 43## Provenance44 45Each row carries `_run_id` and `_dataset_id`, naming the collector run that produced it, so any46row can be traced back to the run that observed it. Rows observed by more than one run are47deduplicated on content; 8 duplicate observations were collapsed.48 49## Files50 51- `lemmy-scraper.jsonl` — one JSON object per row, the canonical form52- `lemmy-scraper.csv` — the same rows flattened; nested values are JSON-encoded within their cell53 so they round-trip54- `dataset.json` — schema.org `Dataset` metadata55 56## Loading it57 58```python59from datasets import load_dataset60ds = load_dataset("reapxdev/lemmy-scraper", split="train")61```62 63## Related64 65- All sources: <https://reapx.dev/data/> · machine-readable index: <https://reapx.dev/llms.txt>66- The collector is a public Apify Actor; agents reach it through <https://mcp.apify.com>67 68## Licence69 70Collected from public sources. This metadata and the published pages are71[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/); the underlying records remain under72the terms of their originating source.73