CoolFace
Datasetpublic

logiover/sitemap-to-url-crawler-sample-data

Sitemap to URL Crawler nstantly extract all public URLs from any website's sitemap.xml recursively. Handles nested sitemap indexes automatically. The fastest & cheapest way to build URL lists for RAG pipelines, LLM training, and SEO audits. Zero-config & blazing fast. What the actor scrapes Sitemap to URL Crawler — RAG & AI Data Feeder Extract every public URL from any website's sitemap.xml — recursively, instantly, and at scale. Handles nested sitemap… See the full description on the dataset page: https://huggingface.co/datasets/logiover/sitemap-to-url-crawler-sample-data.

sourceHugging Facecc-by-4.0updated 4mo agoView on Hugging Face
0likes24downloads
Dataset Card

Sitemap to URL Crawler

nstantly extract all public URLs from any website's sitemap.xml recursively. Handles nested sitemap indexes automatically. The fastest & cheapest way to build URL lists for RAG pipelines, LLM training, and SEO audits. Zero-config & blazing fast.

What the actor scrapes

Sitemap to URL Crawler — RAG & AI Data Feeder Extract every public URL from any website's sitemap.xml — recursively, instantly, and at scale. Handles nested sitemap indexes automatically. The fastest and most cost-efficient way to build complete URL lists for RAG pipelines, LLM training datasets, SEO audits, and content inventories. Zero configuration required. --- ## What Is This Actor? A sitemap is an XML file that lists every page a website wants search engines to discover. Almost eve

Who this is for

This sample is useful for anyone working in data extraction, seo tools, developer tools who wants a quick look at the shape of the data before running the live actor at scale. Treat it as a static snapshot — counts, prices and timestamps reflect the moment the sample was captured and will drift over time.

Source

Live source: This dataset is a static sample. For fresh, customizable extractions, run the live Apify actor: logiover/sitemap-to-url-crawler

Fields

  • `url` — canonical URL of the item
  • `lastmod` — last-modified timestamp from sitemap
  • `changefreq` — sitemap change frequency hint
  • `priority` — sitemap priority hint (0-1)
  • `sourceSitemap` — URL of the parent sitemap

Sample preview

json
[
  {
    "url": "https://apify.com/",
    "lastmod": null,
    "changefreq": null,
    "priority": null,
    "sourceSitemap": "https://apify.com/sitemap/pages.xml"
  },
  {
    "url": "https://apify.com/about",
    "lastmod": null,
    "changefreq": null,
    "priority": null,
    "sourceSitemap": "https://apify.com/sitemap/pages.xml"
  }
]

Snapshot stats

  • Row count in this sample: 100
  • Data quality flag at capture time: EXISTING_USED

License & Attribution

Released under Creative Commons Attribution 4.0 (CC BY 4.0). If you use this data, please credit the source actor and link back to its Apify Store page: logiover/sitemap-to-url-crawler.

The sample is provided as-is for evaluation and research. Source websites retain their own terms of service — downstream users are responsible for complying with them when running the live actor or redistributing scraped content.