Website Crawler – Whole-Site Text, Emails & Contacts
Pricing
from $0.80 / 1,000 results
Give it a website, get every page as clean Markdown plus the contacts found on each: emails, social profiles, WhatsApp, contact page, tech stack. Follows internal links and the sitemap, respects robots.txt, capped by pages and depth. Fast HTTP crawl, no browser, flat price per page.
One or more site homepages (full URLs with https://). Each site is crawled separately, up to 50 sites per run.
Stop after this many pages on a site. You pay per page delivered.
0 = only the start page, 1 = pages linked from it, and so on.
Also follow links to blog.example.com, shop.example.com, etc.
Optional. Keep only URLs containing one of these texts, e.g. ["/docs/"]. The start page is always included.
Optional. Skip URLs containing any of these texts, e.g. ["/tag/", "?page="].
Seed the crawl from sitemap.xml when the site has one - finds pages that are not linked from the homepage.
Skip pages the site asks crawlers not to visit. Leave on.
Markdown keeps headings, lists, links and code; plain text is just the words.
Longer content is cut here and marked truncated.
Business emails (role addresses only), social profiles, WhatsApp, contact page URL, tech stack - from every page, no extra cost.
Add the list of links found on each page (up to 500).
