Substack Scraper - posts, comments & authors
Pricing
from $4.00 / 1,000 posts
Scrape Substack newsletters at scale: full post archives with article text, comments, author profiles, and publication stats like subscriber counts. Works with any Substack URL or custom domain. Fast API-based scraping with no browser, pay per result. Export to CSV, JSON, Excel, or API.
Substack URLs to scrape: publication homepages (subdomain or custom domain), individual post URLs, author profiles (https://substack.com/@handle), and open.substack.com share links. Leave empty only if you enable Discovery mode below.
What to scrape for each publication URL. 'posts' = publication metadata + all posts in the archive. 'publication' = publication metadata only. Post and author URLs are always scraped as posts/authors regardless of this setting ('author' is accepted as an alias of 'publication').
Limit the number of posts scraped per publication (lower = cheaper runs). Set 0 to scrape the ENTIRE archive — large publications can have thousands of posts. Leaving the field empty uses the default of 50.
Fetch each post's full HTML + plain text body. Adds one request per post but gives you the actual article content.
Fetch the full comment thread for each post (adds one request per post). Note: every comment becomes a separate result in the dataset, and popular posts can have hundreds of comments.
Skip paid/subscriber-only posts in publication archives (their bodies can't be fully scraped anyway). Post URLs given directly in Start URLs are always scraped.
Filter each publication archive by a search term. Only posts matching the term are scraped.
Auto-discover publications without providing URLs — seeds from 5 curated top publications and expands through each publication's /recommendations page until the limit below is hit. 'Search' additionally filters every discovered publication's archive by the Discovery query. Off by default: a discovery run scrapes MANY publications and produces a correspondingly large (and more expensive) dataset.
Used with 'Search' Discovery mode: keyword used to filter the archive of each discovered publication. Ignored otherwise.
Cap on how many publications Discovery mode will feed into the scraper (the 5 seed publications count toward it; Start URLs do not). Lower = cheaper runs.
