Substack Newsletter Scraper avatar

Substack Newsletter Scraper

Pricing

from $1.99 / 1,000 results

Go to Apify Store
Substack Newsletter Scraper

Substack Newsletter Scraper

Substack Newsletter scraper extracts publicly available newsletter posts, titles, authors, publication dates, subscriber-facing content, and metadata πŸ“°πŸ“Š Perfect for content research, trend analysis, competitive intelligence, and newsletter monitoring.

Pricing

from $1.99 / 1,000 results

Rating

0.0

(0)

Developer

Scrapers Hub

Scrapers Hub

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

2 days ago

Last modified

Share

An Apify Actor that scrapes Substack newsletters β€” author profile, publication metadata, leaderboard ranking, and posts (optionally with full body text).

Built with requests + parsel + regex, with curl_cffi (Chrome impersonation) as an automatic fallback whenever a plain request gets blocked.

Input

FieldTypeDefaultDescription
newsletterUrlsarray–List of newsletter URLs, e.g. [{ "url": "https://www.lennysnewsletter.com" }]. Custom domains and *.substack.com both work.
postScrapingModestringinformation_and_contentinformation_only (metadata only) or information_and_content (also fetch full post text).
maxPostsPerNewsletterinteger12Max posts scraped per newsletter.
delayBetweenRequestsinteger3000Delay between HTTP requests, in milliseconds.
maxRetriesinteger2Retries per request before giving up (falls back to curl_cffi).
{
"newsletterUrls": [
{ "url": "https://www.lennysnewsletter.com" },
{ "url": "https://andrewsullivan.substack.com" }
],
"postScrapingMode": "information_and_content",
"maxPostsPerNewsletter": 12,
"delayBetweenRequests": 3000,
"maxRetries": 2
}

Output

One dataset item per newsletter: author fields (author_*), publication fields (publication_*), leaderboard ranking, all_publications, and a posts array where each post has url, title, subtitle, date, author, likes, comments_count, content_preview, content, and word_count.

How it works

  1. Fetch the newsletter homepage and extract the embedded window._preloads JSON (regex + parsel) β†’ publication object.
  2. Fetch the author's public profile (substack.com/api/v1/user/{handle}/public_profile) β†’ follower count, socials, leaderboard, all publications.
  3. List posts via /api/v1/archive, and (in content mode) fetch each post's body_html via /api/v1/posts/{slug}, then strip it to plain text.

Run locally

pip install -r requirements.txt
# via the Apify CLI (recommended):
apify run
# or drive the scraper directly without the Apify SDK:
python _local_test.py