Substack Newsletter Scraper
Pricing
from $1.99 / 1,000 results
Substack Newsletter Scraper
Substack Newsletter scraper extracts publicly available newsletter posts, titles, authors, publication dates, subscriber-facing content, and metadata π°π Perfect for content research, trend analysis, competitive intelligence, and newsletter monitoring.
Pricing
from $1.99 / 1,000 results
Rating
0.0
(0)
Developer
Scrapers Hub
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
2 days ago
Last modified
Categories
Share
An Apify Actor that scrapes Substack newsletters β author profile, publication metadata, leaderboard ranking, and posts (optionally with full body text).
Built with requests + parsel + regex, with curl_cffi (Chrome impersonation) as an automatic fallback whenever a plain request gets blocked.
Input
| Field | Type | Default | Description |
|---|---|---|---|
newsletterUrls | array | β | List of newsletter URLs, e.g. [{ "url": "https://www.lennysnewsletter.com" }]. Custom domains and *.substack.com both work. |
postScrapingMode | string | information_and_content | information_only (metadata only) or information_and_content (also fetch full post text). |
maxPostsPerNewsletter | integer | 12 | Max posts scraped per newsletter. |
delayBetweenRequests | integer | 3000 | Delay between HTTP requests, in milliseconds. |
maxRetries | integer | 2 | Retries per request before giving up (falls back to curl_cffi). |
{"newsletterUrls": [{ "url": "https://www.lennysnewsletter.com" },{ "url": "https://andrewsullivan.substack.com" }],"postScrapingMode": "information_and_content","maxPostsPerNewsletter": 12,"delayBetweenRequests": 3000,"maxRetries": 2}
Output
One dataset item per newsletter: author fields (author_*), publication fields
(publication_*), leaderboard ranking, all_publications, and a posts array where
each post has url, title, subtitle, date, author, likes, comments_count,
content_preview, content, and word_count.
How it works
- Fetch the newsletter homepage and extract the embedded
window._preloadsJSON (regex +parsel) β publication object. - Fetch the author's public profile
(
substack.com/api/v1/user/{handle}/public_profile) β follower count, socials, leaderboard, all publications. - List posts via
/api/v1/archive, and (in content mode) fetch each post'sbody_htmlvia/api/v1/posts/{slug}, then strip it to plain text.
Run locally
pip install -r requirements.txt# via the Apify CLI (recommended):apify run# or drive the scraper directly without the Apify SDK:python _local_test.py