Substack Scraper - Newsletter Posts, Authors and Engagement

Scrape newsletter posts from any Substack publication, on a substack.com subdomain or on a custom domain, or find publications by keyword first. Each post becomes one flat dataset item with title, subtitle, authors, publication date, audience (free or paid), word count, likes, comments, restacks, tags, cover image and, optionally, the post text.

The Actor reads the public JSON API that every Substack publication serves (/api/v1/archive, /api/v1/posts/<slug>) and Substack's public publication search. No login, no browser.

What it does

• Takes one or more publications (homepage URL, subdomain, custom domain or any post URL) and returns their posts, newest first or most popular first.
• Optional keyword search (searchQuery) finds publications on Substack and scrapes their posts too.
• Optional postedAfter filter: an absolute date (2026-01-31) or a relative period (30 days, 3 months).
• Optional post text (includeContent): full text of free posts; for paid posts, only the public preview shown before the paywall.
• Posts are deduplicated by postId.

Input

Parameter	Type	Required	Default	Description
publications	array of strings	Yes, unless searchQuery is set	—	Publications to scrape: https://www.lennysnewsletter.com, https://astralcodexten.substack.com, astralcodexten, newsletter.pragmaticengineer.com or any post URL of the publication.
searchQuery	string	No	—	Keyword to find publications with Substack's publication search (e.g. product management).
maxPublications	integer	No	5	Number of publications taken from the search (1-100).
sort	new or top	No	new	Newest posts first, or the publication's most popular posts first.
includeContent	boolean	No	false	Adds contentText (one extra request per post).
postedAfter	string	No	—	Only posts published after this date (2026-01-31, 7 days, 2 weeks, 3 months, 1 year).
maxItems	integer	No	20	Maximum posts per publication (1-5000).

Example:

{
    "publications": ["https://www.lennysnewsletter.com", "astralcodexten"],
    "sort": "new",
    "postedAfter": "30 days",
    "includeContent": true,
    "maxItems": 50
}

Output

One dataset item per post (real example, contentText shortened):

{
    "postId": 170393023,
    "publication": "Climate Town",
    "publicationUrl": "https://www.climatetown.news",
    "title": "We Made a Graph",
    "subtitle": null,
    "description": "This one might be easier for Joe Rogan to understand.",
    "slug": "we-made-a-graph",
    "url": "https://www.climatetown.news/p/we-made-a-graph",
    "postType": "newsletter",
    "authors": "Matt Nelsen, Rollie Williams",
    "postDate": "2025-08-20T13:02:59.406000+00:00",
    "audience": "everyone",
    "isPaywalled": false,
    "wordCount": 1986,
    "reactionCount": 108,
    "commentCount": 31,
    "restacks": 20,
    "coverImage": "https://substackcdn.com/image/fetch/$s_!rHfm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8609ef9-99c8-4046-80ef-a38c40690946_1600x966.jpeg",
    "tags": "climate change, climate, mel gibson, bernie sanders, science, joe rogan",
    "language": "en",
    "contentText": "Hello friend!\nHere at Climate Town we almost never make our own graphs. ...",
    "scrapedAt": "2026-10-28T06:52:06.846700+00:00"
}

Field	Description
postId	Substack post id
publication, publicationUrl	Publication name and homepage (custom domain when it has one)
title, subtitle, description, slug, url	Post identity
postType	newsletter, podcast, thread or video
authors	Public byline names, comma-separated
postDate	Publication date, ISO 8601 UTC
audience, isPaywalled	everyone (free), only_paid or founding; isPaywalled is true when the full post is for paid subscribers
wordCount	Word count of the full post, paid posts included
reactionCount, commentCount, restacks	Likes, comments and restacks
coverImage, tags, language	Cover image URL, tags (comma-separated), language code
contentText	Plain text of the post, only with includeContent (paid posts: public preview only)
scrapedAt	Time of extraction

Pricing

Pay-per-event, no start fee: one POST event per post returned. Proposed grid, in USD per 1,000 posts:

Event	Free	Starter	Scale	Business	Enterprise
POST	$0.80	$0.68	$0.60	$0.48	$0.48

The run stops cleanly when the maximum cost per run you set is reached.

Limitations

• Paid posts: only the public preview (the part before the paywall) is available in contentText; wordCount still reports the full length. The Actor does not log in.
• Comments are counted (commentCount) but not extracted, and subscriber counts are not returned.
• Profile URLs (substack.com/@handle) and Substack Notes are not supported; give the publication URL instead.
• The keyword search returns the publications Substack's own search ranks first (about 20 per page); maxPublications caps how many are used.
• sort: top with postedAfter filters the popular posts by date; it may need to read a large part of the archive for old publications.
• Publication names come from the publication homepage; when it cannot be read, the host name is used.