Substack Scraper: Posts & Newsletters

Get the posts of any Substack newsletter as clean, structured data: title, subtitle, date, author, likes, comment count, restacks, word count, tags, cover image, podcast link, paid/free status and, if you want it, the full text of free posts. Works with *.substack.com publications and custom domains. No login, no browser, fast. $0.90 per 1,000 posts (less on paid plans), the same price with or without full text, and no start fee.

What you can do with it

• Track newsletters you follow or compete with: what they publish, how often, and which posts get the most likes and comments.
• Research a niche: pull the most popular posts of a set of publications (Most popular first) to see what resonates.
• Feed an LLM or a search index: collect the full text of free posts from the newsletters you choose.
• Monitor topics: keep only posts that mention your keywords, or use each publication's own archive search.
• Build a reading list or digest: newest posts since a date across many publications in one dataset.

Input

Field	What it does
publications	Subdomains (astralcodexten), name.substack.com, custom domains (www.lennysnewsletter.com) or any link to a publication or post.
sort	new (newest first) or top (most popular first).
includeContent	Adds bodyText for free posts (default off). includeHtml also adds bodyHtml.
searchWithinPublication	Uses each publication's own archive search.
keywords	Keep only posts that mention one of these. You pay only for posts that match.
dateFrom	Only posts published on or after this date.
maxPostsPerPublication	Cap per publication (default 25).
maxItems	Hard cap for the whole run, and your cost (default 50, 0 = no limit).

Example:

{ "publications": ["astralcodexten", "www.lennysnewsletter.com"], "sort": "top", "includeContent": true, "maxPostsPerPublication": 50 }

Output

One item per post:

{
  "postId": 210686641,
  "title": "Your Book Review: We Present Television",
  "subtitle": "Finalist #11 in the Book Review Contest",
  "url": "https://www.astralcodexten.com/p/your-book-review-we-present-television",
  "postDate": "2026-10-10T01:05:13.124Z",
  "type": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "wordcount": 8651,
  "likes": 134,
  "commentCount": 91,
  "restacks": 7,
  "tags": [],
  "authors": [{ "name": "Scott Alexander", "handle": "astralcodexten" }],
  "publication": { "name": "Astral Codex Ten", "subdomain": "astralcodexten", "customDomain": "www.astralcodexten.com", "author": "Scott Alexander" },
  "bodyText": "…full text when includeContent is on and the post is free…"
}

Field	Meaning
postId, slug, url	Substack's post id, slug and canonical link.
title, subtitle, description, preview	Headline, subtitle, short description and the public preview text.
postDate, type, section, tags, language	Publication date (UTC), newsletter / podcast / thread, section and tags.
audience, isPaywalled	everyone for free posts, only_paid / founding for paid ones.
likes, reactions, commentCount, restacks, wordcount	Engagement and length.
coverImage, podcastUrl, podcastDurationSec	Media links when present.
authors	Name and Substack handle of each byline.
publication	Name, subdomain, custom domain, main author, tagline, logo, created date, whether paid plans are on.
bodyText, bodyHtml	Full post body, free posts only, when requested.
source, input	api (or rss fallback) and the publication you entered.

How to use

1. Enter one or more publications (a name like astralcodexten or a domain).
2. Pick the order, and turn on full text if you need it.
3. Run, then download the dataset as JSON, CSV or Excel, or read it through the Apify API.

Pricing

Pay per result: $0.90 per 1,000 posts on the Free plan, down to $0.48 on higher plans (post-scraped), with or without full text, no start fee. Keyword filters and caps apply before you are charged. A run that finds nothing fails instead of silently returning an empty dataset.

Apify plan tier	Free	Bronze	Silver	Gold	Platinum	Diamond
per 1,000 posts	$0.90	$0.80	$0.62	$0.48	$0.48	$0.48

Limits and notes

• Paywalled posts return metadata and the public preview only. The scraper never tries to get around a paywall.
• Very large archives take a few minutes.
• If a publication's full archive cannot be read, you get its latest ~20 posts with fewer fields.
• Comments themselves, Substack Notes, subscriber lists and Substack-wide search are not included.
• Public data only. Respect the authors' copyright: post text belongs to its authors, so check your use (quotation, analysis, internal research) is allowed before republishing it.
• This actor is not affiliated with, endorsed by or sponsored by Substack Inc.