Substack Scraper

Every post from any Substack newsletter — with likes, comments, authors, and the full text of free posts — as clean JSON.

Substack hosts tens of thousands of newsletters and podcasts, from Lenny's Newsletter and Simon Willison's Newsletter to the Dwarkesh Podcast, on substack.com addresses and on their own custom domains. This Actor reads any of them and returns their posts, newest first: title, subtitle, publish and update dates, authors, likes, comments, restacks, word count, cover image, podcast audio, and the post body as HTML and plain text. Free posts come in full; paid posts come with the preview Substack shows non-subscribers, and are flagged as such. Every row also carries the newsletter's name, URL, author and subscriber count. Paste a newsletter URL; there is no login, no API key, and no proxy to configure. Built for media monitoring, creator and sponsorship research, content analysis, and AI and search datasets.

⚡ Quick start

{
  "startUrls": [{ "url": "https://www.lennysnewsletter.com/" }],
  "maxItems": 5
}

🧩 Input

Field	Type	Required	Default	Description
startUrls	array	yes	—	Substack newsletter URLs (<name>.substack.com or the newsletter's own domain), archive pages (…/archive), single post URLs (…/p/<post>), or open.substack.com/pub/… links.
maxItems	integer	no	—	Maximum number of posts per start URL, newest first. The input form starts at 5; leave it empty to get the whole archive.
includeContent	boolean	no	true	Read each post's body as HTML and plain text. Adds one fast request per post; turn it off for quick archive listings.
proxyConfiguration	object	no	{ "useApifyProxy": false }	Proxy settings. Substack newsletters need no proxy.

Custom domains work. Many newsletters run on their own domain (for example www.lennysnewsletter.com). Paste it as it is: the Actor checks that the site is a Substack newsletter and says so in the log when it is not.

📤 Output

Each post is one dataset row, with the newsletter it belongs to.

Fields

Field	Type	Description
id	number	Substack post ID.
title	string	Post title.
subtitle	string	Post subtitle.
slug	string	The post's name in its URL.
url	string	The post's public page.
postDate	string	When the post was published (ISO 8601).
updatedAt	string	When the post was last edited (ISO 8601). Filled when includeContent is on.
type	string	newsletter, podcast or thread (a discussion thread).
audience	string	Who can read it: everyone for free posts, only_paid for subscriber-only posts.
isPaid	boolean	Whether the post is for paying subscribers.
wordcount	number	Word count of the full post.
reactionCount	number	Number of likes.
commentCount	number	Number of comments.
restacks	number	Number of restacks (Substack's reshares).
coverImage	string	Cover image URL.
sectionName	string	The newsletter section the post belongs to, when it uses sections.
tags	array	Tag names.
language	string	Language of the post.
authors	array	Bylines: id, name, handle, photoUrl and bio.
podcastUrl	string	Audio file of a podcast episode.
podcastDuration	number	Episode length in seconds.
bodyHtml	string	Post body as HTML. null when includeContent is off.
bodyText	string	Post body as plain text. null when includeContent is off.
isContentTruncated	boolean	true when the body is the paywall preview of a paid post, false when it is the full post, null when includeContent is off.
publicationName	string	The newsletter's name.
publicationUrl	string	The newsletter's home page.
publication	object	The newsletter: id, name, url, subdomain, customDomain, description, authorName, authorHandle, logoUrl, subscriberCount (when the newsletter shows it), paidSubscriptions, firstPostDate and language.

💡 Use cases

• Media monitoring — follow the newsletters in your industry and catch every new post, with its engagement.
• Creator and sponsorship research — compare newsletters by subscribers, posting cadence, likes and comments before you pitch or sponsor.
• Content analysis — collect full post text for topic tracking, sentiment, and trend research.
• AI and search datasets — build clean, attributed corpora from free newsletter posts and podcast show notes.

❓ FAQ

• Do I need a Substack account or an API key? No. The Actor reads the same public pages any visitor sees.
• Can I get the full text of paid posts? No. Paid posts come with the preview Substack shows non-subscribers; isContentTruncated is true on those rows, and wordcount still gives the length of the full post.
• Does it work with newsletters on their own domain? Yes. Paste the domain as it is. If the site is not a Substack newsletter, the log says so and the URL is skipped.
• Can I scrape many newsletters at once? Yes. Add one URL per newsletter to startUrls; maxItems applies to each one separately.
• Why is subscriberCount empty for some newsletters? Authors choose whether to show their subscriber count. When they hide it, the field is null.
• Do I need a proxy? No. It runs without one by default.

🔗 More scrapers by Anyx

• Web Content Scraper
• Telegram Channel Scraper
• Bluesky Scraper
• Google Search Scraper

🤝 Anyx Solutions

We build custom scrapers and data-extraction pipelines.

• Email: tantosthor@gmail.com

🖼 Image credit

Image credit: substack.com