Substack Newsletter, Post & Creator Scraper

Get Substack posts, full text, publication profiles and comments, and find newsletters by keyword or category leaderboard, in one actor. It reads the same public JSON API Substack's own website uses, so it needs no login and works on custom domains (www.lennysnewsletter.com) as well as name.substack.com.

What you get

• Posts: title, subtitle, date, audience (free or paid), word count, reactions, comments, restacks, cover image, authors, tags, URL. Switch on includeContent for the plain text of every post (and optionally the HTML).
• Publications: name, author, tagline, paid tier on or off, monthly and yearly price when public, the subscriber label Substack shows ("Over 11,000 subscribers"), launch date, leaderboard rank.
• Discovery: search publications by keyword, or list the top / paid / trending publications of any of the ~33 Substack categories.
• Comments: text, date, reactions, thread position, commenter display name.

Free posts return the full article text. Paid posts return the public preview only, flagged with textIsPreview: true. This actor never logs in and never gets past a paywall.

Pricing

Pay per event. No start fee.

Event	Price	Cost per 1,000
Post (metadata, and full text if you ask for it)	$0.0005	$0.50
Publication	$0.001	$1.00
Comment	$0.0002	$0.20

Example: 20 latest posts with full text from 50 newsletters = 1,000 posts + 50 publication rows = $0.55. Full text costs the same as metadata only. Platform usage (compute, proxy) is included.

Input

Field	Type	Default	Notes
mode	posts / search / leaderboard	posts	What to do.
publications	list	astralcodexten, https://www.lennysnewsletter.com/	Mode posts. Subdomain, x.substack.com, custom domain, or any URL on the publication.
queries	list	machine learning	Mode search. Keywords or a newsletter name.
categories	list	Technology	Mode leaderboard. Name, slug or ID (Technology, Business, Finance, Science, Culture, Crypto...).
leaderboardType	all / paid / trending	all	Mode leaderboard.
includeContent	bool	false	Add plain text of each post. One extra request per post.
includeHtml	bool	false	With includeContent, also add body HTML.
includeComments	bool	false	One comment row per comment, threads flattened.
maxCommentsPerPost	int	100
sinceDate	YYYY-MM-DD	none	Skip older posts; stops paging early.
sort	new / top	new	Post order inside a publication.
maxPostsPerPublication	int	50
maxPublications	int	25	Per query / per category.
postsPerDiscoveredPublication	int	0	Search and leaderboard: also pull the latest N posts of every publication found.
includePublicationRow	bool	true	Set false to get posts only.
maxItems	int	500	Hard cap on all rows. Also respects your max-spend setting.

Example (posts with full text):

{ "mode": "posts", "publications": ["astralcodexten", "https://www.lennysnewsletter.com/"],
  "maxPostsPerPublication": 20, "includeContent": true }

Example (find sponsorship targets): { "mode": "leaderboard", "categories": ["Finance"], "leaderboardType": "paid", "maxPublications": 50 }

Output

Post row:

{
  "type": "post", "id": 215694124, "publication": "Lenny's Newsletter", "publicationId": 10845,
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "postDate": "2026-09-29T13:15:57.512Z", "postType": "newsletter", "audience": "everyone",
  "wordcount": 1276, "reactionCount": 293, "commentCount": 8, "restacks": 5,
  "authors": ["Lenny Rachitsky"], "tags": [],
  "text": "Hey there, I'm Lenny. Each week, I share deeply researched product, growth, and career advice...",
  "textIsPreview": false, "scrapedAt": "2026-10-31T01:58:47Z"
}

Publication row:

{
  "type": "publication", "id": 3393884, "name": "9-To-Thrive", "subdomain": "thrivewithcarrie",
  "url": "https://thrivewithcarrie.substack.com", "authorName": "Carrie Loranger",
  "hasPaidTier": true, "monthlyPrice": 97, "yearlyPrice": 300, "currency": "USD",
  "subscriberCountLabel": "Over 12,000 subscribers", "launchedLabel": "Hundreds of paid subscribers",
  "firstPostDate": "2025-03-08T14:02:43.350Z", "leaderboardRank": null, "profileSource": "search",
  "scrapedAt": "2026-10-31T02:02:31Z"
}

A run summary (status per input, HTTP counts) is saved as the OUTPUT record of the run's key-value store.

Use cases

• Sponsorship buyers: rank newsletters in a category by subscriber label, paid tier and posting frequency.
• PR and outreach: find who writes about a topic, with author name, tagline and URL.
• Newsletter marketers: study what titles, lengths and cadences get reactions in your niche.
• AI and research datasets: bulk plain text of free posts with dates, authors and tags.
• Monitoring: schedule it with sinceDate to pick up new posts from a watchlist.

FAQ

Does it get paid posts? Only the public preview Substack shows non-subscribers. Those rows have textIsPreview: true. Subscriber counts? Substack does not publish exact counts. You get the label it shows ("Over 11,000 subscribers") when there is one, and not every publication has one. Which publications can I scrape? Any public publication, including ones on custom domains. Why do some publication rows say profileSource: "post"? Substack's profile search sometimes returns nothing for a publication. The actor then rebuilds a lighter profile from the publication's latest post: name, author, tagline, paid tier on or off, but no prices or subscriber label. Personal data? Only public newsletter data. Comments carry the commenter's display name only, no handle or ID.

Limits

• Pace is about 2 requests per second per host, so 1,000 posts with full text take roughly 8 to 10 minutes.
• Substack rate limits busy IPs; the actor retries on a fresh proxy IP and reports any input that still fails in the run summary.
• The leaderboard shows the top 25 per page; very deep pages may end early.
• Search returns the publications Substack ranks for your query, not an exhaustive list. Category is only filled in leaderboard mode.
• Comments on very popular posts are large downloads; use maxCommentsPerPost.