Substack Scraper: newsletter posts, authors and engagement

Export posts from any Substack newsletter, on a *.substack.com address or a custom domain, as clean, flat rows for Excel, Google Sheets, BI tools or your database. Every row includes the title, subtitle, URL, publish date, free/paid audience, author names, likes, comments, restacks, word count and cover image. You can also add the full plain text of free posts. You can also search all of Substack by keyword.

It uses Substack's own public JSON endpoints, so it needs no login, no cookies, no proxies and no browser. It costs $2 per 1,000 posts.

What does the Substack Scraper do?

It turns a list of Substack publications (or a search query) into a dataset with one row per post:

• publication name, URL, subdomain, main author and public subscriber badge (e.g. "Over 1,200,000 subscribers")
• post title, subtitle, URL, publish date and post type (newsletter, podcast, thread...)
• audience (free / paid) and an isPaid flag
• author names (all bylines, including guest authors)
• like count (reactions), comment count and restack count
• word count and cover image
• the public preview text, and the full body as plain text (HTML stripped) when you turn on includeBody

Public data only. Paid posts are never unlocked. For them you get the metadata and the public preview, exactly what a logged-out visitor sees.

Who is it for?

• Newsletter creators and growth teams. Benchmark competitors: posting cadence, which topics get the most likes and comments, how long their posts are, and how much they put behind the paywall.
• Marketers and PR teams. Find the newsletters that write about your category, see who the authors and guest writers are, and build outreach and sponsorship lists.
• Investors and analysts. Track what influential writers in finance, tech or politics publish, and feed it to dashboards or alerts.
• Researchers, journalists and AI/NLP teams. Build text datasets from free posts for topic modeling, sentiment analysis or RAG, with engagement numbers attached.
• Monitoring. Schedule a daily run to catch every new post from a set of newsletters.

Why use this Substack scraper?

• Low price. $2 per 1,000 posts, less than half the price of the leading alternative. You pay only for rows saved, and you can set a maximum charge per run.
• Reliable. It reads Substack's JSON API instead of parsing HTML, retries automatically on rate limits (HTTP 429) and server errors, and one broken publication never stops the run.
• Works with custom domains. lennysnewsletter.com, astralcodexten.com and others work just like *.substack.com addresses. Subdomain redirects are followed for you.
• Flat, analysis-ready rows. Authors come as a simple list, engagement as plain numbers, and body text as clean plain text. The Overview table opens straight in the Console.
• Polite. Requests are sequential with a short pause between them, and it only uses paths that Substack's robots.txt allows.
• Fast. The default 2-publication run finishes in seconds.

How to scrape Substack

1. Click Try for free (or Start) on this page.
2. Add publications: a subdomain (astralcodexten), a URL (https://lenny.substack.com) or a custom domain (https://www.lennysnewsletter.com). Any post URL also works.
3. Set Max posts per publication (newest posts first).
4. Optionally turn on Include full text, or enter a Search query to search all of Substack.
5. Click Start, then download the results as CSV, Excel, JSON, XML or HTML, or read them via the API.

Input examples

The 50 latest posts from two newsletters, with full text:

{
  "publications": ["https://www.lennysnewsletter.com", "astralcodexten"],
  "maxPostsPerPublication": 50,
  "includeBody": true
}

Search all of Substack for a topic:

{
  "searchQuery": "AI agents",
  "maxPostsPerPublication": 100
}

Field	What it does	Example
publications	Substack subdomains, URLs, custom domains or post URLs	astralcodexten, https://www.lennysnewsletter.com
maxPostsPerPublication	Max posts per publication, newest first (default 20). Also caps search results	100
includeBody	Add plain-text body: full text for free posts, preview only for paid posts (default false)	true
searchQuery	Optional keyword search across all of Substack (posts)	product management

Output sample

{
  "publication": "Astral Codex Ten",
  "publicationUrl": "https://www.astralcodexten.com",
  "publicationSubdomain": "astralcodexten",
  "publicationAuthor": "Scott Alexander",
  "publicationSubscribers": null,
  "postId": 221215977,
  "title": "Why Global Trepanation?",
  "subtitle": "...",
  "url": "https://www.astralcodexten.com/p/why-global-trepanation",
  "postDate": "2026-10-21T21:53:11.440Z",
  "audience": "free",
  "isPaid": false,
  "postType": "newsletter",
  "authors": ["Scott Alexander"],
  "likeCount": 337,
  "commentCount": 192,
  "restackCount": 10,
  "wordCount": 1307,
  "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/73a7ca5d-..._2340x1316.jpeg",
  "previewText": "...",
  "bodyText": "A friend asked me to investigate the following mystery: why do 5-10% of skulls from 10,000 BC to 3,000 BC...",
  "scrapedAt": "2026-10-23T04:40:12.118Z"
}

Field	Type	Description
publication, publicationUrl, publicationSubdomain	string	Newsletter name, home URL and substack.com subdomain
publicationAuthor	string	Main author of the publication
publicationSubscribers	string	Substack's public subscriber badge, when the publication shows one
postId	number	Substack post ID
title, subtitle	string	Post title and subtitle
url	string	Canonical post URL
postDate	string	Publish date (ISO 8601)
audience, isPaid	string, boolean	free or paid (paid subscribers / founding members only)
postType	string	newsletter, podcast, thread...
authors	array	Byline names
likeCount, commentCount, restackCount	number	Engagement counts
wordCount	number	Length of the full post (also for paid posts)
coverImage	string	Cover image URL
previewText	string	Public preview / description
bodyText	string	Plain-text body when includeBody is on: full for free posts, preview for paid posts
scrapedAt	string	When the row was collected (ISO 8601)

How much does it cost to scrape Substack?

This Actor uses pay-per-event pricing:

• $0.002 per post saved ($2 per 1,000 posts)
• $0.00005 per run start (per GB of memory, a fraction of a cent; the default 256 MB run is charged about $0.0000125)

Worked example: the latest 200 posts from each of 10 newsletters is 2,000 posts × $0.002 = $4.00, plus well under a cent for the run start. The default example (2 newsletters × 10 posts) costs about $0.04, so the $5 of monthly free credit on the Apify Free plan covers about 2,500 posts.

Set Max posts per publication or a maximum charge per run, and the Actor stops as soon as it reaches the limit.

Integrations and API

Call the Actor from your own code with the official apify-client, or connect it to Make, Zapier, n8n, Google Sheets, Slack, webhooks and more from the Integrations tab.

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("deanko/substack-scraper").call(run_input={
    "publications": ["https://www.lennysnewsletter.com", "platformer"],
    "maxPostsPerPublication": 50,
})
for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(post["publication"], post["title"], post["likeCount"], post["commentCount"])

JavaScript / Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('deanko/substack-scraper').call({
    searchQuery: 'climate tech',
    maxPostsPerPublication: 100,
    includeBody: true,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.length, 'posts');

You can also run it with one HTTP request through the run-sync-get-dataset-items endpoint and get the rows back in the response.

FAQ

Do I need a Substack account or login? No. The Actor reads only public data, the same JSON that Substack's own website loads for a logged-out visitor.

Can it get the full text of paid posts? No, by design. For paid posts you get the title, metadata, engagement numbers, word count and the public preview, and isPaid is true. Paywalled content is never accessed.

Does it work with custom domains? Yes. Enter the custom domain (e.g. https://www.lennysnewsletter.com) or the *.substack.com address, and redirects are followed automatically.

What does "likes" mean? likeCount is Substack's reaction count (the heart button) for the post.

How does search work? searchQuery uses Substack's public site search and returns matching posts from any publication, in Substack's relevance order. Notes, comments and profiles are skipped.

How do I get only new posts every day? Schedule the Actor (Console → Schedules) with a small maxPostsPerPublication, such as 5. Posts come newest first, so you can de-duplicate on postId in your sheet or database.

Is scraping Substack allowed? The Actor only uses paths that Substack's robots.txt allows for all crawlers, sends requests one at a time with pauses, and collects only publicly available data. You are responsible for how you use the data. Respect authors' copyright, and check the laws that apply to you, especially for personal data.

Limitations

• No paywalled content. Paid posts return only the public preview.
• Comments themselves (text, commenters) and Substack Notes are not included, only the comment count.
• Search returns posts only and follows Substack's own ranking. Very broad queries return the top matches, not every post.
• Subscriber numbers are Substack's public badge text, not exact counts, and many publications don't show one.
• Engagement numbers are a snapshot at the time of the run.

Legal and privacy

This Actor is not affiliated with or endorsed by Substack Inc. It collects only data that is publicly visible without logging in. You are responsible for using the data in line with Substack's terms, authors' copyright and the laws that apply to you.

Support and feedback

Missing a field you need, or found a bug? Open an issue in the Issues tab on this page. If the Actor saves you time, a short review helps other people find it.