Substack Scraper

Scrape Substack publications: post archive with titles, dates, likes, comments, paywall status and full text of free posts, plus publication details and comments. No login.

Substack Scraper reads the public JSON endpoints behind every Substack site (.substack.com subdomains and custom domains alike) and turns them into a clean dataset. Give it a newsletter name or URL and get the whole post archive newest first: title, subtitle, publish date, post type (newsletter, podcast, video, thread), likes, comment count, restacks, word count, tags, cover image, audio/video links, public bylines and, for free posts, the full bodyHtml/bodyText. Paywalled posts are marked isPaywalled: true with the public preview only. Switch to publication mode for the newsletter's description, authors, subscriber count, paid plans, sections and total post count, or to comments mode for comment threads with handles, likes and dates. Download JSON, CSV or Excel, or call it from Python, Node, n8n, Make or Google Sheets. You pay only for the items you get; no account or API key is needed.

What data does Substack Scraper extract?

Every item has a type field: post, publication or comment.

Post items (type: "post", mode posts)

Field	Description
id, slug, url, canonicalUrl	Substack post ID, slug and public URL (custom domain when the publication has one)
title, subtitle, description	Headline, dek and SEO description
publication	{id, name, subdomain, url} of the newsletter
authors	Public bylines: [{name, handle, url, isGuest}]
publishedAt, updatedAt	ISO 8601 timestamps (updatedAt needs Include post body)
postType	newsletter, podcast, video or thread
isPaywalled, audience	true when audience is not everyone (only_paid, founding)
likes, reactions	Reaction count and the per-emoji breakdown ({"❤": 191})
commentsCount, childCommentsCount, restacks	Engagement counters
wordCount	Word count as reported by Substack (also for paywalled posts)
coverImageUrl	Cover / social image
audioUrl, audioDurationSec	Podcast episode MP3 and duration
hasVideo, videoUrl, videoUploadId, videoPlaybackId, videoDurationSec	Video posts: public MP4 source link and duration
hasVoiceover	Whether the post has an audio narration
tags, section, language	Post tags, section {name, slug} and language code
previewText	Short public teaser (when the author shows one)
bodyHtml, bodyText, bodyIsPreview	Full body of free posts (HTML and plain text); for paywalled posts only the public preview, flagged with bodyIsPreview: true
scrapedAt	When the item was collected

Publication items (type: "publication", mode publication)

Field	Description
id, name, subdomain, customDomain, url, substackUrl	Identity and both URLs
description, longDescription	Tagline and the longer About text
logoUrl, coverImageUrl	Branding images
authors, primaryAuthor	Contributors [{name, handle, role, isOwner, url}] and the owner's name
subscriberCountText, freeSubscriberCount, freeSubscriberCountApprox	Public subscriber figures (Over 1,200,000 subscribers, 1200000, 1.2M+) when the author shows them
paidSubscriberCountText, bestsellerTier	Tens of thousands of paid subscribers, bestseller badge tier (100/1000/10000)
isPaid, paymentsState, plans	Paid plans [{id, name, interval, amount, currency}]
postsCount	Total posts in the archive
sections, hasPodcast, podcastTitle, podcastFeedUrl	Sections and podcast info
language, publicationType, copyright, firstPostAt, createdAt	Metadata
socialLinks	Links the publication lists on its homepage [{title, url, group}]

Comment items (type: "comment", mode comments or Include comments)

postId, id, parentId, depth, body, author ({name, handle, url}), likes, restacks, repliesCount, createdAt, editedAt, isDeleted, isPinned, scrapedAt. Replies are flattened depth-first; parentId rebuilds the thread.

How to use Substack Scraper

1. Paste one or more Publications: a subdomain (lenny), lenny.substack.com, a custom domain (https://www.astralcodexten.com) or a single post URL.
2. Pick a Mode: posts (default), publication or comments.
3. Set Max posts per publication and Max items to cap the run and the cost. Turn off Include post body for a cheap metadata-only pass, or turn on Include comments to get threads with each post.
4. Optionally set Published from to stop at a date (the archive is read newest first).
5. Click Start. Results appear in the Dataset tab; export from there or use the API.

Input example

{
  "publications": ["astralcodexten", "https://www.lennysnewsletter.com"],
  "mode": "posts",
  "maxPostsPerPublication": 200,
  "maxItems": 1000,
  "includePostBody": true,
  "includeComments": false,
  "dateFrom": "2026-01-01"
}

{ "mode": "publication", "publications": ["lenny", "newsletter.pragmaticengineer.com", "thefp"] }

{ "mode": "comments", "publications": ["https://www.astralcodexten.com/p/god-help-us-lets-try-to-learn-about"], "maxCommentsPerPost": 200 }

Output example

{
  "type": "post",
  "id": 207542232,
  "slug": "god-help-us-lets-try-to-learn-about",
  "url": "https://www.astralcodexten.com/p/god-help-us-lets-try-to-learn-about",
  "title": "God Help Us, Let’s Try To Learn About Mechanistic Interpretability Techniques",
  "subtitle": "...",
  "publication": { "id": 89120, "name": "Astral Codex Ten", "subdomain": "astralcodexten", "url": "https://www.astralcodexten.com" },
  "authors": [{ "name": "Scott Alexander", "handle": "astralcodexten", "url": "https://substack.com/@astralcodexten", "isGuest": false }],
  "publishedAt": "2026-09-08T12:04:21.658Z",
  "updatedAt": "2026-09-08T14:10:02.114Z",
  "postType": "newsletter",
  "isPaywalled": false,
  "audience": "everyone",
  "likes": 191,
  "reactions": { "❤": 191 },
  "commentsCount": 110,
  "restacks": 13,
  "wordCount": 4695,
  "coverImageUrl": "https://substackcdn.com/image/fetch/...",
  "audioUrl": null,
  "hasVideo": false,
  "tags": [],
  "section": null,
  "bodyHtml": "<h2>...</h2><p>...</p>",
  "bodyText": "...",
  "bodyIsPreview": false,
  "scrapedAt": "2026-09-08T22:30:00Z"
}

{
  "type": "publication",
  "id": 10845,
  "name": "Lenny's Newsletter",
  "subdomain": "lenny",
  "customDomain": "www.lennysnewsletter.com",
  "url": "https://www.lennysnewsletter.com",
  "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
  "authors": [{ "name": "Lenny Rachitsky", "handle": "lenny", "role": "admin", "isOwner": true, "url": "https://substack.com/@lenny" }],
  "subscriberCountText": "Over 1,200,000 subscribers",
  "freeSubscriberCount": 1200000,
  "paidSubscriberCountText": "Tens of thousands of paid subscribers",
  "isPaid": true,
  "plans": [{ "id": "monthly15", "name": "$15 a month", "interval": "month", "amount": 15.0, "currency": "USD" }],
  "postsCount": 812,
  "sections": [{ "name": "How I AI", "slug": "how-i-ai" }],
  "hasPodcast": true
}

How to scrape a Substack newsletter's full archive

Use Mode posts with the publication name or URL and raise Max posts per publication (and Max items) above the number of posts; the Actor pages through the archive 50 posts at a time until it runs out. Leave Include post body on to get the full text of every free post in one run, or turn it off to list thousands of posts for a fraction of the requests. Published from lets you re-run on a schedule and only pick up posts since your last export.

Does it work without a Substack account?

Yes. Everything comes from the public endpoints that Substack's own website calls when you browse logged out. No login, cookies or API key are involved, which also means paid-subscriber-only content is never accessed: paywalled posts come with their metadata (title, date, likes, comment count, word count) and the public preview, marked isPaywalled: true and bodyIsPreview: true.

Can I scrape custom-domain Substacks like lennysnewsletter.com?

Yes. Paste the custom domain URL, the .substack.com subdomain or just the subdomain name; Substack redirects the API to the custom domain and the Actor follows it, so URLs in the output always use the publication's real domain.

How to get Substack comments

Set Mode comments (or keep posts and switch on Include comments) and set Max comments per post. Each comment item carries the public commenter name and handle, the text, likes, reply count, dates and parentId so you can rebuild threads. Paste a single post URL as the publication to get the comments of just that post.

How to get subscriber counts and prices of a Substack

Use Mode publication. The item includes the subscriber figures the author makes public (Over 1,200,000 subscribers, paid tier text, bestseller badge), monthly/annual plan prices, sections, podcast info and the total number of posts. Publications that hide their subscriber count return null there.

What does it cost?

Pay per item: a post or a publication item costs a fraction of a cent, a comment a quarter of that. There is no start fee and no subscription; Apify's free plan is enough to export a few thousand posts.

Integrations and API

Use the run in Zapier, Make, n8n, Google Sheets, or call it from Python/Node with the Apify client. See the API tab for ready-made snippets, or schedule the Actor to keep an archive or comment feed fresh.

Is it legal to scrape Substack?

This Actor collects only publicly available data that Substack shows to anyone without an account: post archives, free post text, public previews, publication details and public comments. It does not log in and never reads paid-subscriber content. Author and commenter names and handles are the public bylines Substack prints on every page; no emails, avatars or private profile data are collected. You are responsible for how you use the data and for complying with Substack's terms and applicable law.

Support

Found a bug or need a field added? Open an issue in the Issues tab; it is usually answered within a day.