Substack Scraper – Newsletter Posts, Full Text & Comments

Substack scraper without login: newsletter posts with full text of free posts (paywalled posts flagged, free preview only), likes, comment counts, tags and cover images, full comment threads, publication info (subscribers, author) and category leaderboards. Works with custom domains.

Price: $1.00 per 1,000 posts, $0.30 per 1,000 comments. No login, no API key, no browser. Inputs that fail produce a free error row.

What data do you get?

• Posts: title, subtitle, authors, publish/update date, URL, slug, audience (everyone / only_paid / founding), paywalled flag, likes (reactions), restacks, comment count, word count, tags, section, cover image, language, podcast audio URL and duration for podcast posts
• Full text: the post body as clean plain text (bodyText), optionally the original HTML (bodyHtml). Free posts come in full. Paid posts include only the free preview Substack shows to non-subscribers (paywalled: true, fullTextIncluded: false).
• Comments: comment ID, post ID, author display name, text, date, edit date, likes, parentId, depth, reply count, a direct comment link and an isPostAuthor flag
• Publication info: name, URL, subdomain, custom domain, description, author name, language, logo, subscriber count (when Substack shows it publicly), paid-subscriber label, bestseller tier, first post date
• Discovery: the top publications of any Substack category from the Bestseller (paid) and Rising (trending) leaderboards, or the full category list, with rank

Use cases

• Newsletter and creator research: find the top newsletters in a niche and compare audience size, posting frequency and engagement.
• Content monitoring: follow competitors, analysts or journalists and pull every new post on a schedule (onlyNewerThan: "1 day").
• AI, RAG and summarization: feed full post text into LLM pipelines, knowledge bases or daily digests.
• Audience and sentiment analysis: export comment threads with parent/child structure.
• Sponsorship and PR outreach lists: shortlist publications per category by rank and subscriber count (no personal emails are collected).

How to use it

1. Click Try for free or Start.
2. Choose a Mode: posts (default), postDetails, comments or discover.
3. Add newsletters to Publications (lenny, lenny.substack.com, https://www.astralcodexten.com or @handle) or post links to Post URLs.
4. Set Max posts per publication, Only posts newer than and Max items, then click Start. Export as JSON, CSV or Excel, or connect via API, Make, Zapier, n8n or Google Sheets.

Input

Field	Mode	Description
mode	all	posts (default), postDetails, comments, discover
publications	posts, comments	Subdomains, URLs, custom domains or @handles
postUrls	postDetails, comments	https://name.substack.com/p/slug, custom-domain post links, open.substack.com/pub/name/p/slug, substack.com/home/post/p-123
category	discover	Category slug, name or ID: technology, business, finance, culture, us-politics, science, health, crypto, history, sports, music, food, … (subcategories too)
ranking	discover	paid (Bestsellers), trending (Rising) or all
onlyNewerThan	posts, comments	2026-01-01 or 7 days, 1 month …
maxPostsPerPublication	posts, comments	Newest N posts per publication. Default 20, 0 = whole archive
includeFullText	posts	Load bodyText for each post. Default true
includeHtml	posts, postDetails	Also save bodyHtml. Default false
includeComments	posts, postDetails	Also return each post's comments. Default false
maxCommentsPerPost	all	Default 100, 0 = all
includePublicationInfo	posts	One publication row per newsletter. Default true
maxItems	all	Total item limit (= cost cap). Default 100

Example inputs:

{ "publications": ["lenny", "https://www.astralcodexten.com", "@noahpinion"], "maxPostsPerPublication": 10, "onlyNewerThan": "30 days" }

{ "mode": "postDetails", "postUrls": ["https://www.astralcodexten.com/p/open-thread-454"], "includeComments": true, "maxCommentsPerPost": 200 }

{ "mode": "discover", "category": "finance", "ranking": "paid", "maxItems": 100 }

Output

Every item has a type: post, comment, publication or error (free). Real output (shortened):

Post (paid post, so only the free preview is included)

{
    "type": "post",
    "publication": "Lenny's Newsletter",
    "publicationId": 10845,
    "publicationUrl": "https://www.lennysnewsletter.com",
    "postId": 216168140,
    "url": "https://www.lennysnewsletter.com/p/advanced-evals-how-to-find-and-fix",
    "slug": "advanced-evals-how-to-find-and-fix",
    "title": "Advanced evals: How to find (and fix) hidden AI failures in your product",
    "subtitle": "Why you should never skip error discovery",
    "authors": ["Hamel Husain", "Shreya Shankar"],
    "publishedAt": "2026-09-22T12:45:14.998Z",
    "updatedAt": "2026-09-22T19:35:47.938Z",
    "postType": "newsletter",
    "audience": "only_paid",
    "paywalled": true,
    "likes": 350,
    "restacks": 20,
    "commentCount": 4,
    "wordcount": 3807,
    "description": "Why you should never skip error discovery",
    "bodyText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. …",
    "fullTextIncluded": false,
    "coverImage": "https://substackcdn.com/image/fetch/…/6f6018cf-fa53-4f40-99d9-b1c9980dd031_1456x970.png",
    "tags": ["AI"],
    "section": null,
    "language": "en",
    "podcastUrl": null,
    "podcastDurationSec": null
}

Free posts look the same with "paywalled": false, "fullTextIncluded": true and the whole article in bodyText. For example, "An Open Letter To Steven Pinker On AI" on Astral Codex Ten returns 64,000 characters.

Comment

{
    "type": "comment",
    "id": 345437906,
    "postId": 215593613,
    "postUrl": "https://www.astralcodexten.com/p/fall-meetups-2026-call-for-meetup",
    "postTitle": "Fall Meetups 2026 - Call For Meetup Organizers",
    "publication": "Astral Codex Ten",
    "author": "Laniakea",
    "isPostAuthor": false,
    "body": "I am hosting a Petrov Day ceremony with friends (i.e. engineering students) next monday evening in Leuven, Belgium. …",
    "date": "2026-09-25T10:02:50.827Z",
    "editedAt": "2026-09-25T10:03:34.615Z",
    "likes": 1,
    "parentId": null,
    "depth": 1,
    "replyCount": 0,
    "url": "https://www.astralcodexten.com/p/fall-meetups-2026-call-for-meetup/comment/345437906"
}

Publication (mode discover, Finance / Rising)

{
    "type": "publication",
    "rank": 1,
    "category": "Finance",
    "ranking": "trending",
    "publicationId": 630791,
    "name": "Lord Fed's Gazette",
    "url": "https://www.lordfed.co.uk",
    "subdomain": "lordfed",
    "customDomain": "www.lordfed.co.uk",
    "description": "Market analysis, idea generation and education from a London-based portfolio manager. …",
    "authorName": "Lord Fed",
    "language": "en",
    "freeSubscriberCount": 24000,
    "subscriberCountText": "Tens of thousands of subscribers",
    "paidSubscribersText": "Thousands of paid subscribers",
    "paidSubscriptionsEnabled": true,
    "bestsellerTier": 1000,
    "firstPostDate": "2022-04-03T01:25:01.568Z",
    "archiveUrl": "https://www.lordfed.co.uk/archive"
}

Error (free)

{ "type": "error", "input": "https://example.com", "errorCode": "NOT_FOUND", "error": "https://example.com does not look like a Substack publication (no Substack data on the page)." }

Possible errorCode values: INVALID_INPUT, NOT_FOUND, FETCH_FAILED, NO_COMMENTS, NO_RESULTS.

Dataset views in Apify Console: Overview, Full text, Comments, Publications & errors.

Pricing

Pay per event, with platform usage included.

Event	Price	When it is charged
post	$0.001 per post ($1.00 / 1,000)	Each post saved (with or without full text)
comment	$0.0003 per comment ($0.30 / 1,000)	Each comment saved
publication	$0.001 per publication	Each publication-info or leaderboard row
Errors / no results	Free	Invalid inputs, missing newsletters or posts

Job	Cost
20 latest posts of 1 newsletter with full text + publication info	$0.021
1,000 posts	$1.00
A post with 800 comments	$0.24
Top 100 publications in a category	$0.10

Set Max items and Apify's Max cost per run. The Actor stops cleanly at the limit.

Use with AI agents (MCP)

Add the Actor as a tool in the Apify MCP server (https://mcp.apify.com?tools=openharvest/substack-scraper), or let agents find it with search-actors ("substack").

Example call-actor input for "Summarize what Noahpinion wrote this week":

{ "mode": "posts", "publications": ["@noahpinion"], "onlyNewerThan": "7 days", "includeFullText": true, "includePublicationInfo": false, "maxItems": 10 }

Example input for "Find the top 25 rising tech newsletters":

{ "mode": "discover", "category": "technology", "ranking": "trending", "maxItems": 25 }

FAQ

Can it get the full text of paid posts? No. The Actor only reads what Substack shows publicly. For paid posts you get the free preview (paywalled: true, fullTextIncluded: false) and all metadata (title, likes, comment count, word count …).

Does it work with custom domains? Yes. www.astralcodexten.com, www.lennysnewsletter.com and other custom domains work like name.substack.com. Author handles such as @noahpinion are resolved to their main publication.

How many posts can I get? The whole public archive. Set maxPostsPerPublication to 0 and raise maxItems.

Why do some posts with comments return none? Some publications limit their discussion to paid subscribers. Those comments are not public, so they're skipped and you aren't charged.

Where does the subscriber count come from? It's the number Substack displays publicly on the newsletter (for example "1,200,000 subscribers"). Some publications hide it, and then it is null.

How fresh is the data? It is live: every run reads Substack's public endpoints directly.

Is it legal to scrape Substack?

The Actor reads only publicly available data, the same posts, previews and comments anyone can see in a browser without logging in. It does not bypass paywalls and does not access subscriber-only content. It does not collect emails, user IDs, profile photos or other contact details. Comment authors appear only by their public display name. Posts and comments are copyrighted by their authors, so respect their rights, Substack's Terms of Use, and laws such as GDPR when you store or republish data. Do not use the data for spam or unsolicited outreach.

Related actors

• Podcast Scraper – Apple Podcasts Search, Charts, Episodes & RSS Feeds (coming soon)
• Hacker News Scraper – Stories, Comments, Search & Who Is Hiring (coming soon)
• Telegram Channel Scraper – Public Posts, Views & Media (coming soon)
• RSS Feed Reader (coming soon)

Support & feedback

Found a bug or a missing field? Open an issue in the Issues tab with the publication or post URL.