Substack Posts Scraper — Newsletter Archive, Stats & Full Text

Scrape the complete post archive of any Substack publication — including custom-domain publications — with engagement stats and optional full post text. One clean JSON item per post, no login, no API key, no proxy required.

Why use this Actor?

• Whole archives, not just the latest page — Substack's API paginates from newest to oldest with no hard cap; this Actor walks it end-to-end up to your chosen limit.
• Custom domains supported — pass astralcodexten, name.substack.com or https://any-custom-domain.com; all three resolve correctly.
• Engagement data included — likes, comments, word count and paywall status on every post, ready for analysis.
• Full-text mode — set includeBody: true and get clean article text for AI/RAG pipelines (public posts only).
• Paywall-safe — paywalled posts return their public metadata and preview; protected text is never scraped.

What you get per post

Field	Description
title, subtitle, description	Post heading, subheading and preview text
slug, url	Post slug and canonical URL
publishedAt	Publish date (ISO 8601)
type	newsletter, podcast or thread
audience, isPaywalled	Audience setting and paywall flag
wordCount, likes, comments	Engagement metrics
coverImage	Cover image URL
bodyText	Full post text (only when includeBody: true, public posts)

Input

{
  "publications": ["astralcodexten", "https://www.noahpinion.blog"],
  "maxPostsPerPublication": 100,
  "includeBody": false
}

Field	Type	Default	Description
publications	array of strings	—	Bare names (astralcodexten), subdomains (name.substack.com) or full URLs — custom domains work too. Required.
maxPostsPerPublication	integer	50	Maximum posts per publication, newest first (1–5000).
includeBody	boolean	false	Fetch full text of each public post (slower — one extra request per post).

Output (one item per post)

{
  "publication": "https://astralcodexten.substack.com",
  "postId": 158204926,
  "title": "Your Book Review: ...",
  "subtitle": "Or: why the best books are the ones you argue with",
  "slug": "your-book-review",
  "url": "https://www.astralcodexten.com/p/your-book-review",
  "publishedAt": "2026-06-20T12:01:33.000Z",
  "type": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "wordCount": 8214,
  "likes": 312,
  "comments": 447,
  "coverImage": "https://substackcdn.com/image/fetch/...",
  "bodyText": "The full article text when includeBody is enabled..."
}

Use cases

• Newsletter operators — study what performs in your niche before writing your next issue
• Content & trend research — track topics, formats and engagement across publications over time
• AI / RAG pipelines — feed clean long-form text with metadata into embeddings and fine-tuning
• Journalists & analysts — archive and monitor influential newsletters in your beat
• Competitive intelligence — see which headlines and subjects get the most hearts and comments

Pricing

Pay per event: you are charged a small fee per post scraped, plus Apify's standard compute costs. No subscription, no minimums — scan one archive or a hundred.

FAQ

Does it get paywalled content? No. Paywalled posts return their public metadata and preview only; bodyText is fetched exclusively for public posts.

How far back does it go? The whole archive — the API paginates from newest to oldest with no hard cap. Set maxPostsPerPublication to bound the run.

Can I scrape a publication on a custom domain? Yes. Pass the full URL and the Actor resolves it to the publication's Substack API endpoint automatically.

Does it need a login or proxy? No — the public Substack API is used; no cookies, no residential proxies.

What if a publication is inactive or deleted? The run reports it as skipped rather than failing the whole batch, so one dead link never wastes the rest of the run.