Substack Scraper — Posts, Authors & Comments (Bulk)

Scrape posts from any Substack publication as clean, structured JSON, CSV or Excel: title, subtitle, author, URL, publish date, likes, comment count, excerpt, full body text and whether the post is paid or free. Point it at one publication or dozens, cap how many posts you want per publication, and optionally pull the top comments on each post — all in one run, with no account to manage.

What does Substack Scraper do?

This Substack scraper turns any publication's archive into a structured dataset. Give it a handle (like bigtechnology), a Substack URL, or a publication's custom domain, and it walks the archive newest-first and returns one flat record per post — with the headline metadata, the full article text, and the paid/free flag. Turn on comments to also capture the top community replies under each post. Use it to build newsletter datasets, monitor writers and topics, feed research pipelines, or archive your own publication.

What data can you scrape from Substack?

Data point	Description
publication	The publication handle or domain the post belongs to.
postId	Numeric Substack post id.
slug	URL slug of the post.
title	Post title.
subtitle	Post subtitle / deck.
author	Author name(s) / byline.
url	Canonical URL of the post.
publishedAt	Publish date/time (ISO 8601).
type	Post type (newsletter, podcast, thread, etc.).
likes	Number of reactions/likes.
commentCount	Number of comments on the post.
wordCount	Approximate word count of the article.
excerpt	Short preview/teaser text.
bodyText	Full article body as plain text.
isPaid	Whether the post is subscriber-only.
coverImage	Cover image URL, if any.
comments	Top comments (author, body, date, likes) when enabled.

Use cases

• Newsletter datasets — collect a publication's back catalogue for analysis or search.
• Writer & topic monitoring — track new posts from the publications you care about on a schedule.
• Research pipelines — feed structured post text into BI tools, notebooks, models or RAG apps.
• Competitive intelligence — see cadence, engagement (likes/comments) and paid vs free mix across publications.
• Personal archive — export your own Substack posts and comments to JSON, CSV or Excel.

How to scrape Substack data

1. Add Substack Scraper to your Apify account and open its input.
2. Enter one or more publications — a handle (bigtechnology), a Substack URL (https://bigtechnology.substack.com) or a custom domain (https://www.astralcodexten.com).
3. Set maxPosts per publication (or 0 for the entire archive).
4. Optionally enable includeComments and set maxComments.
5. Click Run, then export the results as JSON, CSV or Excel — or fetch them through the Apify API.

Input

{
  "publications": ["bigtechnology", "https://www.astralcodexten.com"],
  "maxPosts": 50,
  "includeComments": false,
  "maxComments": 20
}

Field	Description
publications	Publications to scrape — handles, Substack URLs or custom domains.
maxPosts	Max posts per publication, newest first (0 = entire archive).
includeComments	Also fetch the top comments for each post.
maxComments	Max top-level comments to keep per post when comments are on.

What you get

One flat JSON record per post, ready to export to CSV, Excel, JSON or XML:

{
  "publication": "bigtechnology",
  "postId": 209279848,
  "slug": "when-artificial-intelligence-is-too",
  "title": "When Artificial Intelligence Is Too Valuable To Sell",
  "subtitle": "Just because the AI labs have sold models on a meter until now doesn't mean they'll always do so.",
  "author": "Alex Kantrowitz",
  "url": "https://www.bigtechnology.com/p/when-artificial-intelligence-is-too",
  "publishedAt": "2026-07-31T19:30:17.712Z",
  "type": "newsletter",
  "likes": 83,
  "commentCount": 0,
  "wordCount": 1139,
  "excerpt": "It's time to delete the assumption that the frontier AI labs will always license their best models…",
  "bodyText": "…full article text…",
  "isPaid": true,
  "coverImage": "https://substackcdn.com/image/…",
  "comments": []
}

Related Actors

Building a broader newsletter, news or research dataset? Pair this with:

• Google News Scraper — headlines by keyword and topic.
• Hacker News Scraper — stories, comments and users from Hacker News.
• Reddit Scraper — posts, comments and users, no login.
• Website Content Crawler — crawl any site into clean text for LLM/RAG.

FAQ

Do I need a Substack account or login? No. Just add the publications you want and run — no account, login or setup required.

How many posts can I scrape per run? There's no fixed cap. maxPosts limits how many posts are collected per publication (0 = the whole archive), and maxComments limits comments per post. Larger runs simply take longer.

Can I scrape comments? Yes. Turn on includeComments and the Actor returns the top publicly visible comments — author, body, date and likes — nested under each post.

Does it work with custom domains? Yes. Many publications use their own domain (e.g. www.astralcodexten.com). Enter the handle or the custom domain and the Actor resolves it automatically.

Is scraping Substack legal? The Actor collects only publicly available data. You are responsible for using it in line with Substack's terms and applicable laws — avoid collecting personal data you don't have a lawful basis to process.

What's the output format? A structured JSON dataset with one item per post (comments nest inside their post). Export as JSON, CSV or Excel from the Apify Console or API.

Can I use this with n8n, Make or Zapier? Yes. Run it from n8n via the Apify node, or connect it to Make, Zapier, Slack, Google Drive and many more, plus the Apify API, JavaScript/Python clients, MCP and webhooks.

Notes

Original clean-room implementation. Returns only public data; you are responsible for compliance with Substack's terms. Not affiliated with Substack.