Substack Scraper: Posts, Comments & Notes

Substack scraper: posts, full articles, comments, reactors and Notes

Get Substack data in three ways: every post of a newsletter, single posts by link, or an author's Substack Notes. Paste links or handles, press Start, and download posts with full text, engagement and comment threads.

$0.30 per 1,000 posts or Notes. No login, cookies or proxy needed.

Works with custom domains and .substack.com addresses alike, in any language.

Pick what to paste

Fill one section, or both. Every input works alone or mixed with the others.

You want	Paste into
Every post of a newsletter, with filters and search	urls (newsletter links)
One post, with its comments	urls (post or share links)
The newsletter of an author	urls (substack.com/@handle links)
An author's Substack Notes	notesHandles (handles)

Every input can be scheduled. Run it weekly and you have a newsletter or Notes tracker.

Why this Substack scraper?

• 📚 The whole archive, not just the latest 20. A 1,500-post newsletter comes back complete in under two minutes.
• 🎯 Filters that fill your limit. Ask for 40 paid posts from 2026 and you get 40.
• 🔍 Search inside a newsletter, with the full text of every match.
• 💬 Full comment threads, nested replies included, ready to rebuild as a tree.
• 🔗 Any Substack link works: a newsletter, a post, a share link from the app, or a profile.
• 🧾 Never billed twice. The same post reached through two links is one row, and posts your filters drop are free.

1. Newsletters: every post, with filters and search

Paste newsletter links, for example https://newsletter.pragmaticengineer.com, or a profile link like https://substack.com/@lenny.

Get one row per post: title, full text, author and co-authors, publish date, reactions, comment and restack counts, free or paid, tags, audio, and the newsletter's name.

Option	Field	Default	What it does
Max posts per newsletter	maxPosts	50	0 for the whole archive
Sort	sortBy	newest	newest or oldest
Add full article text	includeContent	true	Turn off for faster metadata-only runs
Content format	contentFormat	both	html, text or both

Filters (all optional)

Filter	Field
Only posts whose title or text match a keyword	searchKeyword
Free or paid posts only	audienceFilter
Newsletter, podcast, thread or video posts only	typeFilter
Published on or after / on or before a date (YYYY-MM-DD)	dateFrom, dateTo

Input

{
  "urls": ["https://newsletter.pragmaticengineer.com"],
  "audienceFilter": "paid",
  "dateFrom": "2026-01-01",
  "maxPosts": 50
}

Output (one real row, selected fields)

{
  "id": 221215977,
  "title": "Why Global Trepanation?",
  "slug": "why-global-trepanation",
  "url": "https://www.astralcodexten.com/p/why-global-trepanation",
  "author": "Scott Alexander",
  "authorHandle": "astralcodexten",
  "publishedAt": "2026-10-21T21:53:11.440Z",
  "updatedAt": "2026-10-22T00:54:19.173Z",
  "contentText": "A friend asked me to investigate the following mystery: why do 5-10% of skulls from 10,000 BC to 3,0...",
  "wordCount": 1307,
  "readingTimeMinutes": 6,
  "coverImageUrl": "https://substack-post-media.s3.amazonaws.com/public/images/73a7ca5d-8a51-4c67-9dd2-9348f3d6ffc3_2340x1316.jpeg",
  "audienceType": "everyone",
  "isPaywalled": false,
  "reactionCount": 383,
  "reactions": { "❤": 383 },
  "commentCount": 204,
  "restackCount": 18,
  "type": "newsletter",
  "hasAudio": true,
  "newsletter": {
    "name": "Astral Codex Ten",
    "description": "P(A|B) = [P(A)*P(B|A)]/P(B), all the rest is commentary.",
    "url": "https://www.astralcodexten.com"
  },
  "comments": null,
  "facepile": null
}

2. Single posts: one post by link, with its comments

Paste post links (https://www.astralcodexten.com/p/why-global-trepanation) or share links from the Substack app. Filters and the post limit do not apply to a post you name.

Get the same row as above for each post. Turn on the extras to add the comment thread and the readers who reacted.

Option	Field	Default	What it does
Add comments	includeComments	false	The full thread, nested replies included. Charged per comment
Add reactors and restackers	includeFacepile	false	Up to 5 of each, as Substack shows them. Charged per entry

The extras work on newsletter links too: every post gets its own thread.

Input

{
  "urls": ["https://www.astralcodexten.com/p/why-global-trepanation"],
  "includeComments": true,
  "includeFacepile": true
}

Output (one real comment from comments; parentId and depth rebuild the thread)

{
  "id": 366989167,
  "parentId": 366969428,
  "depth": 1,
  "bodyText": "I wish we had a like button.",
  "authorName": "Greg Gentschev",
  "authorHandle": "gentschev",
  "publishedAt": "2026-10-21T22:31:56.846Z",
  "isPinned": false,
  "isDeleted": false,
  "reactionCount": 5,
  "reactions": { "❤": 5 }
}

3. Notes: an author's Substack Notes

Paste Substack handles, with or without @, for example sahilbloom.

Get one row per note, newest first: the text, reactions, restacks, reply count, and the author's bio and newsletter.

Option	Field	Default	What it does
Max Notes per handle	maxNotesPerHandle	50	Up to 20,000
Published on or after / before	dateFrom, dateTo	empty	Only Notes in this date range

Input

{
  "notesHandles": ["sahilbloom", "tedgioia"],
  "maxNotesPerHandle": 500,
  "dateFrom": "2026-01-01"
}

Output (one real row)

{
  "id": 368220860,
  "handle": "sahilbloom",
  "authorName": "Sahil Bloom",
  "authorBio": "NYT Bestselling Author of The 5 Types of Wealth. Gave up a grand slam on ESPN in 2012 and still wait...",
  "publishedAt": "2026-10-23T12:28:21.063Z",
  "bodyText": "I believe that 99% of success is just the ability to outlast uncertainty. The one who can tolerate t...",
  "reactionCount": 217,
  "reactions": { "❤": 217 },
  "restackCount": 13,
  "childrenCount": 38,
  "publicationName": "The Curiosity Chronicle",
  "publicationUrl": "https://sahilbloom.substack.com"
}

Use it from an AI agent

Connect Claude, Cursor or any MCP client to Apify's MCP server with this scraper as a tool:

https://mcp.apify.com?tools=sourabhbgp/substack-scraper

Then ask in plain words. The agent fills in the input, and you pay the same per result. For example:

• "Get the 20 newest posts from newsletter.pragmaticengineer.com with their full text."
• "Find every Lenny's Newsletter post about pricing and summarise them."
• "Pull all comments on this Substack post and tell me what readers disagree with."
• "Export Sahil Bloom's Substack Notes since January."

What people use it for

• 📊 Content research: every post a newsletter has published, with engagement, to see what lands.
• 🧠 AI and text analysis: clean full text for summaries, topic models or a searchable archive.
• 💬 Audience research: read what readers actually say in the comments.
• 🎯 Lead generation: find active commenters and reactors around a topic, with their own newsletters.
• 🏁 Competitor tracking: watch rival newsletters' publishing pace, paid share and podcast output.

How much does it cost?

The same price on every Apify plan. Full article text costs nothing extra.

What you get	Per 1,000
Posts or Notes	$0.30
Comments (includeComments)	$0.30
Reactors and restackers (includeFacepile)	$0.20

Each run also costs $0.00005 to start.

• Free plan: your $5 monthly credit covers about 16,600 posts.
• Starter plan ($29): about 96,600 posts a month.

Limitations

• Paid posts come back as the free preview. Substack shows the full text only to paying subscribers.
• Paid posts have no comments. Substack shows them only to subscribers, so comments is empty even when commentCount is above zero.
• Up to 5 reactors and 5 restackers per post, whatever the reaction count. Their handle and bio stay empty.
• Video posts usually come back as podcast. That's how Substack labels them, so typeFilter: "video" rarely matches.
• authorTwitter is usually empty. Substack no longer sends it.

FAQ

How do I scrape Substack without coding?

Paste a newsletter or post link, or a handle under Notes, and press Start. Download the results as Excel, CSV or JSON.

Can I get a newsletter's whole archive?

Yes. Set Max posts per newsletter to 0. Set Sort to oldest to start from the first post.

Why did I get fewer posts than I asked for?

The newsletter has fewer posts that match your filters. The run's status message says so, and you pay only for posts delivered.

Do I need a Substack account or a proxy?

No. It reads only what any logged-out visitor can see, and handles blocks by itself.

Can I use it from code or an AI agent?

Yes. Call it through the Apify API, or connect it to an AI agent through Apify's MCP server.

Is scraping Substack legal?

It collects only what Substack shows any visitor, with no login and no paywall bypass. How you use the data is your responsibility, especially personal data such as commenter names.

Your feedback

Something missing or wrong? Open an issue.