Substack Newsletter, Archive & Comments Scraper
Pricing
from $1.20 / 1,000 substack post returneds
Substack Newsletter, Archive & Comments Scraper
Extract Substack newsletter posts, full clean text/Markdown content, engagement metrics, and comments from any publication or custom domain using Substack's cursor-paginated JSON API.
Pricing
from $1.20 / 1,000 substack post returneds
Rating
0.0
(0)
Developer
Pulse Extract
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
3 days ago
Last modified
Categories
Features
Share
Extract complete Substack newsletter archives, full clean article text, RAG-ready Markdown, engagement metrics (reactionCount, commentCount, restacks), and post comments from any Substack publication, custom domain (like lennysnewsletter.com), single-post URL, or global keyword search.
Unlike legacy scrapers breaking with HTTP 400 on Substack's retired offset= parameter, this Actor uses Substack's 2026 nextCursor pagination API — running purely over fast HTTP/2 with zero browser and zero proxy costs.
Key Features
- Works Across All Substack Formats: Custom domains (
https://www.lennysnewsletter.com),*.substack.comsubdomains, bare publication handles (lennysnewsletter), single post URLs (/p/slug), and global keyword searches (searchQueries). - 2026 Cursor Pagination: Paginates deep archives smoothly via
nextCursorwithout HTTP 400 offset errors. - RAG & AI-Ready Markdown: Strips subscribe/widget clutter cleanly and converts headings, lists, blockquotes, and code blocks into structured
contentMarkdownalongside plaincontentText. - Optional Comment Extraction: Enable
includeComments: trueto fetch reader comments (commentId,authorName,date,body,reactionCount). - Pre-Push Filtering: Filter by
audienceFilter(everyonevsonly_paid),publishedWithinDays, andkeywordsbefore records are pushed and billed.
Example Input
{"publications": ["https://www.lennysnewsletter.com","https://newsletter.pragmaticengineer.com"],"searchQueries": [],"sortBy": "new","maxPostsPerSource": 20,"includeContent": true,"includeComments": false,"maxCommentsPerPost": 20,"keywords": [],"audienceFilter": "all","publishedWithinDays": 0}
Example Output Item
{"sourceInput": "https://www.lennysnewsletter.com","publicationDomain": "www.lennysnewsletter.com","publicationName": "www.lennysnewsletter.com","postId": 224972682,"slug": "35-premium-tools-free-for-a-yearnow","title": "35 premium tools free for a year","subtitle": "...","postType": "newsletter","audience": "everyone","isPaywalled": false,"publishedAt": "2026-11-17T18:02:02.477Z","authors": ["Lenny Rachitsky"],"tags": [],"wordCount": 1752,"reactionCount": 208,"commentCount": 12,"restacks": 15,"canonicalUrl": "https://www.lennysnewsletter.com/p/35-premium-tools-free-for-a-yearnow","coverImageUrl": "https://substackcdn.com/...","contentText": "...","contentMarkdown": "## ...","scrapedAt": "2026-11-19T03:25:00+00:00"}
Pricing & Billing Transparency
This Actor uses Pay-Per-Event (PPE) billing:
- Actor Start:
$0.00005per run. - Per Post Returned (
post-scraped):$0.0012per post ($1.20 per 1,000 posts). - Optional Comments (
comment-scraped):$0.0004per extracted comment (only charged whenincludeCommentsistrue).
Use Cases & Integrations
- AI Knowledge Bases & Competitive Research: Ingest full
contentMarkdownarchives of top industry newsletters into vector databases via n8n, Make, or LangChain. - Trend & Audience Monitoring: Run weekly with
publishedWithinDays: 7andincludeComments: trueto spot high-engagement topics and reader pain points. - AI Agents & MCP: Connect through Apify's MCP server so Claude or Cursor can query newsletter insights cleanly.
FAQ
Can this bypass paid subscriber paywalls?
No. It extracts all public metadata and free preview/article bodies returned by Substack's public API; paywalled posts report isPaywalled: true and any public preview text.