Substack Scraper

Scrape any Substack newsletter: every post with title, date, reactions, comments count, restacks, paywall status and the full text of free posts, plus comments and the newsletter's subscriber count. Or search all of Substack by keyword. No login, no API key.

At a glance
You give it	Newsletter URLs or search keywords, plus optional date, free-only and comment settings
You get	One row per post (full text for free posts, reactions, comments count, paywall status) and optionally per comment
Price	$0.005 per run + $0.00098 per post, $0.002 per post with full text, $0.00049 per comment (Free plan)
Speed	10 posts and 35 comments in about 11 seconds
Needs	Nothing: no login, no API key, no proxy

What you get

• Whole archives, newest first, with a date range that stops paging as soon as it reaches older posts.
• Full text for free posts as HTML and as clean plain text, ready for AI. Paid posts come with their public preview.
• Fair billing on paid posts. A paywalled post is always charged the lower metadata price, because there is no full text to deliver.
• Comments and replies, one row each with the parent comment id and nesting level, so threads can be rebuilt.
• Publication details on every post: name, URL, author, language, description, and subscriber count (for example "1.2M+" and 1,200,000).
• Keyword search across all of Substack: posts on a topic from many newsletters, with each newsletter's details.
• Filters: free posts only, post type (newsletter, podcast, thread), date range. Filters apply to newsletters, single posts and keyword search alike.
• Cheap to run: 10 posts plus 35 comments used $0.0004 of platform usage.

Use cases

• Content research. What the top newsletters in your niche publish, and which posts get the most reactions and comments.
• Newsletter sponsorship and partnership prospecting. Find newsletters on a topic with their subscriber counts and authors.
• AI and RAG. Clean full text from the writers you trust, updated on a schedule.
• Audience research. Read what commenters ask and argue about.
• Competitive monitoring. Get every new post from competitor newsletters.

Example inputs

Latest 20 posts from two newsletters, with comments

{ "urls": ["https://www.lennysnewsletter.com", "importai"], "maxPostsPerNewsletter": 20, "includeComments": true }

A whole archive since January, metadata only

{ "urls": ["lenny"], "maxPostsPerNewsletter": 0, "dateFrom": "2026-01-01", "includeContent": false }

Search Substack for a topic

{ "searchKeywords": ["ai agents", "vertical saas"], "maxSearchResultsPerKeyword": 50, "onlyFree": true }

Specific posts

{ "urls": ["https://www.lennysnewsletter.com/p/announcing-lennys-jobs-the-best-place"], "includeComments": true }

Input

Field	What it does
urls	Newsletter URLs (https://www.lennysnewsletter.com, lenny.substack.com, or lenny) and post URLs (anything with /p/).
searchKeywords	Search all of Substack.
maxPostsPerNewsletter	Newest first. 0 means the whole archive. Default 50.
maxSearchResultsPerKeyword	Default 20.
includeContent	Full text of free posts. Default on.
includeComments, maxCommentsPerPost	Comments and replies, best first.
onlyFree, postType, dateFrom, dateTo	Filters.

Output

Each row has a type: post or comment. The Posts and Comments tabs split them. Real rows from a test run on 2026-09-26 (long text shortened here):

Post

{
  "type": "post",
  "postId": 208730073,
  "title": "Announcing Lenny’s Jobs: The best place in the world to find, vet, and land your dream job",
  "subtitle": "Where product managers, engineers, designers, and growth/marketing professionals discover high-quality open roles at tech companies",
  "url": "https://www.lennysnewsletter.com/p/announcing-lennys-jobs-the-best-place",
  "slug": "announcing-lennys-jobs-the-best-place",
  "publishedAt": "2026-08-18T15:40:06.921Z",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaid": false,
  "wordCount": 933,
  "reactionCount": 378,
  "commentCount": 18,
  "restacks": 9,
  "tags": [
    "Career"
  ],
  "section": null,
  "description": "Where product managers, engineers, designers, and growth/marketing professionals discover high-quality open roles at tech companies",
  "previewText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: Lenny’s Jobs ...",
  "bodyHtml": "<p><em><span>👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: ...",
  "bodyText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: Lenny’s Jobs | Lenny’s Podcast | How I AI | Lennybot | Become an AI-Native Builder and some o...",
  "coverImage": "https://substackcdn.com/image/fetch/$s_!PFZn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0f696879-6374-42d6-a171-47933f6eb126_1456x970.png",
  "authors": [
    "Lenny Rachitsky"
  ],
  "authorName": "Lenny Rachitsky",
  "authorHandle": "lenny",
  "podcastUrl": null,
  "podcastDurationSecs": null,
  "hasVoiceover": false,
  "publicationId": 10845,
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "publicationAuthor": "Lenny Rachitsky",
  "publicationAuthorHandle": "lenny",
  "publicationLanguage": "en",
  "publicationDescription": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious bu",
  "publicationLogo": "https://substackcdn.com/image/fetch/$s_!8MSN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441213db-4824-4e48-9d28-a3a18952cbfc_592x592.png",
  "publicationPaidEnabled": true,
  "freeSubscriberCount": 1200000,
  "subscriberCountText": "1.2M+",
  "searchKeyword": null
}

Comment

{
  "type": "comment",
  "commentId": 317389445,
  "parentCommentId": null,
  "level": 0,
  "postId": 208730073,
  "postTitle": "Announcing Lenny’s Jobs: The best place in the world to find, vet, and land your dream job",
  "postUrl": "https://www.lennysnewsletter.com/p/announcing-lennys-jobs-the-best-place",
  "body": "The Lenny 100 list is GENIUS 👏",
  "date": "2026-08-18T15:48:15.832Z",
  "editedAt": null,
  "authorName": "Tom Orbach",
  "authorHandle": "marketingideas",
  "reactionCount": 7,
  "restacks": 0,
  "replyCount": 0,
  "publicationName": "Lenny's Newsletter"
}

Pricing

Pay per event. Each post is charged one of the first two, never both:

• $0.005 per run start
• $0.00098 per post with metadata (paid posts, or when full text is off)
• $0.002 per post with full text (free posts, full text on)
• $0.00049 per comment

Lower on higher Apify plans. Apify platform usage is billed separately to your account and is tiny: 10 posts plus 35 comments used $0.0004.

What you scrape	Actor cost
50 posts, metadata only	about $0.054
50 free posts with full text	about $0.105
50 posts + 500 comments	about $0.30

Automate it with n8n

Each workflow uses n8n's official Apify node, operation Run actor and get dataset, actor labrat011/substack-scraper. Paste the input into Input JSON; switch it to Expression when it contains {{ }}.

1. New posts from your favorite newsletters, summarized by AI

Schedule Trigger (daily 07:00)
  > Apify: Run actor and get dataset
      { "urls": ["lenny", "importai"], "dateFrom": "{{ $now.minus({ days: 1 }).toISODate() }}", "onlyFree": true }
  > OpenAI / Anthropic: "Summarize this post in 3 bullets" on bodyText
  > Slack or Gmail: one message per post with the summary and url

2. Sponsorship prospect list

Manual Trigger
  > Apify: Run actor and get dataset   { "searchKeywords": ["b2b marketing", "product management"], "maxSearchResultsPerKeyword": 100 }
  > Remove Duplicates: on publicationUrl
  > Filter: freeSubscriberCount > 5000
  > Google Sheets: publicationName, publicationUrl, publicationAuthor, freeSubscriberCount

3. Newsletter RAG knowledge base

Schedule Trigger (weekly)
  > Apify: Run actor and get dataset   (your newsletters, last 7 days, full text on)
  > Filter: type is post and bodyText is not empty
  > Embeddings + Vector Store insert (Pinecone, Qdrant, Supabase), metadata: title, url, publishedAt

4. Top-performing posts report

Schedule Trigger (monthly)
  > Apify: Run actor and get dataset   (competitor newsletters, dateFrom = first of last month, includeContent false)
  > Sort: reactionCount descending
  > Google Sheets: top 20 with title, reactions, comments, restacks, url

5. Comment insights

Manual Trigger
  > Apify: Run actor and get dataset   (one newsletter, 20 posts, includeComments true, 100 per post)
  > Filter: type is comment
  > Aggregate: all comment bodies
  > OpenAI / Anthropic: "What questions and objections come up most? Quote examples."

For AI agents

• Actor: labrat011/substack-scraper
• Smallest input: { "urls": ["https://www.lennysnewsletter.com"] }
• One row = one post (type: "post") or one comment (type: "comment"). Key fields: posts: title, url, publishedAt, isPaid, reactionCount, commentCount, bodyText; comments: commentId, postId, body, date.
• Billing: apify-actor-start once per run, post-with-content or post-metadata per post (never both), comment per comment row. Cap spend with maxPostsPerNewsletter, maxCommentsPerPost or a maximum cost per run.
• Run it: POST https://api.apify.com/v2/acts/labrat011~substack-scraper/run-sync-get-dataset-items with the input as the JSON body, or call it from the Apify MCP server.
• Done signal: the run's status message reads Saved N posts and M comments in R requests.

FAQ

Can it read paid posts? No. Paid posts return their metadata and the public preview, and are charged the lower metadata price.

Where does the subscriber count come from? From the number Substack itself shows on the newsletter's page. Substack rounds it, for example "1.2M+".

Do I need a proxy? No. The actor spaces its requests and backs off if Substack slows it down.

Support

Open an issue on the actor's Issues tab with the run ID and it will be looked at.