Substack Scraper: newsletter posts, full text and comments, no login

Scrape any public Substack newsletter, post URL or Substack-wide search into JSON: title, subtitle, authors, date, full text of free posts, likes, comment count, restacks, tags and, if you want, the comments. No Substack account, no cookies, no API key, no browser.

Price: $1.50 per 1,000 posts ($0.0015 per post, full text included) and $0.30 per 1,000 comments. 100 posts cost 15 cents.

Why use this one

• Post text is in the row. Full text and HTML of free posts come back by default at no extra charge. Paywalled posts return the public preview and are marked isPaywalled: true and isBodyTruncated: true, so you know which text is complete.
• Custom domains work (https://www.lennysnewsletter.com, a name.substack.com address or just the subdomain).
• A bad newsletter does not sink the run. Unavailable newsletters are reported in the log and run summary, skipped and not billed; one bad post never loses the others.
• Three inputs in one run: newsletters, single posts and search queries, de-duplicated together; postsSince ("7 days") for scheduled feeds.
• Honest limit: nothing behind a paywall is fetched, and search returns what Substack's own search ranks first.

Minimal input

{"publications": ["https://www.lennysnewsletter.com"], "maxPostsPerPublication": 20, "postsSince": "30 days"}

Sample output row (real platform run, long text shortened)

{
  "type": "post",
  "id": 215694124,
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "slug": "all-of-the-lenny-and-friends-summit",
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "authors": [
    {
      "name": "Lenny Rachitsky",
      "handle": "lenny",
      "url": "https://substack.com/@lenny"
    }
  ],
  "publishedAt": "2026-09-29T13:15:57+00:00",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "isBodyTruncated": false,
  "wordCount": 1276,
  "likes": 284,
  "reactions": {
    "❤": 284
  },
  "commentCount": 6,
  "restacks": 5,
  "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-0e02-4190-a590-1236719b99de_9213x6142.jpeg",
  "tags": [],
  "bodyText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: Len...",
  "bodyHtml": "<p><em>👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For mo...",
  "scrapedAt": "2026-10-11T06:37:27+00:00",
  "input": "https://www.lennysnewsletter.com"
}

What does Substack Scraper do?

It reads a newsletter's archive (newest or most popular first), fetches each post's body, and returns one row per post in the Apify dataset, exportable as JSON, CSV or Excel or readable through the API. Typical uses:

• Newsletter monitoring. Track what competitors, analysts or creators publish, and how many likes and comments each post gets.
• Content research. Find the top posts on a topic with Substack-wide keyword search.
• Datasets for AI and RAG. Get clean text (and HTML) of free posts for summaries, embeddings or training pipelines.
• Audience and sentiment research. Collect the comments under a post, with their reply structure.
• Lead and creator discovery. Search a topic, list the newsletters that write about it, and rank them by likes per post.
• Recurring feeds. Use postsSince to pull only the last day or week on a schedule.

Paywalled posts return only the public preview Substack shows to anonymous readers. They are marked isPaywalled: true and isBodyTruncated: true, so you always know which text is complete.

Input

Field	Type	What it does
publications	list of text	Newsletter URLs or subdomains: https://www.lennysnewsletter.com, https://lenny.substack.com or lenny. Custom domains work.
postUrls	list of text	Single posts, https://<newsletter>/p/<slug>.
searchQueries	list of text	Keywords to search across all of Substack.
searchType	posts / publications	posts (default) returns the matching posts. publications finds matching newsletters and returns their latest posts.
sort	new / top	Order of a newsletter's archive. Default new.
maxPostsPerPublication	number	Cap per newsletter. Default 20. Also your cost cap per newsletter.
maxPostsPerSearch	number	Cap per search query (posts search). Default 20.
maxPublicationsPerSearch	number	With searchType: publications, how many newsletters to read per query. Default 5.
postsSince	text	Skip older posts. A date (2026-10-01) or relative time (7 days, 2 weeks, 3 months).
includeBody	true/false	Fetch the full text and HTML of each post. Default true, no extra charge.
includeComments	true/false	Also return the comments of each post. Default false.
maxCommentsPerPost	number	Cap per post when comments are on, replies included. Default 50.
requestDelaySeconds	number	Pause between requests, default 0.5 s (randomised up to 1.4x).
proxyConfiguration	object	Optional Apify Proxy. Default is no proxy.

{
  "publications": ["https://www.lennysnewsletter.com", "https://www.astralcodexten.com"],
  "searchQueries": ["ai agents"],
  "maxPostsPerPublication": 50,
  "maxPostsPerSearch": 30,
  "postsSince": "30 days",
  "includeComments": true,
  "maxCommentsPerPost": 100
}

Posts are deduplicated across all inputs of a run.

Output

Two row types share the dataset, told apart by type.

Post row (type: "post")

Field	Meaning
id, title, subtitle, url, slug	Post identity and link.
publicationName, publicationUrl	The newsletter the post belongs to.
authors	List of {name, handle, url}.
publishedAt	Publication time, ISO 8601 UTC.
postType	newsletter, podcast, thread and so on.
audience, isPaywalled, isBodyTruncated	everyone / only_free / only_paid / founding; whether the post is paid; whether bodyText is only the public preview.
wordCount	Word count of the whole post, also for paywalled ones.
likes, reactions, commentCount, restacks	Engagement. reactions maps each emoji to its count.
coverImage, tags	Cover image URL and post tags.
bodyText, bodyHtml	Plain text with paragraph breaks, and the original HTML. With includeBody off, bodyText is the short preview and bodyHtml is null.
scrapedAt, input	ISO 8601 time of the scrape and the input value the row came from.

{
  "type": "post",
  "id": 215694124,
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "slug": "all-of-the-lenny-and-friends-summit",
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "authors": [{"name": "Lenny Rachitsky", "handle": "lenny", "url": "https://substack.com/@lenny"}],
  "publishedAt": "2026-09-29T13:15:57+00:00",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "isBodyTruncated": false,
  "wordCount": 1276,
  "likes": 281,
  "reactions": {"❤": 281},
  "commentCount": 6,
  "restacks": 5,
  "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-....jpeg",
  "tags": [],
  "bodyText": "Hey there, I'm Lenny. Each week, I share deeply researched product, growth, and career advice ...",
  "bodyHtml": "<p>Hey there, I'm Lenny ...</p>",
  "scrapedAt": "2026-10-09T10:12:14+00:00",
  "input": "https://www.lennysnewsletter.com"
}

Comment row (type: "comment")

{
  "type": "comment",
  "id": 349085941,
  "postId": 215694124,
  "postUrl": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "author": "Product Mindset",
  "authorHandle": "productmindset",
  "body": "Thank you Lenny for putting this together ...",
  "likes": 3,
  "restacks": 0,
  "createdAt": "2026-09-29T23:34:47+00:00",
  "editedAt": null,
  "parentId": null,
  "scrapedAt": "2026-10-09T10:12:14+00:00",
  "input": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit"
}

parentId is the id of the comment a reply answers, and null for top-level comments. A reply comes right after its parent.

Pricing

Pay per event, with the prices final:

• $0.0015 per post ($1.50 per 1,000 posts), full text of free posts included
• $0.0003 per comment ($0.30 per 1,000 comments)

No start fee. You pay only for rows you receive: nothing for newsletters that are unavailable, failed requests or retries, and the run stops by itself when your maximum total charge is reached. Platform usage is included.

Example: 10 newsletters with 100 posts each is 1,000 posts, so 1,000 x $0.0015 = $1.50. Add 20 comments on each of 200 posts and you pay 4,000 x $0.0003 = $1.20 more.

Use it from the API

curl -X POST "https://api.apify.com/v2/acts/datagleaner~substack-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"publications": ["lenny"], "maxPostsPerPublication": 10}'

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("datagleaner/substack-scraper").call(run_input={
    "searchQueries": ["ai agents"], "maxPostsPerSearch": 25, "postsSince": "7 days"})
for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(post["publishedAt"], post["title"], post["likes"])

Use it from n8n, Make, Zapier or an AI agent

Actor ID: datagleaner/substack-scraper

Minimal input:

{"publications": ["https://www.lennysnewsletter.com"], "maxPostsPerPublication": 5}

Each tool below runs this Actor with your own Apify API token.

• n8n: add the Apify node (@apify/n8n-nodes-apify). On n8n Cloud you install it from the community node registry. Choose Run an Actor and get dataset, set Actor to datagleaner/substack-scraper and paste the input above.

• Make: use the Apify app's Run an Actor module, then Get Dataset Items to read the results. Watch Actor Runs can trigger a scenario when a run finishes.

• Zapier: use the Apify action Run Actor, then the search Fetch dataset items. The trigger Finished Actor run starts a Zap when a run ends.

• AI agents (MCP): connect to https://mcp.apify.com/?tools=datagleaner/substack-scraper. Without a token, an agent can still find Actors with the search-actors tool, but running this one needs your Apify account. In Claude Code:

    claude mcp add --transport http apify "https://mcp.apify.com/?tools=datagleaner/substack-scraper"
  

  Then run /mcp to sign in to Apify in your browser. Then ask in plain language, for example: "Use the Substack scraper to get the latest 10 posts of lennysnewsletter.com and summarise in English what each one argues, with the like counts."

• LangChain (Python):

# pip install langchain-apify, then set APIFY_TOKEN in your environment
import json
from langchain_apify import ApifyActorsTool
tool = ApifyActorsTool("datagleaner/substack-scraper")
result = tool.invoke({"run_input": json.loads('{"publications": ["https://www.lennysnewsletter.com"], "maxPostsPerPublication": 5}')})

Limits

• Paywalled posts return only the public preview. Nothing behind a paywall is fetched.
• Search returns what Substack's own search ranks first; posts, not notes. It pages until your cap or until Substack stops returning results.
• Comments are the ones Substack shows anonymous readers. Posts with comments limited to subscribers still show them to readers where Substack does.
• Very large newsletters (thousands of posts) take a while at the default pace; lower requestDelaySeconds carefully.

FAQ

Do I need a Substack account or API key? No. Only public data is read.

Does it work on custom domains? Yes, any newsletter that is public on the web, whether name.substack.com or its own domain.

Why is bodyText short for some posts? Those are paywalled posts. isBodyTruncated is true and wordCount still tells you the real length.

Can it get a newsletter's subscriber count? Substack does not show it publicly for most newsletters, so it is not returned.

What happens if I start it with no input? It runs a small built-in example, the 5 latest posts of Lenny's Newsletter (5 x $0.0015 = $0.0075), and says so in the run's status message. Give publications, postUrls or searchQueries for your own data.

What happens if a newsletter does not exist? It is skipped, reported in the log and in the SUMMARY record, and not billed.

Responsible use

This Actor reads publicly available data only. You are responsible for how you use it, including copyright in the text you collect and any personal-data rules (for example GDPR) that apply to author names and comments. Do not use it to harass people or to republish paywalled content.