Substack Newsletter Scraper – Posts, Comments & Publication Stats

Scrape any public Substack newsletter into clean rows: one row per post, optionally one row per comment, and one summary row per publication with subscriber count, posting cadence, paid share, average engagement and top posts.

Works with name.substack.com, custom domains (lennysnewsletter.com, noahpinion.blog, …) and single post URLs. Uses Substack's public JSON API – no login, no API key, no proxies, no browser. Three publications with 15 posts and 258 comments take about 3 seconds.

What you get

Posts (type: "post") – title, subtitle, URL, post type (newsletter / podcast / video / thread), free or paid, post date, authors, word count, reactions, restacks, comment count, cover image, tags, podcast audio URL, public preview text, and optionally the full body (HTML + plain text).

Comments (type: "comment", with includeComments) – comment text, author name and handle, date, reactions, restacks, reply count, nesting depth and parent, whether the comment is by the post author, and the commenter's membership (free / paid).

Publication summary (type: "publication") – subscribers (free count, e.g. 1,200,000) and paid-subscriber tier ("Tens of thousands of paid subscribers"), bestseller tier, leaderboard ranking, description, logo, created date, paid plan enabled, author name / bio / followers / social links, posts per week, posts in the last 30 days, paid share, average reactions / comments / restacks / word count, post-type mix, top tags, latest and oldest post, top 3 posts by reactions.

Use cases

• Sponsorship / PR prospecting – filter newsletters by subscribers, cadence and engagement in one run.
• Newsletter research & benchmarking – compare dozens of publications side by side.
• Audience research – who comments, what they say, which posts spark discussion.
• Content monitoring – run on a schedule with onlyNewPosts to get only new posts each time.
• LLM / RAG datasets – includeBody returns clean plain text per post.

Input

Field	Description
publications (required)	lenny, on.substack.com, https://www.lennysnewsletter.com, or a post URL …/p/slug.
maxPostsPerPublication	Max post rows per publication (default 50, max 5000).
sortBy	new (default) or top.
audience	all (default), free, paid.
keywords	Search terms – uses Substack's own search (title + body), one query per keyword.
dateFrom	Keep posts on/after this date.
onlyNewPosts	Incremental mode for scheduled runs – only posts newer than the last run.
includeBody	Fetch the full body of each post (extra request per post, separate event).
includeComments	Fetch comments with replies (extra request per post with comments, per-comment event).
maxCommentsPerPost	Max comment rows per post (default 100).
includePublicationInfo	Save the summary row (default true).

{
  "publications": ["https://www.lennysnewsletter.com", "https://www.noahpinion.blog", "on.substack.com"],
  "maxPostsPerPublication": 50,
  "includeComments": true
}

Output examples

{
  "type": "publication",
  "publication": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "status": "OK",
  "subscribers": 1200000,
  "subscribersLabel": "1,200,000 subscribers",
  "paidSubscribersLabel": "Tens of thousands of paid subscribers",
  "bestsellerTier": 10000,
  "authorName": "Lenny Rachitsky",
  "createdAt": "2019-06-01T15:35:37.885Z",
  "postsPerWeek": 1.9,
  "paidShare": 0.52,
  "avgReactions": 412.6,
  "avgComments": 9.4,
  "topPosts": [{ "title": "How to turn your AI into a world-class designer", "url": "…", "reactions": 1113 }]
}

{
  "type": "comment",
  "publication": "Noahpinion",
  "postTitle": "Maybe the Rapture is already here",
  "author": "Reader Name",
  "authorHandle": "readername",
  "isPostAuthor": false,
  "membership": "paid",
  "date": "2026-10-09T10:12:00.000Z",
  "body": "…",
  "reactions": 64,
  "replies": 3,
  "depth": 0
}

Pricing (pay per event)

• post – one post row
• post-body – full body fetched for a post (only with includeBody)
• comment – one comment row (only with includeComments)
• publication – one publication summary row

Publications that cannot be read (NOT_FOUND, ERROR) and bodies that fail to download are not charged. Platform usage is included.

Limitations

• Only public data: paywalled posts return metadata and the public preview, never the paid body.
• Subscriber counts are the public figures Substack shows on the publication page / author profile; publications that hide them (e.g. Substack's own company newsletters) return null.
• Substack Notes are not included.
• A few publications rate-limit body downloads aggressively; the scraper backs off and retries, and the rare failed body is left empty and not charged.