Substack Scraper

Scrape any Substack newsletter: every post with title, date, authors, tags, likes, comments and restacks, the full text ready for AI and RAG, all comments and replies, and the newsletter itself with author and subscriber count. Find newsletters by keyword with the built-in Substack search.

The scraper reads Substack's public JSON API with plain HTTP requests through Apify Proxy. No login, no browser. A newsletter with 100 posts takes a few seconds.

What you can use it for

• Content and trend research: pull the archive of the newsletters in your niche and see which topics get the most likes, comments and restacks.
• AI and RAG: feed clean post text into your vector database, chatbot or summarizer.
• Sponsor and partner research: find newsletters by keyword and compare subscriber counts.
• Audience research: read what readers actually say in the comments.
• Monitoring: schedule a daily run with dateFrom and get only the new posts.

Input

Field	Description	Default
startUrls	Newsletters or posts, one per line: https://name.substack.com, a custom domain, just name, or a post URL .../p/slug
searchQueries	Keywords to find newsletters on Substack
maxPublicationsPerSearch	Newsletters taken from each search	10
maxPostsPerPublication	Newest posts first, 0 reads the whole archive	20
dateFrom	Only posts published on or after this date, YYYY-MM-DD
onlyFree	Skip paid posts	false
includeContent	Add bodyText and bodyHtml	false
includeComments	Add comments and replies as rows	false
maxCommentsPerPost	Comments per post, 0 means no limit	100
includePublicationInfo	One row per newsletter with subscriber count	true

At least one start URL or search query is required.

{
    "startUrls": ["https://www.lennysnewsletter.com", "astralcodexten"],
    "searchQueries": ["climate"],
    "maxPostsPerPublication": 50,
    "dateFrom": "2026-01-01",
    "includeContent": true,
    "includeComments": true
}

Output

Every row has a type: post, comment or publication. The dataset has a view for each.

A post:

{
    "type": "post",
    "postId": 208730073,
    "title": "Announcing Lenny’s Jobs: The best place in the world to find, vet, and land your dream job",
    "subtitle": "Where product managers, engineers, designers, and growth/marketing professionals discover high-quality open roles at tech companies",
    "url": "https://www.lennysnewsletter.com/p/announcing-lennys-jobs-the-best-place",
    "postDate": "2026-08-18T15:40:06.921Z",
    "postType": "newsletter",
    "audience": "everyone",
    "isPaywalled": false,
    "wordCount": 933,
    "reactionCount": 378,
    "commentCount": 18,
    "restackCount": 9,
    "authors": ["Lenny Rachitsky"],
    "authorHandles": ["lenny"],
    "tags": ["Career"],
    "section": null,
    "coverImage": "https://substackcdn.com/image/fetch/...png",
    "podcastUrl": null,
    "language": "en",
    "publicationId": 10845,
    "publicationName": "Lenny's Newsletter",
    "publicationUrl": "https://www.lennysnewsletter.com",
    "bodyText": "Each week, I share deeply researched product, growth, and career advice...",
    "bodyHtml": "<p>...</p>",
    "isFullText": true
}

A comment:

{
    "type": "comment",
    "commentId": 319080271,
    "parentCommentId": 317678273,
    "postId": 208730073,
    "postUrl": "https://www.lennysnewsletter.com/p/announcing-lennys-jobs-the-best-place",
    "postTitle": "Announcing Lenny’s Jobs: ...",
    "authorName": "Lenny Rachitsky",
    "authorHandle": "lenny",
    "date": "2026-08-20T21:05:49.819Z",
    "body": "Will think about this...",
    "reactionCount": 1,
    "replyCount": 0,
    "publicationName": "Lenny's Newsletter"
}

A newsletter:

{
    "type": "publication",
    "publicationId": 10845,
    "publicationName": "Lenny's Newsletter",
    "publicationUrl": "https://www.lennysnewsletter.com",
    "subdomain": "lenny",
    "customDomain": "www.lennysnewsletter.com",
    "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
    "language": "en",
    "authorName": "Lenny Rachitsky",
    "authorHandle": "lenny",
    "authorBio": "Writing • Angel investing • Advising",
    "freeSubscriberCount": 1200000,
    "subscriberRanking": "Millions of subscribers",
    "paidSubscriberRanking": "Tens of thousands of paid subscribers",
    "paidSubscriptionsEnabled": true,
    "createdAt": "2019-06-01T15:35:37.885Z"
}

Export the results as JSON, CSV, Excel or HTML, or pull them through the Apify API, n8n, Make or Zapier.

Pricing

You pay per row in the dataset (post, comment or newsletter), see the pricing tab. Full text costs nothing extra. Newsletters that return no posts cost only their newsletter row, which you can switch off.

Limits

• Paid posts contain only the public preview that Substack shows to non-subscribers. isFullText tells you which rows are complete. Some paid posts have no preview at all, then bodyText is null.
• freeSubscriberCount is the number Substack displays publicly. Not every newsletter shows one.
• Profile pages (substack.com/@name) and Notes are not supported. Use the newsletter address.

FAQ

Do I need a Substack account? No. Only public data is read.

Can I scrape a whole archive? Yes, set maxPostsPerPublication to 0.

How do I get only new posts every day? Schedule the Actor and set dateFrom, or keep maxPostsPerPublication small, since the newest posts come first.