Substack Scraper (Posts, Publications, Comments)

Get posts from any Substack newsletter, including ones on their own custom domain. Paste newsletter names or URLs, or find newsletters by keyword or category, and get back one clean JSON row per post: title, subtitle, authors, publish date, post type, paywall flag, word count, likes, comment count, restacks and cover image. Optionally add the full post text, the comments, and one row of publication info per newsletter.

You only pay for publications that succeed. Each publication gets a bounded number of attempts (3 by default, at most 5), each on a fresh proxy session that resumes where the last one stopped. A publication that still fails is listed in the FAILED_PUBLICATIONS record and none of its rows are saved or charged. A newsletter that does not exist fails at once, without retries. Every request has a timeout and every publication a post cap, so a run never hangs until the platform timeout.

It uses the same public JSON that Substack's own website loads for logged-out readers. No login, no cookies.

What you can use it for

• Newsletter research: compare posting frequency, length and engagement across the newsletters in a niche.
• Content monitoring: schedule a daily run with Newest and Published since to catch new posts from a list of newsletters.
• Lead and sponsor lists: search a keyword or category and get each newsletter's name, author, description and Substack's subscriber tier.
• Audience analysis: pull the comments on a newsletter's most-discussed posts.
• AI and BI pipelines: a flat, predictable row per post, ready for an LLM, a spreadsheet or a dashboard.

How to use it

1. Add publications, one per line. Any of these work:
   • example (the name before .substack.com)
   • example.substack.com or https://example.substack.com
   • https://www.example-newsletter.com (a custom domain)
   • any post URL on either, e.g. https://example.substack.com/p/some-post
2. Or let the Actor find newsletters: add search queries (Substack's publication search) and/or category ids (e.g. 4 Technology, 62 Business, 153 Finance, 134 Science, 96 Culture). Publications per search or category caps how many each one adds.
3. Choose sort (Newest or Top), the maximum posts per publication, and optionally published since.
4. Turn on Fetch full post text, Fetch comments or Add a publication row if you need them. Each adds requests, so leave them off when you only need post metadata.
5. Click Start.

Example input

{
  "publications": ["example", "https://www.example-newsletter.com"],
  "searchQueries": ["climate"],
  "maxPublicationsPerQuery": 5,
  "maxPostsPerPublication": 50,
  "sort": "new",
  "publishedSince": "2026-01-01",
  "includeBody": false,
  "includeComments": false,
  "includePublicationInfo": true
}

Output

Every row has a type: post, comment or publication. The dataset has a view for each.

A post (the newsletter here is fictional):

{
  "type": "post",
  "publication": "The Example Gazette",
  "publicationUrl": "https://examplegazette.substack.com",
  "postId": 123456789,
  "slug": "why-small-towns-are-growing-again",
  "title": "Why small towns are growing again",
  "subtitle": "Five charts on the remote-work migration",
  "url": "https://examplegazette.substack.com/p/why-small-towns-are-growing-again",
  "canonicalUrl": "https://examplegazette.substack.com/p/why-small-towns-are-growing-again",
  "authors": ["Jane Example"],
  "publishedAt": "2026-10-20T13:00:00+00:00",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "section": null,
  "wordCount": 1840,
  "reactionCount": 412,
  "commentCount": 57,
  "restacks": 23,
  "coverImageUrl": "https://substack-post-media.s3.amazonaws.com/public/images/example_1456x816.png",
  "description": "Five charts on the remote-work migration",
  "podcastUrl": null,
  "scrapedAt": "2026-10-28T12:00:00+00:00"
}

• postType is Substack's own type: newsletter, podcast, thread or video.
• audience is everyone, paid (paid or founding members only) or freeSubscribers (free sign-up required). isPaywalled is true for anything but everyone.
• With Fetch full post text, post rows also get bodyHtml, bodyText and isPreviewOnly.

A comment (with Fetch comments):

{
  "type": "comment",
  "publication": "The Example Gazette",
  "publicationUrl": "https://examplegazette.substack.com",
  "postId": 123456789,
  "postUrl": "https://examplegazette.substack.com/p/why-small-towns-are-growing-again",
  "commentId": 987654321,
  "parentCommentId": null,
  "author": "Sample Reader",
  "body": "Our town doubled its library hours last year, so this tracks.",
  "date": "2026-10-20T15:42:10+00:00",
  "editedAt": null,
  "likes": 12,
  "replyCount": 2,
  "scrapedAt": "2026-10-28T12:00:00+00:00"
}

Replies have parentCommentId set to the comment they answer.

A publication row (with Add a publication row) has publication, publicationUrl, publicationId, subdomain, customDomain, description, author, subscribersText (Substack's own wording, e.g. "Tens of thousands of subscribers", or null when the newsletter hides it), rankingDetail, language, paidSubscriptions, hasPodcast, createdAt, firstPostDate and logoUrl.

Publications that failed are listed in the key-value store record FAILED_PUBLICATIONS as {publication, error, attempts}, for example not found (HTTP 404 for https://no-such-name.substack.com/).

Pricing

You pay per result: one result per dataset row (post, comment or publication row). Publications that fail are never pushed to the dataset, so they cost nothing, and a run where every publication fails exits with an error and charges nothing. Use Maximum rows in total to cap a run's cost. See the Pricing tab for current rates.

Limitations

• Paid posts are previews. For paid-only posts Substack gives logged-out readers only the free preview, so bodyText is the preview, isPreviewOnly is true and wordCount is Substack's count for the whole post.
• Comments depend on the newsletter's settings. Many newsletters show comments only to subscribers or only on free posts; those posts return no comment rows even when commentCount is above zero.
• Subscriber counts are tiers, not numbers. Substack publishes ranges ("Thousands of subscribers"), and some newsletters hide them.
• Top with a date filter scans at most 10x the post cap (at least 500 posts) per publication, so very old top posts may be missed.
• Search and category results come in Substack's own order and are capped by Publications per search or category.
• Substack can change these undocumented endpoints without notice. When that happens, the affected publications land in FAILED_PUBLICATIONS (not charged) and a fix follows.

Tips

• Proxy: the default Apify proxy (datacenter) is usually enough. If the log shows many retries, switch to residential.
• Large lists: keep Publications in parallel at 2-4.
• Monitoring: run on a schedule with Newest and Published since, then deduplicate on postId.

Legal and data

This Actor reads newsletter posts that publishers make public on Substack, the same data any logged-out visitor sees. It stores author and commenter names exactly as Substack displays them next to the post or comment. It does not collect subscriber lists, emails, user ids, profile links, profile pictures or any paid content beyond the free preview. You are responsible for using the data in line with Substack's terms, the authors' copyright and the privacy law that applies to you (for example GDPR).

FAQ

Why does a run return fewer posts than I asked for? The newsletter has fewer posts, or Published since cut the list.

Does it work on custom domains? Yes. Paste the domain or any post URL on it.

Something broke? Open an issue on the Issues tab with your run ID.