Substack Posts Scraper

Substack scraper for full newsletter archives, with no login and no API key. Pull every post from any Substack publication (name.substack.com or a custom domain like newsletter.pragmaticengineer.com): title, date, authors, paywall flag, reactions, comments, and optionally the full body as HTML and plain text. It uses Substack's public JSON archive API, so runs are fast and you pay per result.

What you can do with it

• Newsletter monitoring — schedule sort: new for the publications you follow, push new posts to Slack, email or an LLM summarizer.
• Competitor & creator research — compare posting cadence, wordcount, reactions, comments and restacks across publications; sort: top lists a newsletter's most popular posts.
• Content datasets — includeBody: true adds body_html and body_text for RAG, topic analysis or archiving.
• Paywall mapping — paywalled shows which posts are paid-only, so you can measure a publication's free / paid mix.
• Automated feeds — Schedule it on Apify and pipe results to Google Sheets, Slack, n8n or Make via the Apify integration.
• Podcast episode lists — filter rows by post_type: podcast to get a publication's episodes with podcast_url.

Output example

{
  "type": "post",
  "publication": "newsletter.pragmaticengineer.com",
  "id": 222810010,
  "title": "The Pulse: Anthropic kills the coding interview",
  "subtitle": "AI is rendering the Leetcode-style interview pointless …",
  "slug": "the-pulse-anthropic-kills-the-coding",
  "url": "https://newsletter.pragmaticengineer.com/p/the-pulse-anthropic-kills-the-coding",
  "post_date": "2026-10-30T16:37:51.401Z",
  "post_type": "newsletter",
  "audience": "only_paid",
  "paywalled": true,
  "authors": ["Gergely Orosz"],
  "description": "AI is rendering the Leetcode-style interview pointless …",
  "cover_image": "https://substackcdn.com/image/fetch/…",
  "wordcount": 4545,
  "reactions": 49,
  "comment_count": 2,
  "restacks": 1,
  "podcast_url": null
}

With includeBody: true each row also has body_html and body_text.

Input

Field	Description
publications	List of newsletters: stratechery, stratechery.substack.com, a custom domain, or any post URL (reduced to its host).
sort	new (default) or top.
includeBody	Default false. Adds body_html and body_text (one extra request per post). Paywalled posts only return the free preview, if any.
maxItems	Stop after N posts total across all publications. Default 100.

Pricing

Pay per item: one event per post row. Nothing is charged for posts that are never produced.

Notes

• post_type is newsletter, podcast or thread; reactions is the sum of all reaction counts; podcast_url is set for podcast posts.
• Domains that are not Substack publications return no posts: a warning is logged and the run continues.
• Posts are paged 50 per request; very large archives (thousands of posts) are bounded by maxItems.

FAQ

Do I need an API key or account?

No. It only reads Substack's public archive and post endpoints, without logging in. Requests are paged 50 posts at a time; very frequent runs on one publication may be rate-limited.

How much does it cost?

$0.002 per result and nothing else: with pay-per-event pricing you are not billed for Apify compute. 100 results cost $0.20, and the Apify free plan's $5 monthly credit covers about 2,500 results. Rows that are never produced are never charged.

How do I get the data into Google Sheets or my app?

Export the run's dataset as CSV, JSON or Excel from the Apify Console, read it from the dataset API endpoint, or connect it to Google Sheets, Slack, n8n, Make or Zapier with Apify integrations.

Is it legal to scrape Substack?

It only reads public data via the platform's public API and respects rate limits. You are responsible for complying with Substack's terms and applicable law.

Can I get the full text of paywalled posts?

No. With includeBody: true, paywalled posts only return the free preview the publication shows to logged-out readers, if any. The paywalled field tells you which posts are affected.

Related Actors

• Hacker News Scraper
• Wikipedia Scraper
• Dev.to Articles Scraper