Substack Scraper — posts, engagement and paywall status in one schema

Scrape any Substack publication's full archive with engagement already turned into numbers: reactions, comments, restacks, word count, and whether each post sits behind the paywall.

Why this one

It gets the whole archive. Substack's archive endpoint returns 23 posts on the first page however large a limit you ask for, and 50 on every page after it. A scraper that treats "fewer than requested" as the end of the data stops at 23 and reports that as an entire publication. This pages until the archive actually ends — verified past 573 posts on a single publication.

Reactions come back as a number. Substack returns them keyed by emoji: {"❤": 21}. That breaks CSV exports and forces you to guess which emoji a given publication uses. You get reactionCount as an integer, plus topReaction if you want the emoji.

Custom domains survive. The Free Press publishes from the bariweiss slug but lives at thefp.com. url is the canonical one, so links actually work.

Paywall status as a boolean. isPaid instead of decoding audience values like only_paid and founding yourself.

Input

{
  "publications": ["noahpinion", "platformer", "www.thefp.com"],
  "sort": "new",
  "audience": "all",
  "searchQuery": "",
  "postedWithinDays": 0,
  "maxItemsPerPublication": 100
}

Slugs, subdomains and custom domains all work: noahpinion, bariweiss.substack.com and www.thefp.com are all valid.

Output

One row per post:

{
  "postId": "212192141",
  "publication": "www.thefp.com",
  "title": "'It's Just Dog Racism'",
  "subtitle": "'The myth of the pit bull is a lot of bullshit,' said actor Jon Bernthal.",
  "url": "https://www.thefp.com/p/its-just-dog-racism",
  "postDate": "2026-08-21T20:29:06.512Z",
  "type": "newsletter",
  "audience": "only_paid",
  "isPaid": true,
  "reactionCount": 21,
  "topReaction": "❤",
  "commentCount": 9,
  "restacks": 0,
  "wordCount": 2049,
  "sectionName": null
}

A RUN_SUMMARY record reports per-publication counts and any that could not be found.

Who this is for

• Newsletter operators benchmarking their engagement against comparable publications
• Media researchers tracking what gets read, restacked and paywalled
• Content teams mining a niche's archive for topics that landed
• Investors and analysts sizing a publication before a deal

Notes

• Reads Substack's public archive API. No login, no cookies, no CAPTCHA solving, no proxies.
• Post bodies are not included; this returns metadata and engagement.
• maxItemsPerPublication defaults to 100. Set it to 0 for the entire archive, which on a long-running publication can be thousands of posts.