Substack Scraper — Posts, Notes, Profiles & Leaderboards

One actor for every public Substack surface: full posts with complete article HTML, nested comment threads, Substack Notes search, author profiles with subscriber signals, publication profiles, and category leaderboards. No login, no cookies, no proxy configuration, no code.

Everything this Actor covers

What you want	How	What you get
Posts from a publication	publications: ["astralcodexten.com"]	Title, subtitle, date, authors, likes, comment count, restacks, cover image, podcast audio URL & duration, paywall flag
Full article content	toggle Include full article content	Complete bodyHtml + clean bodyText + word count — ready for research, RAG, and AI datasets
Single posts	postUrls: ["https://…/p/slug"]	The same rich record for exactly the posts you point at
Comment threads	toggle Include comment threads	The full nested tree, flattened with parentId + depth, author, likes
Search inside a publication	Search within publications	Server-side archive search (sort: new/top, date window)
Author profiles	authorHandles: ["@handle"]	Follower count, subscriber signals ("Thousands of subscribers"), bestseller tier, bio, social links, primary publication
Authors' Notes	toggle Include authors' recent Notes	Their recent short-form Notes with engagement
Notes keyword search	noteSearchQueries: ["ai agents"]	Platform-wide Notes matching your query: text, author, likes, restacks, images
Category leaderboards	leaderboardCategories: ["technology"]	Ranked publications per category with subscriber signals, paid tier, author — the market map of Substack

Works with custom domains (astralcodexten.com, slowboring.com), *.substack.com subdomains, bare names, and full URLs — all input forms auto-normalize, and redirect/www variants are retried automatically instead of failing with a 404.

Honest paywall handling — read this

• Paywalled posts are delivered as metadata with paywalled: true and an empty body.
• They are never charged at the full-content price — you pay the metadata price only.
• This Actor does not bypass paywalls, ever. If you need paid-post bodies, subscribe to the publication.

Output example (post with content)

{
  "type": "post",
  "publicationHost": "www.astralcodexten.com",
  "id": "212058143",
  "slug": "your-review-tomorrow-and-tomorrow",
  "title": "Your Book Review: Tomorrow, and Tomorrow, and Tomorrow",
  "url": "https://www.astralcodexten.com/p/your-review-tomorrow-and-tomorrow",
  "postDate": "2026-08-22T13:01:33.000Z",
  "postType": "newsletter",
  "audience": "everyone",
  "paywalled": false,
  "authors": [{ "id": 12009663, "name": "Scott Alexander", "handle": "astralcodexten" }],
  "reactionCount": 214,
  "commentCount": 361,
  "podcast": null,
  "bodyHtml": "<p>…full article HTML…</p>",
  "bodyText": "…clean plain text…",
  "wordCount": 5843,
  "contentRetrieved": true
}

Every record type (post, comment, note, author-profile, publication-profile, leaderboard-entry, status) is a flat, documented shape. Export JSON, CSV, Excel, or read via API.

Pricing — what a run actually costs

Pay per delivered record. Zero results = zero record charges, with a status record explaining why.

Event	Price
Actor start	$0.005
Post (metadata)	$0.002
+ Full article content (only when actually delivered)	+$0.002
Comment	$0.0005
Note	$0.001
Author profile	$0.005
Publication profile	$0.005
Leaderboard entry	$0.001

Worked examples: 25 latest posts with full content = $0.105. A 100-entry technology leaderboard = $0.105. 50 notes on a keyword = $0.055. An author profile + their 50 recent notes = $0.06.

What this Actor does NOT do

• ❌ No paywall bypass — paid post bodies stay with paying subscribers.
• ❌ No global post search — Substack's site-wide post search requires a logged-in session, so it is not offered. Per-publication archive search is offered (server-side), and Notes search is platform-wide.
• ❌ No people search by keyword — that endpoint is login-gated; profile lookup by handle is offered.
• ❌ No private data — emails, subscriber lists, or revenue are not public and are not returned. Subscriber signals are the same rounded figures Substack shows publicly ("Thousands of subscribers", bestseller tiers).
• ❌ No exact subscriber counts where Substack publishes only rounded/hidden values — you get null or the public text, never a guess.

Reliability notes

• Pure HTTP against Substack's public JSON endpoints — fast and cheap (the whole default demo run finishes in ~2 s).
• Polite pacing + automatic retry with backoff on rate limits.
• Every cap (maxPostsPerPublication, maxItemsTotal, …) is honored exactly, so cost is bounded before you press Start.
• If anything fails (unknown publication, bad handle, unknown category), you get a plain-English status record and no charge for it.

Fair-use & compliance

This Actor reads the same public, unauthenticated endpoints that power Substack's own web pages, at a polite request rate, and collects only published content and public profile data. You are responsible for complying with applicable laws, Substack's terms, and content licenses for your use case. Respect authors: cite what you quote.

---

Missing an angle you need? Open an issue on the Actor's Issues tab — the roadmap is complaint-driven.

More data tools by Automation Craft

• ATS Job Scraper — Greenhouse, Lever, Ashby, Workable & SmartRecruiters in one unified schema.
• LinkedIn Jobs Scraper — LinkedIn public job search, no login.
• G2 Reviews Scraper — G2 reviews without DataDome headaches.