Substack Newsletter Scraper

Scrape any Substack publication — posts, full text, comment threads and newsletter recommendation graphs. No API key, no login, no phone number, no proxy required.

Point it at a handle (astralcodexten), a subdomain (bigtechnology.substack.com), a custom domain (newsletter.pragmaticengineer.com) or any post URL. All four work.

Modes

posts — one row per post

Title, subtitle, publish date, author, section, tags, wordcount, likes, comment count, restacks, paid/free status, cover image, podcast duration and canonical URL.

Turn on Include full post text to also get body_html and clean content_text. Free posts return in full; paid posts return the free preview Substack serves to logged-out visitors.

comments — one row per comment

The whole reply tree, flattened, with depth and parent_comment_id so you can rebuild the thread. Includes author name, handle, date, body and reaction count, stamped with the post it belongs to.

recommendations — discover newsletters in a niche

The publications each seed recommends, with name, description, author, subdomain and URL. Feed those subdomains back in to walk the graph outward — a practical way to build a list of newsletters in one topic for sponsorship or competitor research.

Useful options

Option	What it does
Search within publication	Substack's own search, e.g. only posts mentioning openai.
Sort	Newest first, or the publication's most popular posts.
Only posts after	ISO date. Stops at the first older post, so a daily run collects only what is new.
Max posts per publication	Archive rows per publication, 50 per request.
Max comments per post	Caps comments kept per post, replies included.

Example input

{
  "publications": ["astralcodexten", "bigtechnology"],
  "mode": "posts",
  "maxPostsPerPublication": 100,
  "sort": "new",
  "includeContent": false
}

Example output (posts)

{
  "publication": "astralcodexten.substack.com",
  "publication_id": 89120,
  "post_id": 209742524,
  "slug": "open-thread-446",
  "title": "Open Thread 446",
  "post_date": "2026-08-10T16:17:33.848Z",
  "type": "newsletter",
  "audience": "everyone",
  "is_paid": false,
  "author": "Scott Alexander",
  "wordcount": 516,
  "reaction_count": 74,
  "comment_count": 729,
  "url": "https://www.astralcodexten.com/p/open-thread-446"
}

Notes

• Only publicly visible content is read. Nothing paywalled is bypassed and no account is used.
• Some newsletters keep their domain after leaving Substack. Those are reported with a clear message and the run continues with the rest.
• Runs stop cleanly before the platform timeout and keep everything collected so far, rather than failing.

Pricing

Pay per result:

Event	What it is
post	One post from the archive.
post-content	One post including its full text (one extra request).
comment	One comment.
recommendation	One recommended publication.

Support

Found a field that stopped populating, or a publication that won't load? Open an issue on the Actor's page and include the input you used.