Substack Scraper - Posts, Authors & Search

Thirdwatch portfolio: 8,800+ users across 140+ public Actors, 2M+ records delivered, >95% run success. Explore all Thirdwatch Actors.

Scrape Substack publications — post titles, dates, authors, and full text from newsletter archives.

What you get

Scrape Substack publications — post titles, dates, authors, and full text from newsletter archives.

Why use this Substack scraper

• Archive coverage — posts from any public Substack publication.
• Post fields — title, publish date, author, excerpt, and canonical URL; optional full text.
• Keyword filtering — restrict to posts matching a topic.

Scrape Substack data in minutes

{
  "publicationUrls": [
    "https://www.platformer.news"
  ]
}

{
  "publicationUrls": [
    "https://newsletter.pragmaticengineer.com"
  ]
}

Input parameters

Parameter	Required	Description
publicationUrls	Yes	Substack publication URLs or hostnames, e.g. https://newsletter.pragmaticengineer.com or platformer.news. Custom domains work too.
search	No	Only return posts matching this search inside each publication. Leave empty for the latest posts.
maxPosts	No	Maximum posts to return per publication. Default: 20.
includeContent	No	Fetch each post's full HTML body and plain text. Slower and roughly doubles request count. Default: False.
sort	No	Substack archive sort order. One of: new, top. Default: new.

Output fields

• title
• subtitle
• publication
• url
• slug
• postDate
• type
• author
• reactions
• commentCount
• coverImage
• audience
• description
• podcastUrl

Example result

{
  "title": "How the best engineering teams ship faster",
  "subtitle": "A deep dive into the processes, tooling, and culture behind high-output engineering organizations, with examples from several large tech companies.",
  "publication": "newsletter.pragmaticengineer.com",
  "url": "https://newsletter.pragmaticengineer.com/p/how-the-best-teams-ship-faster",
  "slug": "how-the-best-teams-ship-faster",
  "postDate": "2026-09-15T15:41:25.109Z",
  "type": "newsletter",
  "author": "Gergely Orosz",
  "reactions": 161,
  "commentCount": 6,
  "coverImage": "https://substackcdn.com/image/fetch/$s_!Eiw9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1e73721e-e4d8-473f-a5a1-b6211c14f6ca_2048x1762.png",
  "audience": "only_paid",
  "description": "A deep dive into the processes, tooling, and culture behind high-output engineering organizations, with examples from several large tech companies.",
  "podcastUrl": ""
}

Common use cases

• Archive newsletters for research
• Monitor publication output
• Feed content datasets and RAG pipelines
• Track writers across publications

Compared to alternatives

• vs other Substack scrapers on the Store — pay-per-result billing means empty or failed searches cost nothing; you are not charged for a run that returns zero rows.
• vs building your own — no proxy management, markup-change maintenance, or infrastructure to run; paste inputs and consume the dataset.

Limitations

• Free/public posts only — paywalled content returns metadata (title, subtitle, author, date) without the post body.
• Very large archives paginate — bound runs with maxPosts.
• Full-text mode roughly doubles fetch time per post.
• Publications that have disabled their public archive page return no posts.

FAQ

Do I need an account or cookies?

No — the Actor reads public data only. There is no login to configure and no account to risk.

What happens when a search returns nothing?

The run completes with zero (or an explicit error) rows. Per-result billing means you are not charged for empty output.

Can I run it on a schedule?

Yes — attach an Apify schedule or use the saved tasks to re-run the same inputs daily or weekly.

Related guides and Actors

• Substack Archive Snapshot
• Substack Competitor Monitor
• Substack Full Content Dataset

Last verified: 2026-09

More scrapers at thirdwatch.dev.