Substack Scraper 📰

Discover Substack newsletters and creators — and scrape their posts.

Find the top newsletters in any category (with author, subscriber count and contact), or pull any publication's posts with full content, reactions, comments and podcast data. A complete toolkit for creator lead generation, content research and AI pipelines — no code required.

---

⚡ At a glance

🔎 Discover	Top newsletters by category — Technology, Finance, Crypto…
👤 Creator leads	Author, subscriber count, tagline, contact, monetization
📄 Full posts	Title, full content, reactions, comments, word count
🎧 Podcasts	Episode audio URL and duration
💵 Free vs paid	Audience filter and paywall flag
📦 Bulk	Many categories or publications at once
⚡ Fast	Pure-HTTP — no browser
💾 Export	Excel, CSV, JSON or API

---

📌 Why this Actor?

Most Substack scrapers only pull posts. This one does that and turns Substack's category leaderboards into a creator lead list:

• 🔎 Newsletter discovery — browse any category and get the top publications ranked, each with the author, subscriber count, tagline, and whether they run paid subscriptions or a podcast.
• 👤 Creator intelligence — perfect for sponsorships, PR outreach, influencer research and competitive analysis.
• 📄 Full post scraping — title, subtitle, full plain-text content, reactions, comments, word count and cover image.
• 🎧 Podcast support — audio URL and duration for podcast posts.
• 💵 Free / paid aware — filter by audience and see which posts are paywalled.

---

✨ What you get (data fields)

👤 Publication (creator lead)

Field	Description
name	Newsletter name
author	Creator / author name
subscribers	Subscriber count (e.g. 773,000)
tagline	Publication tagline
contactEmail	Contact email, when published
category	Discovery category
categoryRank	Rank within the category
isMonetizing	Has paid subscriptions
hasPodcast	Runs a podcast
authorBestsellerTier	Substack bestseller tier
url / subdomain	Publication links

📄 Post

Field	Description
title / subtitle	Post title & subtitle
content	Full post text (plain)
contentHtml	Raw HTML (optional)
author	Post byline
postDate	Publish date
audience	free / paid
reactionCount	Likes / reactions
commentCount	Number of comments
wordcount	Word count
podcastUrl / podcastDurationSeconds	Podcast media
publication	Source newsletter
url	Post URL

---

🚀 Quick start

Find newsletters in a category:

1. Under Discover newsletters by category, pick e.g. Finance.
2. Click Start → get a ranked list of top finance newsletters with authors and subscriber counts.

Scrape a publication's posts:

1. Under Publication URLs, add e.g. astralcodexten or https://www.honest-broker.com.
2. Click Start → get their posts with full content.

💡 Tip: Leave everything blank to get a sample of top Technology newsletters.

---

🎯 Use cases

Use case	What it delivers
🧲 Creator lead generation	Ranked newsletters by category with author, subscribers and contact — for sponsorships and outreach.
📊 Market & competitor research	See who leads a niche, how big they are, and what they publish.
🎙️ PR & influencer outreach	Build targeted lists of writers and podcasters to pitch.
🤖 AI / RAG pipelines	Feed full post text into LLMs, embeddings or a vector DB.
📰 Content monitoring	Track a publication's posts, reactions and comments over time.
💵 Monetization analysis	See which creators run paid subscriptions and podcasts.

---

⚙️ Input reference

Input	Type	Default	Description
Discover newsletters by category	list	—	Categories to find top newsletters in.
Publication URLs	list	—	Subdomains, URLs or custom domains to scrape posts from.
Post URLs	list	—	Individual post URLs.
Scrape posts from publications	boolean	true	Off = publication profile only (lead).
Include full post content	boolean	true	Fetch each post's full text.
Post audience	select	All	All / Free only / Paid only.
Max posts per publication	number	20	0 = all.
Max publications (discovery)	number	50	Cap for category discovery.
Only posts since (date)	string	—	e.g. 2026-01-01.
Proxy / concurrency	—	Datacenter / 5	Sensible defaults.

📥 Example — discover finance creators

{
  "discoverCategories": ["153"],
  "maxPublications": 50,
  "scrapePosts": false
}

📥 Example — scrape a publication's posts

{
  "publications": ["astralcodexten", "https://www.honest-broker.com"],
  "maxPostsPerPublication": 30,
  "includePostContent": true,
  "audience": "free"
}

---

📤 Example output

Creator lead:

{
  "recordType": "publication",
  "name": "The Honest Broker",
  "author": "Ted Gioia",
  "subscribers": "296,000",
  "tagline": "Essays on music, books, media & culture",
  "isMonetizing": true,
  "hasPodcast": false,
  "category": "Culture",
  "categoryRank": 3,
  "url": "https://www.honest-broker.com"
}

Post:

{
  "recordType": "post",
  "publication": "Astral Codex Ten",
  "title": "The Hugging Face Incident",
  "postDate": "2026-07-24T...",
  "audience": "free",
  "reactionCount": 416,
  "commentCount": 382,
  "wordcount": 1325,
  "content": "…full post text…",
  "url": "https://www.astralcodexten.com/p/the-hugging-face-incident"
}

---

💡 Tips & best practices

• ✅ Discover first, scrape later — find the newsletters in a niche, then feed the best ones back in to scrape their posts.
• ✅ Turn off content for a fast metadata-only list; turn it on for full text.
• ✅ Filter by audience to focus on free (public) posts.
• ✅ Sort by subscribers to find the biggest creators in a category.
• ✅ Use since date for incremental content runs.

---

❓ FAQ

Can I find newsletters by topic? Yes — pick one or more categories and get the top publications ranked, with author and subscriber count.

Can I get subscriber counts? Yes, when the publication exposes them (shown as e.g. 296,000).

Can I get full post content? Yes for public (free) posts. Paid posts return the public preview only.

Do I need to code? No. Pick categories or paste publication URLs, click Start, export.

In what format is the data? Structured JSON by default, exportable to CSV, Excel, HTML or via API.

---

📊 Output & integrations

Results are stored in a standard Apify dataset — preview as a table, export to JSON, CSV, Excel, HTML, pull via the API, or push to Google Sheets, Make, Zapier and more.

---

⚖️ Legal & responsible use

This Actor reads publicly available Substack information for legitimate research, discovery and monitoring.

• Post content remains the property of its authors — respect copyright and terms of service.
• Any personal data (e.g. contact emails) requires a lawful basis for processing and outreach — ensure you have one and honour opt-out requests.
• You are responsible for how you use the collected data.

---

🛟 Support

Found a bug, need an extra field, or want a tweak? Open an issue on the Actor's Issues tab.

Happy discovering! 📰