Substack Scraper – Posts, Comments, Leaderboards, Search

Substack Scraper — posts, comments, leaderboards, search & profiles

The all-in-one Substack scraper. One Actor, six tools — no login, no cookies:

1. Posts of any newsletter — the full archive (or only posts after a date / matching a word), with the full text of free posts.
2. Comments — every comment of a post with all replies (thread depth and parent ID kept).
3. Leaderboards — top newsletters in 33 categories (Trending or Bestsellers) with subscriber counts, paid tier and subscription prices.
4. Search newsletters by keyword — with subscribers, prices, author, X/Twitter.
5. Search posts by keyword across all of Substack.
6. Author profiles — followers, subscribers, bestseller badge, newsletters they write and read.

Use cases

• Influencer & sponsor research: find the biggest newsletters of a niche with their audience size and price.
• Content research: what topics get the most likes, comments and restacks.
• Competitive monitoring: schedule a daily run to get every new post of competitor newsletters.
• Datasets for AI / NLP: full text of public posts and comment threads.

What you get

Post: title, subtitle, URL, newsletter, authors, date, free/paid, word count, likes, comments, restacks, tags, section, cover image, podcast URL & duration, preview, full text + HTML (free posts).

Comment: text, author, handle, profile URL, date, likes, number of replies, depth, parent comment ID.

Newsletter (leaderboard & search): rank, name, URL, author, description, subscribers (number + label), paid subscribers tier, bestseller tier, monthly & yearly price, currency, language, podcast, first post date, X/Twitter, public support email, logo.

Profile: name, handle, bio, followers, subscribers, bestseller tier, main newsletter, newsletters written, links, newsletters subscribed to.

Example (leaderboard)

{
  "type": "newsletter",
  "rank": 3,
  "category": "technology",
  "ranking": "bestseller",
  "name": "The Pragmatic Engineer",
  "url": "https://newsletter.pragmaticengineer.com",
  "author": "Gergely Orosz",
  "subscribers": 1100000,
  "subscribersText": "Over 1,100,000 subscribers",
  "paidSubscribersText": "Tens of thousands of paid subscribers",
  "monthlyPrice": 15,
  "yearlyPrice": 150,
  "currency": "USD",
  "twitter": "https://x.com/GergelyOrosz"
}

How to use

1. Choose What to scrape.
2. Fill the matching field: newsletter URLs or names (lennysnewsletter.com, astralcodexten, substack.com/@handle), post URLs, categories, keywords or profile handles.
3. Set Max results (per newsletter / category / keyword) and click Start. Export to JSON, CSV, Excel, or schedule it.

Pricing

$1.50 per 1,000 results (posts with full text, comments, newsletters or profiles), cheaper on paid Apify plans. No extra fee for full text.

Use with AI agents (MCP)

You can call this Actor from Claude, ChatGPT, Cursor or any other MCP client through the Apify MCP server (https://mcp.apify.com), using the Actor ID rt-solutions/substack-scraper. For example: "List the top 20 technology newsletters on Substack's bestseller leaderboard with their subscriber counts and yearly price." It also works through the Apify API, Make, n8n and Zapier.

FAQ

How do I scrape Substack without an API? Choose what to scrape, paste newsletter URLs, post URLs, categories, keywords or profile handles, and click Start. No login or cookies are needed; export to JSON, CSV or Excel.

How much does it cost to scrape 10,000 Substack posts? $15 at $1.50 per 1,000 results, full text included — less on paid Apify plans. The same price applies to comments, newsletters and profiles.

Can I monitor new posts of a newsletter? Yes — schedule a daily run and use the "only posts after a date" option to collect just the new ones.

Can I get paid posts? Only what Substack shows publicly: title, stats and preview. The full text of paid posts needs a paid subscription and is not collected.

Are subscriber counts exact? They are the public figures Substack displays ("Over 120,000 subscribers"). Some newsletters hide them; the field is then empty.

Does it work with custom domains? Yes — e.g. www.lennysnewsletter.com, www.astralcodexten.com.

Is it legal? The Actor only collects data shown publicly to any visitor without login. Use it in line with applicable laws (e.g. GDPR for personal data) and Substack's terms.

Related scrapers

Other ready-to-use Actors by RT Solutions:

• Google News Scraper
• Weibo Scraper
• Bilibili Scraper