Substack Scraper — Discover Newsletters, Posts & Author Data

The only Substack actor that lets you find publications you don't already know exist.

Every other Substack scraper on the Apify Store requires you to bring a publication URL. This one lets you search all of Substack by keyword ("AI newsletters with 10K+ subscribers", "crypto blogs launched in 2025") and then pipe the results straight into a full-metadata scrape.

What you can do with it

• Lead generation — build a list of every Substack in your niche, with author names and subscriber counts
• Competitive intelligence — track what publications your competitors recommend, what their post cadence is, which posts get the most engagement
• Sponsorship research — filter for publications above a subscriber threshold, with a paid tier, in a specific category
• AI / RAG datasets — bulk export post HTML with clean metadata, ready for LlamaIndex / LangChain

Three modes, one actor

1. discovery — search all of Substack

Provide any list of keywords. The actor searches Substack's own search index and returns matching publications with:

• Name, subdomain, custom domain
• Author name, handle, bio
• Subscriber count (when public) + explicit subscriber_count_hidden flag
• Has-paid-tier, has-podcast, language, first post date

Optionally filter results by minSubscribers and language.

Substack's search index does not expose category or post count, so those fields are null in this mode — use category mode if you need the category.

Cost: $0.003 per publication returned.

2. publication — deep-scrape by URL

Paste one or more publication URLs (works with *.substack.com and custom domains). Returns:

• Full publication metadata (as above)
• All posts with title, subtitle, publish date, wordcount, reactions per emoji, comment count, tags
• Optional: full post HTML body (paid post previews included where available)

Cost: $0.001/post metadata-only, $0.003/post with body.

3. category — browse Substack's leaderboards

Pick a category (technology, business, crypto, …) and return the top publications on Substack's own leaderboard, in ranked order.

Cost: $0.003 per publication.

Pricing

Pay per result. No monthly rental.

Event	Cost
Actor start (per run)	$0.005
Publication returned	$0.003
Post metadata only	$0.001
Post with full body	$0.003

Example: discovering 500 AI newsletters ≈ $1.51. Scraping the archive (50 posts each, metadata only) of 20 of them ≈ $1.01.

Example inputs

Find every AI publication with 5K+ subscribers:

{
  "mode": "discovery",
  "searchQueries": ["artificial intelligence", "machine learning", "LLM"],
  "maxItemsPerQuery": 500,
  "minSubscribers": 5000
}

Scrape a specific publication's last 200 posts with full bodies:

{
  "mode": "publication",
  "publicationUrls": ["https://stratechery.com"],
  "maxPostsPerPublication": 200,
  "includeFullBody": true
}

Get the top 100 tech Substacks by leaderboard rank:

{
  "mode": "category",
  "category": "technology",
  "maxItemsPerQuery": 100
}

Output schema

Every row is either a publication or a post (indicated by record_type). Both live in the same dataset so a single run can produce a unified CSV.

Publications include an explicit subscriber_count_hidden: bool so you can distinguish "0 subscribers" from "count hidden by publication" — no other actor does this and it matters for filtering.

Known limitations

Honest about what any Substack scraper can and can't do:

• Paywalled post bodies: Substack returns only the preview text to unauthenticated requests. Only previews are scraped; there is no way to get full paid content without a subscriber cookie for each publication.
• Subscriber counts are free-tier only: Substack publicly exposes a publication's free subscriber count (never paid counts). When the exact number is hidden, the actor still returns an order-of-magnitude band in subscriber_count_display (e.g. "93K+") and sets subscriber_count_hidden: true. The minSubscribers filter uses whichever is available.
• Notes (Substack's Twitter-like feed): Not yet supported — planned for v1.1.
• Substack changes their API: This actor uses undocumented endpoints. If they change, we ship a fix within 24 hours. Success rate is monitored publicly on the actor page.

Roadmap

• v1.1 — Notes feed scraping, comment tree extraction
• v1.2 — Cross-publication recommendation graph
• v1.3 — Growth tracking (subscriber deltas across scheduled runs)
• v1.4 — Author search (find someone's Substacks by email or name)

Support

Open an issue on this actor's page. Median first response < 24h.