Substack Scraper & API – Newsletter Leads & Full Posts

What does Substack Scraper & API (Newsletter Leads & Full Posts) do?

Substack Scraper & API is a dual-mode B2B creator prospecting and content extraction engine for Substack. Use creator_leads mode to discover niche newsletters by keyword (subscriber counts, bestseller badge tiers, custom domains, author bios, X/Twitter handles, external links, and Substack addresses), or use posts mode to extract complete plain-text newsletter archives, word counts, reactions, comments, and podcast audio URLs from any Substack subdomain or custom domain.

Unlike multi-fee scrapers that charge a run-start fee plus double rates ($2.30/1K) for post content, this Substack API alternative charges one flat rate with $0.00 start fee and full post text included at no extra cost.

• Flat Pay-Per-Event Pricing: $0.90 per 1,000 leads or full posts ($0.0009 per item, $0.00 run-start fee). Platform compute is 100% free.
• 5,555 Free Leads or Posts / Month: Apify's $5.00/month free plan credit covers 5,555 Substack creator leads or full-text newsletter posts every month at $0.
• Supports Custom Domains Natively: Works seamlessly on both *.substack.com URLs and custom-domain publications (www.fintechbrainfood.com, newsletter.pragmaticengineer.com, etc.).

---

Why scrape Substack newsletters, creator leads, and post archives?

Substack hosts over 35 million active subscriptions across high-intent B2B, AI, fintech, SaaS, VC, and consumer niches — yet offers no official public API for sponsorship discovery or RAG archive ingestion.

High-ROI B2B & AI workflows

1. Newsletter Sponsorship & Creator Partnership Prospecting: Search keywords like "b2b saas", "fintech", "devtools", or "cybersecurity" to build instant media-buying lists pre-qualified by subscriberCount (42K+ subscribers), bestsellerTier (100 / 1,000 / 10,000 paid subscribers), and paidSubscriptionsEnabled.
2. Cold-Outreach & Influencer Lead Workflows (Clay, Instantly, Smartlead, n8n): Export authorName, publicationUrl, customDomain, twitterHandle, externalLinks, and substackAddress directly into your CRM or enrichment waterfall.
3. AI Agents, RAG & Deep Industry Research (Claude, Cursor, ChatGPT via MCP): Ingest full long-form essays from top Substack analysts with clean HTML-stripped contentText, wordCount, and reactionsCount for high-signal LLM grounding.
4. Viral Content & Topic Benchmarking: Sort any publication's archive by reactionsCount and commentsCount to pinpoint which headlines and formats drive maximum reader engagement.

---

What data can Substack Scraper extract?

Mode 1: creator_leads (B2B Newsletter & Author Discovery)

Field Name	Type	Description
publicationName	string	Official Substack newsletter title (e.g., Fintech Brain Food).
subdomain	string	Substack handle/subdomain (sytaylor).
publicationUrl	string	Canonical URL (uses https://<customDomain> when configured).
customDomain	string | null	Custom web domain (www.fintechbrainfood.com) — ideal for Hunter/Apollo domain enrichment.
authorName	string | null	Creator's full display name.
bio	string | null	Creator bio / newsletter value proposition.
subscriberCount	integer | null	Numeric subscriber count when publicly exposed (42000).
subscriberCountLabel	string | null	Formatted subscriber badge ("42K+ subscribers").
followerCount	integer | null	Public Substack follower count.
bestsellerTier	integer | null	Official Substack Bestseller badge tier (100, 1000, 10000 paid subscribers).
paidSubscriptionsEnabled	boolean	Whether the publication monetizes via paid Stripe subscriptions.
substackAddress	string	Direct Substack publication address (<subdomain>@substack.com).
twitterHandle	string | null	Linked X / Twitter username when connected.
externalLinks	array<string>	Personal websites, LinkedIn, media kits, or sponsorship booking links from profile.

Mode 2: posts (Full Newsletter Archive & Engagement)

Field Name	Type	Description
postTitle	string	Full newsletter issue title.
subtitle	string | null	Post deck / subtitle summary.
authorName	string | null	Published byline author name.
publishedAt	string	ISO-8601 publication timestamp.
audience	string	Access tier (everyone vs only_paid).
wordCount	integer | null	Exact word count of the issue.
reactionsCount	integer	Total hearts / likes on the post.
commentsCount	integer	Total reader comments on the post.
postUrl	string	Canonical URL to the newsletter issue.
audioUrl	string | null	Direct podcast/audio enclosure URL when the issue includes audio.
contentText	string	Clean plain-text article body ready for LLMs, RAG, and NLP analysis.

---

How to scrape Substack step-by-step

1. Click Try for free to open Substack Scraper & API in Apify Console.
2. Select Extraction Mode (mode):
   • creator_leads — find newsletters & authors by topic keywords.
   • posts — extract full articles & engagement from specific Substack URLs.
   • both — run discovery and archive extraction together.
3. Enter your Niche Keywords (searchKeywords) or Substack URLs (publications).
4. Set Max Leads / Max Posts (maxItemsPerTarget) and click Start.
5. Export your dataset in CSV, Excel, JSON, or HTML.

---

How much does it cost to scrape Substack?

Simple flat Pay-Per-Event ($0.90 / 1,000 items) pricing with $0.00 start fee and compute included:

Volume	Our Price ($0.90/1K)	Typical Rival ($2.30/1K + start fee)	Covered by Apify $5/mo Free Credit?
100 creator leads or full posts	$0.09	$0.24+	Yes ($0)
1,000 creator leads or full posts	$0.90	$2.31+	Yes ($0)
5,555 leads or posts / month	$5.00	$12.78+	100% Free every month

---

Use with AI Assistants & MCP (Claude, Cursor, VS Code, ChatGPT)

Connect Substack discovery and essay reading directly to Claude Desktop, Cursor, or VS Code via Apify MCP:

{
  "mcpServers": {
    "substack-intelligence": {
      "url": "https://mcp.apify.com/?actors=ramgoxxx/substack-newsletter-leads-posts-scraper",
      "headers": {
        "Authorization": "Bearer YOUR_APIFY_API_TOKEN"
      }
    }
  }
}

Ask your AI agent:

• "Find 30 Substack newsletters in ai agents and devtools with over 5,000 subscribers for our Q4 sponsorship campaign."
• "Pull the latest 5 essays from pragmaticengineer.substack.com and summarize their engineering hiring trends."

---

Input examples

Example 1: B2B Newsletter Sponsorship Lead Discovery (creator_leads)

{
  "mode": "creator_leads",
  "searchKeywords": ["fintech", "b2b saas", "devtools"],
  "maxItemsPerTarget": 50
}

Example 2: Full-Text Newsletter Archive for RAG (posts)

{
  "mode": "posts",
  "publications": [
    "pragmaticengineer.substack.com",
    "www.fintechbrainfood.com"
  ],
  "maxItemsPerTarget": 25,
  "includeFullContent": true
}

---

Output example (creator_leads mode)

[
  {
    "recordType": "creator_lead",
    "publicationName": "Fintech Brain Food",
    "subdomain": "sytaylor",
    "publicationUrl": "https://www.fintechbrainfood.com",
    "customDomain": "www.fintechbrainfood.com",
    "authorName": "Simon Taylor",
    "authorHandle": "sytaylor",
    "bio": "Fintech Brainfood is my personal space to make sense of fintech, payments and banking...",
    "subscriberCount": 42000,
    "subscriberCountLabel": "42K+ subscribers",
    "followerCount": 58400,
    "bestsellerTier": 1000,
    "paidSubscriptionsEnabled": false,
    "substackAddress": "sytaylor@substack.com",
    "twitterHandle": "sytaylor",
    "externalLinks": ["https://www.fintechbrainfood.com"],
    "matchedKeyword": "fintech"
  }
]

---

Integrate with Python, Clay, n8n & Make

Python (apify-client)

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")
run = client.actor("ramgoxxx/substack-newsletter-leads-posts-scraper").call(
    run_input={
        "mode": "creator_leads",
        "searchKeywords": ["cybersecurity", "cloud native"],
        "maxItemsPerTarget": 40,
    }
)

for lead in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(lead["publicationName"], "|", lead["subscriberCountLabel"], "|", lead["publicationUrl"])

cURL / n8n Webhook

curl -X POST "https://api.apify.com/v2/acts/ramgoxxx~substack-newsletter-leads-posts-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"creator_leads","searchKeywords":["ai startup"],"maxItemsPerTarget":25}'

---

Frequently Asked Questions (FAQ)

Can this extract paywalled subscriber-only content?

No. Respecting creator paywalls, the Actor only extracts publicly accessible free issues (audience: "everyone") and public preview excerpts on paid-only posts.

Legal, GDPR & Ethical Notice

Our Actors are ethical and do not bypass paywalls or extract private subscriber data. They only aggregate public publication profiles, creator bios, public links, and openly published newsletter issues. Creator names and social handles may constitute personal data under GDPR in the EU and similar regulations globally; ensure any B2B sponsorship or partnership outreach complies with applicable privacy and anti-spam laws. Open a ticket in the Issues tab for support.