Substack Scraper — Newsletters, Subscribers, Posts & Full Article Text

Scrape any Substack newsletter, post or author — no login, no browser, no API key. Find newsletters by keyword or straight from Substack category leaderboards, and get clean JSON with subscriber counts, paid plan prices, leaderboard rank, author X/LinkedIn/YouTube/website links, plus every post with likes, comments, restacks, word count, paywall status and (optionally) the full article text.

Export to JSON, CSV or Excel, or send it to Google Sheets, Make, Zapier, n8n, a vector database or your AI agent.

Who uses it

• Sponsorship & influencer marketing: build lists of newsletters in your niche with audience size, price and author contacts.
• B2B lead generation: writers with 10k+ subscribers, their websites and social profiles.
• Content & competitor research: which posts get the most likes and comments, how often competitors publish, what they put behind the paywall.
• AI / RAG datasets: full text of free posts as clean HTML and plain text.
• Investors & analysts: track leaderboard rankings and subscriber growth over time with scheduled runs.

What you get

Record	Fields
Newsletter (type: "newsletter")	name, URL, custom domain, description, language, subscriber count, paid-subscriber tier, monthly / annual price, founding plan, leaderboard category & rank, bestseller tier, podcast & RSS feed, author name, handle, bio, follower count, social links (X, LinkedIn, YouTube, Instagram, TikTok, website…), Stripe country, first post date, logo
Post (type: "post")	title, subtitle, URL, publish date, type (newsletter / podcast / thread), audience (everyone, only_paid, founding), likes, comments, restacks, word count, tags, section, authors, cover image, preview text, podcast URL & duration — and with Include full article text: contentHtml + contentText

Four ways to choose what to scrape

1. Newsletter URLs — https://name.substack.com, custom domains (https://www.lennysnewsletter.com) or just the subdomain (astralcodexten).
2. Post URLs — https://name.substack.com/p/post-slug (also open.substack.com/pub/... links).
3. Author profiles — https://substack.com/@handle.
4. Discovery
   • Keyword search (searchQueries): e.g. marketing, AI startups, real estate investing — returns the newsletters of matching writers.
   • Category leaderboards (categories): technology, business, finance, culture, science, crypto, … — Bestsellers (top paid, 25 per category) or Rising / all (up to ~525 per category).

Mix them freely in one run. Newsletters found through several sources are returned once.

Pricing

$3.00 per 1,000 results ($0.003 per result). One result = one newsletter or one post. Full article text, author enrichment and retries are included — no start fee, no extra per-request charges.

Example	Results	Cost
Top 25 paid newsletters in Technology, details only (scrapePosts: false)	25	$0.08
100 marketing newsletters + their last 10 posts	1,100	$3.30
Full archive of one newsletter (≈1,000 posts) with article text	≈1,000	≈$3.00

Use Max results (cost cap) to put a hard ceiling on any run. The actor also stops by itself when your spending limit is reached.

Input example

{
  "startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"],
  "searchQueries": ["marketing"],
  "categories": ["technology"],
  "leaderboardType": "paid",
  "maxNewslettersPerSource": 25,
  "includeNewsletterDetails": true,
  "enrichAuthors": true,
  "scrapePosts": true,
  "maxPostsPerNewsletter": 10,
  "postedAfter": "30 days",
  "includeContent": false
}

Option	Default	Notes
maxNewslettersPerSource	25	Per keyword / category.
includeNewsletterDetails	true	Turn off if you only want posts.
enrichAuthors	true	Adds author social links and follower count.
scrapePosts	true	Turn off for a pure newsletter/lead list.
maxPostsPerNewsletter	20	Newest first. -1 = whole archive.
postedAfter	—	2026-01-01 or 30 days, 2 weeks, 1 year.
includeContent	false	Full text of free posts; preview for paywalled ones.
maxResults	0	0 = unlimited.

Output example

Newsletter

{
  "type": "newsletter",
  "id": 10845,
  "name": "Lenny's Newsletter",
  "url": "https://www.lennysnewsletter.com",
  "subdomain": "lenny",
  "customDomain": "www.lennysnewsletter.com",
  "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
  "language": "en",
  "subscriberCount": 1200000,
  "subscriberCountText": "Millions of subscribers",
  "paidSubscribersText": "Tens of thousands of paid subscribers",
  "isPaid": true,
  "monthlyPrice": 20,
  "annualPrice": 200,
  "currency": "USD",
  "leaderboardCategory": "Business",
  "leaderboardRank": 1,
  "leaderboardType": "paid",
  "hasPodcast": true,
  "rssFeedUrl": "https://www.lennysnewsletter.com/feed",
  "authorName": "Lenny Rachitsky",
  "authorHandle": "lenny",
  "authorProfileUrl": "https://substack.com/@lenny",
  "authorFollowerCount": 1384322,
  "authorLinks": {
    "twitter": "https://x.com/lennysan",
    "youtube": "https://www.youtube.com/c/LennysPodcast",
    "website": "https://www.lennyrachitsky.com/"
  },
  "stripeCountry": "US",
  "firstPostDate": "2019-06-14T15:00:48.833Z",
  "source": "url"
}

Post (with includeContent: true)

{
  "type": "post",
  "id": 215243350,
  "title": "The Specter Of Neuralese",
  "url": "https://www.astralcodexten.com/p/the-specter-of-neuralese",
  "postType": "newsletter",
  "publishedAt": "2026-09-24T22:12:39.073Z",
  "audience": "everyone",
  "isPaywalled": false,
  "wordCount": 3037,
  "reactionCount": 359,
  "commentCount": 165,
  "restackCount": 28,
  "tags": [],
  "authors": [{ "id": 12009663, "name": "Scott Alexander", "handle": "astralcodexten" }],
  "newsletterName": "Astral Codex Ten",
  "newsletterUrl": "https://www.astralcodexten.com",
  "contentHtml": "<h4>I. At Long Last, We Have Created Neuralese Recurrence…</h4><p>…",
  "contentText": "I. At Long Last, We Have Created Neuralese Recurrence…\nIn the public debates after the release of AI 2027…",
  "isContentComplete": true
}

The dataset has two ready-made views in the Apify Console: Newsletters and Posts. Filter by type when exporting via API, e.g. .../items?format=csv&fields=name,url,subscriberCount,authorLinks.

Use with AI agents (MCP), LangChain, RAG & the API

This actor is a ready-made tool for AI agents: an LLM can call it, wait for the run and read clean JSON back.

MCP server (Claude Desktop, Claude Code, Cursor, VS Code, any MCP client) — add Apify's hosted MCP server with this actor enabled as a tool:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=murch1k/substack-scraper",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}

LangChain / LangGraph (pip install langchain-apify, set APIFY_API_TOKEN):

from langchain_apify import ApifyActorsTool, ApifyWrapper
from langchain_core.documents import Document

# 1) give the actor to an agent as a tool
tool = ApifyActorsTool("murch1k/substack-scraper")

# 2) or load results as Documents for a RAG index
loader = ApifyWrapper().call_actor(
    actor_id="murch1k/substack-scraper",
    run_input={"startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"], "searchQueries": ["marketing"], "categories": ["technology"], "leaderboardType": "paid", "maxNewslettersPerSource": 25, "includeNewsletterDetails": True, "enrichAuthors": True, "scrapePosts": True, "maxPostsPerNewsletter": 10, "postedAfter": "30 days", "includeContent": False},
    dataset_mapping_function=lambda item: Document(page_content=item["contentText"], metadata={"url": item.get("url")}),
)
docs = loader.load()

REST API — one synchronous call, dataset items in the response:

curl -X POST "https://api.apify.com/v2/acts/murch1k~substack-scraper/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"], "searchQueries": ["marketing"], "categories": ["technology"], "leaderboardType": "paid", "maxNewslettersPerSource": 25, "includeNewsletterDetails": true, "enrichAuthors": true, "scrapePosts": true, "maxPostsPerNewsletter": 10, "postedAfter": "30 days", "includeContent": false}'

Also works with the Apify Python/JS clients, Make, Zapier, n8n, Google Sheets, webhooks and schedules. Export as JSON, CSV, Excel, XML or RSS.

RAG tip: With includeContent: true every post row carries the full article text (contentText) — embed it per post and keep newsletterName, title, publishedAt and url as metadata for cited answers.

Speed

The actor reads Substack's own JSON endpoints over plain HTTP — no headless browser — so it is fast and cheap:

• ~700 records (120 newsletters + 590 posts) in under a minute on 256–512 MB memory.
• A full 180-post archive in ~4 seconds.
• When Substack rate-limits, the request is retried immediately through a new proxy IP — no waiting, no lost records.

FAQ

Do I need a Substack account or cookies? No. Only publicly available data is collected.

Can it get paywalled articles? No. For paid posts you get all metadata plus the public preview text; isContentComplete tells you whether the text is the full article.

Does it work with custom domains? Yes — any newsletter hosted on Substack, whatever its domain.

Are subscriber counts exact? They are the numbers Substack shows publicly (e.g. "1,200,000" or "95K+"). Substack hides counts for some newsletters; those get null plus the textual tier in subscriberCountText when available.

Do I need a proxy? The actor uses Apify datacenter proxy by default (included in every Apify plan). Substack rate-limits cloud IPs, and the proxy lets the actor switch to a fresh IP immediately instead of waiting, which makes runs several times faster. You can turn it off for small runs.

Email addresses? Substack does not publish writer emails; the actor returns supportEmail when a newsletter sets one, plus author website and social links, which usually lead to contact details.

Legal

This actor only extracts publicly available data. Make sure your use complies with Substack's Terms of Use and applicable data-protection law (e.g. GDPR) — especially when processing personal data such as author names and social profiles.

Support

Found a bug or need an extra field? Open an issue on the Issues tab — feature requests are usually shipped within days.