Substack Scraper — Newsletters, Subscribers, Posts & Full Article Text

Scrape any Substack newsletter, post or author — no login, no browser, no API key. Find newsletters by keyword or straight from Substack category leaderboards, and get clean JSON with subscriber counts, paid plan prices, leaderboard rank, author X/LinkedIn/YouTube/website links, plus every post with likes, comments, restacks, word count, paywall status and (optionally) the full article text.

Export to JSON, CSV or Excel, or send it to Google Sheets, Make, Zapier, n8n, a vector database or your AI agent.

How to run it in 3 steps

1. Add your input. In Newsletter, post or profile URLs paste Substack newsletter or post links — e.g. https://www.lennysnewsletter.com. The form is already filled with a working example, so you can skip this step for a first try.
2. Press Start. The prefilled run returns 21 results in about 3 s and costs about $0.063. Set Max results (cost cap) to cap the cost of bigger runs; you only pay for results actually delivered.
3. Get the data. Open the Output tab and export JSON, CSV or Excel — or run it from the API, Make, Zapier, n8n or an AI agent (see below).

Who uses it

• Sponsorship & influencer marketing: build lists of newsletters in your niche with audience size, price and author contacts.
• B2B lead generation: writers with 10k+ subscribers, their websites and social profiles.
• Content & competitor research: which posts get the most likes and comments, how often competitors publish, what they put behind the paywall.
• AI / RAG datasets: full text of free posts as clean HTML and plain text.
• Investors & analysts: track leaderboard rankings and subscriber growth over time with scheduled runs.

What you get

Record	Fields
Newsletter (type: "newsletter")	name, URL, custom domain, description, language, subscriber count, paid-subscriber tier, monthly / annual price, founding plan, leaderboard category & rank, bestseller tier, podcast & RSS feed, author name, handle, bio, follower count, social links (X, LinkedIn, YouTube, Instagram, TikTok, website…), Stripe country, first post date, logo
Post (type: "post")	title, subtitle, URL, publish date, type (newsletter / podcast / thread), audience (everyone, only_paid, founding), likes, comments, restacks, word count, tags, section, authors, cover image, preview text, podcast URL & duration — and with Include full article text: contentHtml + contentText

Four ways to choose what to scrape

1. Newsletter URLs — https://name.substack.com, custom domains (https://www.lennysnewsletter.com) or just the subdomain (astralcodexten).
2. Post URLs — https://name.substack.com/p/post-slug (also open.substack.com/pub/... links).
3. Author profiles — https://substack.com/@handle.
4. Discovery
   • Keyword search (searchQueries): e.g. marketing, AI startups, real estate investing — returns the newsletters of matching writers.
   • Category leaderboards (categories): technology, business, finance, culture, science, crypto, … — Bestsellers (top paid, 25 per category) or Rising / all (up to ~525 per category).

Mix them freely in one run. Newsletters found through several sources are returned once.

Pricing

• The prefilled input is billed for 21 results → $0.063.

$3.00 per 1,000 results ($0.003 per result). One result = one newsletter or one post. Full article text, author enrichment and retries are included — no start fee, no extra per-request charges.

Example	Results	Cost
Top 25 paid newsletters in Technology, details only (scrapePosts: false)	25	$0.08
100 marketing newsletters + their last 10 posts	1,100	$3.30
Full archive of one newsletter (≈1,000 posts) with article text	≈1,000	≈$3.00

Use Max results (cost cap) to put a hard ceiling on any run. The actor also stops by itself when your spending limit is reached.

Input example

{
  "startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"],
  "searchQueries": ["marketing"],
  "categories": ["technology"],
  "leaderboardType": "paid",
  "maxNewslettersPerSource": 25,
  "includeNewsletterDetails": true,
  "enrichAuthors": true,
  "scrapePosts": true,
  "maxPostsPerNewsletter": 10,
  "postedAfter": "30 days",
  "includeContent": false
}

Option	Default	Notes
maxNewslettersPerSource	25	Per keyword / category.
includeNewsletterDetails	true	Turn off if you only want posts.
enrichAuthors	true	Adds author social links and follower count.
scrapePosts	true	Turn off for a pure newsletter/lead list.
maxPostsPerNewsletter	20	Newest first. -1 = whole archive.
postedAfter	—	2026-01-01 or 30 days, 2 weeks, 1 year.
includeContent	false	Full text of free posts; preview for paywalled ones.
maxResults	0	0 = unlimited.

Output example

Real output (default input)

First 2 of 21 records from a run with the prefilled input (3.4 s on a test machine; long values shortened here):

[
  {
    "type": "newsletter",
    "id": 10845,
    "name": "Lenny's Newsletter",
    "url": "https://www.lennysnewsletter.com",
    "subdomain": "lenny",
    "customDomain": "www.lennysnewsletter.com",
    "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
    "language": "en",
    "subscriberCount": 1200000,
    "subscriberCountText": "Millions of subscribers",
    "paidSubscribersText": "Tens of thousands of paid subscribers",
    "isPaid": true,
    "monthlyPrice": 20,
    "annualPrice": 200,
    "currency": "USD",
    "foundingPlanName": "Insider Tier",
    "leaderboardCategory": "Business",
    "leaderboardRank": 1,
    "leaderboardType": "paid",
    "bestsellerTier": 10000,
    "hasPodcast": true,
    "podcastFeedUrl": null,
    "rssFeedUrl": "https://www.lennysnewsletter.com/feed",
    "authorName": "Lenny Rachitsky",
    "authorHandle": "lenny",
    "authorBio": "Writing • Angel investing • Advising",
    "authorProfileUrl": "https://substack.com/@lenny",
    "authorFollowerCount": 1385080,
    "authorLinks": {
      "twitter": "https://x.com/lennysan",
      "tiktok": "https://tiktok.com/@lennyrachtsky",
      "youtube": "https://www.youtube.com/c/LennysPodcast",
      "website": "https://www.lennyrachitsky.com/"
    },
    "supportEmail": null,
    "emailFromName": "Lenny's Newsletter",
    "copyright": null,
    "stripeCountry": "US",
    "firstPostDate": "2019-06-14T15:00:48.833Z",
    "createdAt": "2019-06-01T15:35:37.885Z",
    "logoUrl": "https://substackcdn.com/image/fetch/$s_!8MSN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fp…",
    "coverImageUrl": "https://substack-post-media.s3.amazonaws.com/public/images/4a25905e-c484-4885-80fa-9c2e683e362b_592x592.png",
    "source": "url",
    "scrapedAt": "2026-10-15T12:27:47.178Z"
  },
  {
    "type": "post",
    "id": 219482066,
    "title": "How to tell if a startup has a moat ",
    "subtitle": "A framework for assessing whether the company you’re considering joining is built to last",
    "url": "https://www.lennysnewsletter.com/p/how-to-tell-if-a-startup-has-a-moat",
    "slug": "how-to-tell-if-a-startup-has-a-moat",
    "postType": "newsletter",
    "publishedAt": "2026-10-13T12:45:41.160Z",
    "audience": "only_paid",
    "isPaywalled": true,
    "wordCount": 4153,
    "reactionCount": 301,
    "commentCount": 2,
    "restackCount": 10,
    "description": "A framework for assessing whether the company you’re considering joining is built to last",
    "previewText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: Lenny’s Jobs | Lenny’s Podcast | …",
    "coverImageUrl": "https://substackcdn.com/image/fetch/$s_!7Uce!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fp…",
    "tags": [
      "Career"
    ],
    "sectionName": null,
    "authors": [
      {
        "id": 2607892,
        "name": "Dan Hockenmaier",
        "handle": "danhock"
      }
    ],
    "podcastUrl": null,
    "podcastDurationSec": null,
    "newsletterId": 10845,
    "newsletterName": "Lenny's Newsletter",
    "newsletterUrl": "https://www.lennysnewsletter.com",
    "scrapedAt": "2026-10-15T12:27:47.420Z"
  }
]

Newsletter

{
  "type": "newsletter",
  "id": 10845,
  "name": "Lenny's Newsletter",
  "url": "https://www.lennysnewsletter.com",
  "subdomain": "lenny",
  "customDomain": "www.lennysnewsletter.com",
  "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
  "language": "en",
  "subscriberCount": 1200000,
  "subscriberCountText": "Millions of subscribers",
  "paidSubscribersText": "Tens of thousands of paid subscribers",
  "isPaid": true,
  "monthlyPrice": 20,
  "annualPrice": 200,
  "currency": "USD",
  "leaderboardCategory": "Business",
  "leaderboardRank": 1,
  "leaderboardType": "paid",
  "hasPodcast": true,
  "rssFeedUrl": "https://www.lennysnewsletter.com/feed",
  "authorName": "Lenny Rachitsky",
  "authorHandle": "lenny",
  "authorProfileUrl": "https://substack.com/@lenny",
  "authorFollowerCount": 1384322,
  "authorLinks": {
    "twitter": "https://x.com/lennysan",
    "youtube": "https://www.youtube.com/c/LennysPodcast",
    "website": "https://www.lennyrachitsky.com/"
  },
  "stripeCountry": "US",
  "firstPostDate": "2019-06-14T15:00:48.833Z",
  "source": "url"
}

Post (with includeContent: true)

{
  "type": "post",
  "id": 215243350,
  "title": "The Specter Of Neuralese",
  "url": "https://www.astralcodexten.com/p/the-specter-of-neuralese",
  "postType": "newsletter",
  "publishedAt": "2026-09-24T22:12:39.073Z",
  "audience": "everyone",
  "isPaywalled": false,
  "wordCount": 3037,
  "reactionCount": 359,
  "commentCount": 165,
  "restackCount": 28,
  "tags": [],
  "authors": [{ "id": 12009663, "name": "Scott Alexander", "handle": "astralcodexten" }],
  "newsletterName": "Astral Codex Ten",
  "newsletterUrl": "https://www.astralcodexten.com",
  "contentHtml": "<h4>I. At Long Last, We Have Created Neuralese Recurrence…</h4><p>…",
  "contentText": "I. At Long Last, We Have Created Neuralese Recurrence…\nIn the public debates after the release of AI 2027…",
  "isContentComplete": true
}

The dataset has two ready-made views in the Apify Console: Newsletters and Posts. Filter by type when exporting via API, e.g. .../items?format=csv&fields=name,url,subscriberCount,authorLinks.

Ask in plain English

Using Apify AI, Claude, ChatGPT or another MCP client? Just describe what you need, the agent fills the input for you:

You ask	Input the agent sends
"Get subscriber count and paid price of Lenny's Newsletter"	{"startUrls": ["https://www.lennysnewsletter.com"]}
"Find Substack newsletters about AI"	{"searchQueries": ["AI"]}
"Get the latest 10 posts with full text from Platformer"	{"startUrls": ["https://www.platformer.news"], "scrapePosts": true, "maxPostsPerNewsletter": 10}

Use with AI agents (MCP), LangChain, RAG & the API

This actor is a ready-made tool for AI agents: an LLM can call it, wait for the run and read clean JSON back.

MCP server (Claude Desktop, Claude Code, Cursor, VS Code, any MCP client) — add Apify's hosted MCP server with this actor enabled as a tool:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=torix33/substack-scraper",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}

LangChain / LangGraph (pip install langchain-apify, set APIFY_API_TOKEN):

from langchain_apify import ApifyActorsTool, ApifyWrapper
from langchain_core.documents import Document

# 1) give the actor to an agent as a tool
tool = ApifyActorsTool("torix33/substack-scraper")

# 2) or load results as Documents for a RAG index
loader = ApifyWrapper().call_actor(
    actor_id="torix33/substack-scraper",
    run_input={"startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"], "searchQueries": ["marketing"], "categories": ["technology"], "leaderboardType": "paid", "maxNewslettersPerSource": 25, "includeNewsletterDetails": True, "enrichAuthors": True, "scrapePosts": True, "maxPostsPerNewsletter": 10, "postedAfter": "30 days", "includeContent": False},
    dataset_mapping_function=lambda item: Document(page_content=item["contentText"], metadata={"url": item.get("url")}),
)
docs = loader.load()

REST API — one synchronous call, dataset items in the response:

curl -X POST "https://api.apify.com/v2/acts/torix33~substack-scraper/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"], "searchQueries": ["marketing"], "categories": ["technology"], "leaderboardType": "paid", "maxNewslettersPerSource": 25, "includeNewsletterDetails": true, "enrichAuthors": true, "scrapePosts": true, "maxPostsPerNewsletter": 10, "postedAfter": "30 days", "includeContent": false}'

Also works with the Apify Python/JS clients, Make, Zapier, n8n, Google Sheets, webhooks and schedules. Export as JSON, CSV, Excel, XML or RSS.

RAG tip: With includeContent: true every post row carries the full article text (contentText) — embed it per post and keep newsletterName, title, publishedAt and url as metadata for cited answers.

Speed

The actor reads Substack's own JSON endpoints over plain HTTP — no headless browser — so it is fast and cheap:

• ~700 records (120 newsletters + 590 posts) in under a minute on 256–512 MB memory.
• A full 180-post archive in ~4 seconds.
• When Substack rate-limits, the request is retried immediately through a new proxy IP — no waiting, no lost records.

Limitations

• Paywalled posts return only the public preview, not the full text.
• Subscriber counts are what Substack shows publicly; some newsletters hide them.
• Only public newsletters and posts; no login.

FAQ

Do I need a Substack account or cookies? No. Only publicly available data is collected.

Can it get paywalled articles? No. For paid posts you get all metadata plus the public preview text; isContentComplete tells you whether the text is the full article.

Does it work with custom domains? Yes — any newsletter hosted on Substack, whatever its domain.

Are subscriber counts exact? They are the numbers Substack shows publicly (e.g. "1,200,000" or "95K+"). Substack hides counts for some newsletters; those get null plus the textual tier in subscriberCountText when available.

Do I need a proxy? The actor uses Apify datacenter proxy by default (included in every Apify plan). Substack rate-limits cloud IPs, and the proxy lets the actor switch to a fresh IP immediately instead of waiting, which makes runs several times faster. You can turn it off for small runs.

Email addresses? Substack does not publish writer emails; the actor returns supportEmail when a newsletter sets one, plus author website and social links, which usually lead to contact details.

Legal

This actor only extracts publicly available data. Make sure your use complies with Substack's Terms of Use and applicable data-protection law (e.g. GDPR) — especially when processing personal data such as author names and social profiles.

Support

Found a bug or need an extra field? Open an issue on the Issues tab — feature requests are usually shipped within days.