Substack Scraper: Newsletter Posts, Search & Leaderboards

Get posts from any Substack newsletter as clean JSON/CSV/Excel: title, subtitle, date, likes, comments, restacks, word count, free/paid flag, authors, tags, cover image, podcast audio and (optionally) the full post text. Search all of Substack by keyword, and list the newsletters on Substack's category leaderboards (bestsellers and rising) with paid tier, subscriber count, prices and author. No login, no API key, no paid proxy.

🇪🇸 Español: extrae publicaciones de cualquier newsletter de Substack: título, fecha, me gusta, comentarios, restacks, palabras, si es de pago, autores, etiquetas y el texto completo (opcional). Busca en todo Substack por palabra clave y lista los newsletters de los rankings por categoría (bestsellers y en ascenso) con nivel de suscriptores de pago, suscriptores, precios y autor. Sin cuenta ni API key. 🇧🇷 Português: extrai posts de qualquer newsletter do Substack: título, data, curtidas, comentários, restacks, palavras, se é pago, autores, tags e o texto completo (opcional). Busca em todo o Substack por palavra-chave e lista as newsletters dos rankings por categoria (bestsellers e em alta) com faixa de assinantes pagos, assinantes, preços e autor. Sem login nem API key.

What you get

• 📰 Newsletter archives: any publication by subdomain (lenny), custom domain (www.lennysnewsletter.com), URL or author @handle. Newest first or Substack's top posts order, back to the first post.
• 🔎 Substack-wide search (searchQueries): posts matching a keyword across all newsletters, with their newsletter and subscriber count.
• 🏆 Leaderboards (leaderboardCategories): Substack's bestseller and rising lists per category (technology, business, finance, culture, politics, ...): rank, newsletter, URL, paid tier ("Thousands of paid subscribers"), subscribers when shown, monthly/yearly price, author, bio, first post date. Built for sponsorship prospecting, influencer and competitor research.
• 📄 Full text (includeContent): plain text + HTML of every post (free preview only for paid posts). Ideal for AI/RAG datasets, summaries and content analysis.
• 📈 Newsletter info on every post: name, URL, subscriber count as shown by Substack ("Over 1,200,000 subscribers") and paid tier.
• 📅 Date cut-off (postedAfter) for daily monitoring; duplicates removed across sources.
• 💸 $1 per 1,000 posts, $2 per 1,000 leaderboard newsletters, plus $0.005 per run. Runs with no results cost nothing.

Pricing

Pay-per-event: $0.001 per post and $0.002 per leaderboard newsletter (Store discounts for Silver/Gold/Platinum/Diamond plans) + $0.005 once per run, charged only if at least one result is found. Full text costs the same. Unknown newsletters, missing posts and duplicates are free. A spending limit is respected: the run stops cleanly at it.

Input (all optional)

{
  "publications": ["lenny", "https://www.astralcodexten.com", "@noahpinion"],
  "postUrls": ["https://www.lennysnewsletter.com/p/how-to-tell-if-a-startup-has-a-moat"],
  "searchQueries": ["ai agents"],
  "leaderboardCategories": ["technology", "finance"],
  "leaderboardType": "paid",
  "maxPostsPerSource": 100,
  "sort": "new",
  "includeContent": false,
  "postedAfter": "2026-10-01",
  "maxResults": 1000
}

Empty input returns the 25 newest posts of Lenny's Newsletter and The Pragmatic Engineer in a few seconds.

• maxPostsPerSource applies to each newsletter, each search query and each leaderboard category.
• postedAfter is a UTC date (YYYY-MM-DD). With sort: new the archive stops at the date, so older posts are never fetched or charged.

Output example (real run, 2026-10-28, shortened)

{
  "recordType": "post",
  "id": 219482066,
  "title": "How to tell if a startup has a moat",
  "subtitle": "A framework for assessing whether the company you're considering joining is built to last",
  "url": "https://www.lennysnewsletter.com/p/how-to-tell-if-a-startup-has-a-moat",
  "date": "2026-10-13T12:45:41.160Z",
  "type": "newsletter",
  "audience": "only_paid",
  "isPaid": true,
  "wordcount": 4153,
  "reactionCount": 372,
  "commentCount": 2,
  "restacks": 21,
  "authors": [{"name": "Dan Hockenmaier", "handle": "danhock", "profileUrl": "https://substack.com/@danhock"}],
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "publicationSubscribers": "Over 1,200,000 subscribers",
  "publicationPaidSubscribers": "Tens of thousands of paid subscribers"
}

Leaderboard row:

{
  "recordType": "publication",
  "rank": 1,
  "leaderboard": "Bestsellers in Technology (paid)",
  "publicationName": "SemiAnalysis",
  "publicationUrl": "https://newsletter.semianalysis.com",
  "paidSubscribers": "Tens of thousands of paid subscribers",
  "monthlyPrice": 50,
  "currency": "USD",
  "authorName": "Dylan Patel",
  "authorProfileUrl": "https://substack.com/@semianalysis"
}

Honest limits (read this)

• Public data only. It reads the same public JSON Substack's website uses. Paid posts return only the free preview (contentIsPreviewOnly: true); there is no login and no paywall bypass.
• Subscriber numbers are Substack's rounded labels ("Over 92,000 subscribers", "Thousands of paid subscribers"), and only when the newsletter shows them. Exact counts and subscriber lists/emails are not public, so they are not returned.
• Search is Substack's own search (relevance order, posts only; notes and comments are skipped).
• No paid proxy. In our tests (Oct 28, 2026) on Apify, every request worked without a proxy. If Substack answers 429/5xx, the Actor retries with backoff and then uses the Apify datacenter proxy included in your plan (capped per run); results already collected are kept.
• Posts are public but copyrighted and may contain personal data. Check Substack's terms and the laws that apply to your use (copyright, GDPR) before you republish or store them.

Use cases

Newsletter sponsorship and partnership prospecting · competitor and creator research · content and topic trend analysis · AI/RAG datasets from newsletters you follow · daily monitoring of newsletters with postedAfter · lead lists of top creators per category.

Scheduling and integrations

Run on a schedule with Apify Schedules and send new data through webhooks, Make, Zapier, n8n or the API. Plain REST: POST https://api.apify.com/v2/acts/datadesk-io~substack-scraper/run-sync-get-dataset-items?token=....

Use with AI agents (MCP)

This Actor works as a tool for AI agents through the Apify MCP server. Add it to Claude, ChatGPT, Cursor, VS Code or any MCP client:

https://mcp.apify.com/?tools=datadesk-io/substack-scraper

Sign in with OAuth or Authorization: Bearer <APIFY_TOKEN>. Local stdio config:

{"mcpServers": {"substack": {"command": "npx", "args": ["-y", "@apify/actors-mcp-server", "--tools", "datadesk-io/substack-scraper"], "env": {"APIFY_TOKEN": "<token>"}}}}

What goes in → what comes out: Input: newsletters (subdomain, domain, URL or @handle), post URLs, search keywords and/or leaderboard categories, with optional caps, order, full text and date cut-off. Output: one JSON row per post (title, date, engagement, authors, newsletter, optional full text) or per leaderboard newsletter (rank, paid tier, prices, author). Every input is optional and every output field is documented.

Example prompt: "Get the last 30 days of posts from Lenny's Newsletter and The Pragmatic Engineer with full text and summarize the main topics." Use maxPostsPerSource, postedAfter or maxResults to cap the cost of one call.

FAQ

Do I need a Substack account or API key? No. Can it read paid posts in full? No. Only the free preview that Substack shows to everyone. Can it export subscriber emails? No. Substack does not publish them. Issues? Report them on the Issues tab. We usually reply within 48 h.