Substack Scraper — Newsletter Posts, Search & Profiles

Substack Scraper extracts posts and newsletter profiles from substack.com and from any newsletter that runs on Substack under its own domain. Give it newsletter addresses, keywords or categories and get one flat JSON row per post: title, subtitle, link, publication date in ISO 8601, free or paid, word count, likes, comment count, restacks, tags, authors, cover image, podcast audio, and the newsletter it belongs to. Turn on the body add-on to also get each post's text or HTML. No browser and no login: the Actor reads the same public JSON that Substack's own web pages load.

Pay per event. Post bodies are an add-on, charged per post returned with text. Newsletter profiles are charged per newsletter row.

What you can scrape

• Whole newsletter archives. Paste simonw, https://simonw.substack.com or a custom domain such as https://www.lennysnewsletter.com. The archive is read newest first, or most popular first with sort: "top", 50 posts per request, as deep as you want (maxPostsPerSource: 0 reads the whole archive).
• Single posts. Post links (https://name.substack.com/p/slug, custom-domain /p/ links, open.substack.com/pub/name/p/slug).
• Search inside a newsletter. archiveSearch uses the newsletter's own archive search, for example every Lenny's Newsletter post about pricing.
• Post search across Substack. postSearchQueries returns the posts Substack's search finds for a keyword, from any newsletter.
• Newsletter discovery. searchQueries finds newsletters by keyword; categories takes the top of Substack's category leaderboards (32 categories, from Technology, Business and Finance to Culture, Politics and Science). Their posts are then scraped like any other newsletter.
• Newsletter profiles. Set outputType: "newsletters" to get one row per newsletter instead of posts.

Output

One row per post:

Field	Meaning
postId, title, subtitle, url, slug	Identity and canonical link (custom domain when there is one)
publishedAt, updatedAt	ISO 8601 UTC dates (updatedAt for single post links and the body add-on)
type, audience, isPaywalled	newsletter or podcast; everyone or only_paid
description, previewText	SEO description and the opening lines shown in the archive
wordCount	Length of the full post (also known for paid posts)
likes, commentCount, restacks	Engagement as shown on Substack
tags, section, language, coverImageUrl	Post metadata
podcastUrl, podcastDurationSecs, hasVoiceover	Audio for podcast episodes
authors, authorNames	Bylines: name, handle, profile link, guest flag
newsletterId, newsletterName, newsletterUrl, newsletterSubdomain	The newsletter
contentText, contentHtml, contentWordCount, contentIsPreview	Body add-on (includeContent): the full post, or the free preview for paid posts
source, sourceQuery, scrapedAt	Which input produced the row

With outputType: "newsletters" each row describes a newsletter: name, url, substackUrl, customDomain, description, authorName, authorHandle, contributors, language, launchedAt, subscriberCount (as displayed by Substack, when the author shows it), subscriberCountLabel (for example 73K+), audienceLabel, paidSubscriptionsEnabled, monthlyPrice, yearlyPrice, foundingMemberPrice, priceCurrency, hasPodcast, podcastTitle, logoUrl, twitterHandle, rank (position in the search or leaderboard).

The key-value store also gets SUMMARY (rows saved and charged, posts seen and filtered, requests, stop reason) and SOURCES (status of every newsletter, search and category, with the reason when something returned nothing).

Input examples

Default run: the 10 newest posts of two newsletters.

{
  "newsletters": ["https://simonw.substack.com", "https://www.lennysnewsletter.com"],
  "maxPostsPerSource": 10,
  "maxItems": 20
}

Every free post of the last 90 days from the top 10 Finance newsletters, with the text:

{
  "categories": ["finance"],
  "maxNewslettersPerSearch": 10,
  "maxPostsPerSource": 0,
  "publishedAfter": "90 days",
  "audience": "free",
  "includeContent": true,
  "maxItems": 2000
}

Sponsorship research: profiles of newsletters about product management, with subscriber labels and plan prices:

{
  "outputType": "newsletters",
  "searchQueries": ["product management", "growth marketing"],
  "maxNewslettersPerSearch": 25
}

Posts about a topic from all of Substack, most liked only:

{ "postSearchQueries": ["AI agents"], "maxPostsPerSource": 50, "minLikes": 50 }

Example output

{
  "postId": 221405112,
  "title": "Battle Drones - a vibe coded RTS with audio-based multiplayer",
  "subtitle": "Plus Muse Wonder and Gemini 4 Argon",
  "url": "https://simonw.substack.com/p/battle-drones-a-vibe-coded-rts-with",
  "publishedAt": "2026-10-21T00:01:39.484Z",
  "type": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "wordCount": 6518,
  "likes": 38,
  "commentCount": 6,
  "restacks": 1,
  "tags": ["anthropic", "llms", "ai"],
  "authors": [{ "name": "Simon Willison", "handle": "simonw", "profileUrl": "https://substack.com/@simonw", "isGuest": false }],
  "newsletterName": "Simon Willison’s Newsletter",
  "newsletterUrl": "https://simonw.substack.com",
  "source": "newsletter",
  "scrapedAt": "2026-10-21T13:19:11.526Z"
}

Use cases

• Content and trend research: what the biggest newsletters in a niche publish, how often, and which posts earn the most likes and restacks.
• Sponsorship and partnership prospecting: newsletter profiles with audience labels, paid plan prices and authors, ranked by category.
• Competitive monitoring: schedule a daily run with publishedAfter: "1 day" on the newsletters you follow and push new posts to Slack, a sheet or a CRM.
• AI and RAG pipelines: contentText of free posts, ready to summarise or embed; an AI agent can call the Actor with a keyword and get the latest posts on the topic.
• Academic and media analysis: full archives with dates and engagement for a set of writers.

Pricing

Pay per event, no subscription:

• post-result: charged per post row saved.
• post-content (add-on, only with includeContent): charged per post whose body was returned.
• newsletter-result: charged per newsletter row.

Apify adds its small compute cost; the default run takes a few seconds. maxItems is exact and the Actor also stops at the maximum charge you set for the run. See the Pricing tab for current prices; paid Apify plans get lower rates.

Compared with alternatives

	This Actor	A newsletter's RSS feed	Browser-based scrapers
Archive depth	entire archive, paged	recent posts only	slow, page by page
Likes, comments, restacks, paid flag, word count	yes	no	sometimes
Search and category discovery	yes	no	rarely
Speed	50 posts per request	1 request	one page load per post

If you only need the latest items of many different feeds (Substack or not), our RSS & Atom Feed Reader is the general tool; this Actor is built on the same HTTP core and adds what only Substack's API has: engagement, paid/free status, full archives, search and newsletter profiles.

Integrations

Use the Apify API, the JavaScript or Python client, the CLI, schedules and webhooks, and export to JSON, CSV, Excel, Google Sheets, Make, Zapier, n8n or Airbyte. The Actor also works through the Apify MCP server, so AI agents can ask for "the 20 most liked posts of a newsletter" or "newsletters about climate tech".

FAQ

Is it legal? The Actor reads public pages and the public JSON Substack serves to every visitor, without logging in, and skips paywalled text beyond the preview the author makes public. Reader comments are never collected, and e-mail addresses and international phone numbers found in text are removed. Respect each author's copyright when you reuse the content.

Do I get paid posts in full? No. Paid posts give their metadata (title, date, likes, word count) and, with the add-on, the free preview the author chose to show (contentIsPreview: true).

Do I need a proxy? Normally not: requests go out directly. If Substack blocks the run, the Actor switches to Apify datacenter proxies and then residential proxies by itself (turn this off with autoProxyFallback: false).

Why is subscriberCount empty for some newsletters? Substack shows a count only when the author allows it; subscriberCountLabel and audienceLabel still give the range.

Limits? Up to 100,000 rows per run. Post search pages through whatever Substack's own search returns for a keyword, so very generic keywords stop when Substack stops returning results.

En español

Substack Scraper extrae los posts de cualquier newsletter de Substack (también con dominio propio): título, fecha ISO, gratis o pago, palabras, likes, comentarios, restacks, etiquetas, autores y el texto opcional. También busca posts en todo Substack, encuentra newsletters por palabra clave o categoría y entrega perfiles de newsletters con suscriptores y precios. Consulta la pestaña Pricing para ver los precios actuales; los planes de pago de Apify tienen tarifas más bajas.

Changelog

• 0.1 (October 2026): first release. Archives (new/top, search inside a newsletter), single post links, post search, newsletter search, category leaderboards, newsletter profiles, date / type / audience / likes filters, body add-on, automatic proxy fallback.