📬 Substack Newsletter & Publication Scraper

Extract structured Substack newsletter articles, post metrics, authors, publication metadata, reading times, and full clean article text across any public Substack publication without API keys or browser automation overhead.

---

🥊 Why This Actor Beats the Competition

Feature	📬 Substack Newsletter Scraper	Generic Puppeteer Scrapers	Manual RSS Feeds
Speed	150+ articles / sec (Direct JSON protocol)	~3 to 5 seconds per article	Slow & limited to 10 latest items
Pricing	Pay-Per-Result ($1.50 / 1k posts)	Expensive compute runtime ($10+/mo)	Manual setup cost
Data Completeness	Post IDs, reactions, comment counts, word count, clean text	Partial HTML with messy styling	No reaction counts or full metrics
Custom Domains	Supported out-of-the-box	Requires manual selectors	Often fails on custom DNS
RAG / AI Ready	Clean text/markdown ready for vector DBs	Raw bloated DOM tags	Raw truncated summary only

---

🛠️ Step-by-Step: How to Use

Method 1: Apify Console (No-Code GUI)

1. Enter one or more Substack publication slugs (e.g. astralcodexten, lenny) or full URLs.
2. Select your desired maxPostsPerPublication.
3. Click Start and export your data as JSON, CSV, or Excel.

Method 2: Python SDK

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run_input = {
    "publications": ["astralcodexten", "lenny"],
    "maxPostsPerPublication": 15,
    "includeBodyText": True
}

run = client.actor("litezero/substack-newsletter-scraper").call(run_input=run_input)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"[{item['publishedAt']}] {item['title']} - {item['likesCount']} likes")

Method 3: Node.js SDK

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('litezero/substack-newsletter-scraper').call({
    publications: ['astralcodexten'],
    maxPostsPerPublication: 10,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Fetched ${items.length} Substack posts`);

Method 4: cURL / REST API

curl --request POST \
  --url "https://api.apify.com/v2/acts/litezero~substack-newsletter-scraper/runs?token=YOUR_APIFY_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{"publications": ["astralcodexten"], "maxPostsPerPublication": 5}'

---

📥 Input Parameters

Parameter	Type	Default	Description
publications	Array	["astralcodexten"]	List of Substack subdomains (e.g. astralcodexten) or custom URLs.
maxPostsPerPublication	Integer	10	Number of recent articles to scrape per publication.
includeBodyText	Boolean	true	Extract full clean text content of the article.

---

📤 Sample Output JSON

{
  "publication": "astralcodexten",
  "postId": 14928172,
  "title": "Why Foundation Models Are Becoming Commoditized",
  "subtitle": "An analysis of open weights vs closed API margins",
  "canonicalUrl": "https://astralcodexten.substack.com/p/why-foundation-models-commoditized",
  "publishedAt": "2026-09-28T14:20:00.000Z",
  "author": "Scott Alexander",
  "wordCount": 2450,
  "readingTimeMinutes": 12.3,
  "likesCount": 412,
  "commentsCount": 184,
  "audience": "everyone",
  "coverImage": "https://substackcdn.com/image/fetch/w_1456,c_limit...",
  "bodyText": "The frontier lab dynamics have shifted noticeably over the past twelve months...",
  "scrapedAt": "2026-10-02T16:45:00.000Z"
}

---

💡 Top Commercial Use Cases

1. AI RAG & LLM Ingestion: Feed authoritative industry newsletters into embeddings and knowledge graphs.
2. Competitive Intelligence: Monitor thought leadership, messaging, and announcement cadences across tech and finance.
3. Media Monitoring & Sponsorship Scouting: Identify top-performing newsletters with high engagement ratios for advertising deals.