# Substack Scraper: Newsletter Posts, Subscribers & Authors (`torix33/substack-scraper`) Actor

Substack scraper and newsletter API, no login: subscriber counts, paid plan prices, leaderboard rank, author X/LinkedIn/website links, plus posts with likes, comments and full article text. Find newsletters by keyword or category. Fast JSON, no browser. Pay per result.

- **URL**: https://apify.com/torix33/substack-scraper.md
- **Developed by:** [Torix](https://apify.com/torix33) (community)
- **Categories:** AI, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.10 / 1,000 result scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper — Newsletters, Subscribers, Posts & Full Article Text

Scrape **any Substack newsletter, post or author** — no login, no browser, no API key. Find newsletters **by keyword** or straight from **Substack category leaderboards**, and get clean JSON with **subscriber counts, paid plan prices, leaderboard rank, author X/LinkedIn/YouTube/website links**, plus every post with **likes, comments, restacks, word count, paywall status and (optionally) the full article text**.

Export to JSON, CSV or Excel, or send it to Google Sheets, Make, Zapier, n8n, a vector database or your AI agent.

### How to run it in 3 steps

1. **Add your input.** In **Newsletter, post or profile URLs** paste Substack newsletter or post links — e.g. `https://www.lennysnewsletter.com`. The form is already filled with a working example, so you can skip this step for a first try.
2. **Press Start.** The prefilled run returns 21 results in about 3 s and costs about **$0.063**. Set **Max results (cost cap)** to cap the cost of bigger runs; you only pay for results actually delivered.
3. **Get the data.** Open the **Output** tab and export JSON, CSV or Excel — or run it from the API, Make, Zapier, n8n or an AI agent (see below).

### Who uses it

- **Sponsorship & influencer marketing**: build lists of newsletters in your niche with audience size, price and author contacts.
- **B2B lead generation**: writers with 10k+ subscribers, their websites and social profiles.
- **Content & competitor research**: which posts get the most likes and comments, how often competitors publish, what they put behind the paywall.
- **AI / RAG datasets**: full text of free posts as clean HTML and plain text.
- **Investors & analysts**: track leaderboard rankings and subscriber growth over time with scheduled runs.

### What you get

| Record | Fields |
|---|---|
| **Newsletter** (`type: "newsletter"`) | name, URL, custom domain, description, language, **subscriber count**, paid-subscriber tier, **monthly / annual price**, founding plan, **leaderboard category & rank**, bestseller tier, podcast & RSS feed, **author name, handle, bio, follower count, social links (X, LinkedIn, YouTube, Instagram, TikTok, website…)**, Stripe country, first post date, logo |
| **Post** (`type: "post"`) | title, subtitle, URL, publish date, type (newsletter / podcast / thread), audience (`everyone`, `only_paid`, `founding`), **likes, comments, restacks**, word count, tags, section, authors, cover image, preview text, podcast URL & duration — and with *Include full article text*: **`contentHtml` + `contentText`** |

### Four ways to choose what to scrape

1. **Newsletter URLs** — `https://name.substack.com`, custom domains (`https://www.lennysnewsletter.com`) or just the subdomain (`astralcodexten`).
2. **Post URLs** — `https://name.substack.com/p/post-slug` (also `open.substack.com/pub/...` links).
3. **Author profiles** — `https://substack.com/@handle`.
4. **Discovery**
   - **Keyword search** (`searchQueries`): e.g. `marketing`, `AI startups`, `real estate investing` — returns the newsletters of matching writers.
   - **Category leaderboards** (`categories`): `technology`, `business`, `finance`, `culture`, `science`, `crypto`, … — *Bestsellers* (top paid, 25 per category) or *Rising / all* (up to ~525 per category).

Mix them freely in one run. Newsletters found through several sources are returned once.

### Pricing

- The prefilled input is billed for 21 results → **$0.063**.

**$3.00 per 1,000 results** ($0.003 per result). One result = one newsletter **or** one post. Full article text, author enrichment and retries are included — no start fee, no extra per-request charges.

| Example | Results | Cost |
|---|---|---|
| Top 25 paid newsletters in *Technology*, details only (`scrapePosts: false`) | 25 | $0.08 |
| 100 marketing newsletters + their last 10 posts | 1,100 | $3.30 |
| Full archive of one newsletter (≈1,000 posts) with article text | ≈1,000 | ≈$3.00 |

Use **Max results (cost cap)** to put a hard ceiling on any run. The actor also stops by itself when your spending limit is reached.

### Input example

```json
{
  "startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"],
  "searchQueries": ["marketing"],
  "categories": ["technology"],
  "leaderboardType": "paid",
  "maxNewslettersPerSource": 25,
  "includeNewsletterDetails": true,
  "enrichAuthors": true,
  "scrapePosts": true,
  "maxPostsPerNewsletter": 10,
  "postedAfter": "30 days",
  "includeContent": false
}
```

| Option | Default | Notes |
|---|---|---|
| `maxNewslettersPerSource` | 25 | Per keyword / category. |
| `includeNewsletterDetails` | `true` | Turn off if you only want posts. |
| `enrichAuthors` | `true` | Adds author social links and follower count. |
| `scrapePosts` | `true` | Turn off for a pure newsletter/lead list. |
| `maxPostsPerNewsletter` | 20 | Newest first. `-1` = whole archive. |
| `postedAfter` | — | `2026-01-01` or `30 days`, `2 weeks`, `1 year`. |
| `includeContent` | `false` | Full text of free posts; preview for paywalled ones. |
| `maxResults` | 0 | 0 = unlimited. |

### Output example

#### Real output (default input)

First 2 of 21 records from a run with the prefilled input (3.4 s on a test machine; long values shortened here):

```json
[
  {
    "type": "newsletter",
    "id": 10845,
    "name": "Lenny's Newsletter",
    "url": "https://www.lennysnewsletter.com",
    "subdomain": "lenny",
    "customDomain": "www.lennysnewsletter.com",
    "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
    "language": "en",
    "subscriberCount": 1200000,
    "subscriberCountText": "Millions of subscribers",
    "paidSubscribersText": "Tens of thousands of paid subscribers",
    "isPaid": true,
    "monthlyPrice": 20,
    "annualPrice": 200,
    "currency": "USD",
    "foundingPlanName": "Insider Tier",
    "leaderboardCategory": "Business",
    "leaderboardRank": 1,
    "leaderboardType": "paid",
    "bestsellerTier": 10000,
    "hasPodcast": true,
    "podcastFeedUrl": null,
    "rssFeedUrl": "https://www.lennysnewsletter.com/feed",
    "authorName": "Lenny Rachitsky",
    "authorHandle": "lenny",
    "authorBio": "Writing • Angel investing • Advising",
    "authorProfileUrl": "https://substack.com/@lenny",
    "authorFollowerCount": 1385080,
    "authorLinks": {
      "twitter": "https://x.com/lennysan",
      "tiktok": "https://tiktok.com/@lennyrachtsky",
      "youtube": "https://www.youtube.com/c/LennysPodcast",
      "website": "https://www.lennyrachitsky.com/"
    },
    "supportEmail": null,
    "emailFromName": "Lenny's Newsletter",
    "copyright": null,
    "stripeCountry": "US",
    "firstPostDate": "2019-06-14T15:00:48.833Z",
    "createdAt": "2019-06-01T15:35:37.885Z",
    "logoUrl": "https://substackcdn.com/image/fetch/$s_!8MSN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fp…",
    "coverImageUrl": "https://substack-post-media.s3.amazonaws.com/public/images/4a25905e-c484-4885-80fa-9c2e683e362b_592x592.png",
    "source": "url",
    "scrapedAt": "2026-10-15T12:27:47.178Z"
  },
  {
    "type": "post",
    "id": 219482066,
    "title": "How to tell if a startup has a moat ",
    "subtitle": "A framework for assessing whether the company you’re considering joining is built to last",
    "url": "https://www.lennysnewsletter.com/p/how-to-tell-if-a-startup-has-a-moat",
    "slug": "how-to-tell-if-a-startup-has-a-moat",
    "postType": "newsletter",
    "publishedAt": "2026-10-13T12:45:41.160Z",
    "audience": "only_paid",
    "isPaywalled": true,
    "wordCount": 4153,
    "reactionCount": 301,
    "commentCount": 2,
    "restackCount": 10,
    "description": "A framework for assessing whether the company you’re considering joining is built to last",
    "previewText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: Lenny’s Jobs | Lenny’s Podcast | …",
    "coverImageUrl": "https://substackcdn.com/image/fetch/$s_!7Uce!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fp…",
    "tags": [
      "Career"
    ],
    "sectionName": null,
    "authors": [
      {
        "id": 2607892,
        "name": "Dan Hockenmaier",
        "handle": "danhock"
      }
    ],
    "podcastUrl": null,
    "podcastDurationSec": null,
    "newsletterId": 10845,
    "newsletterName": "Lenny's Newsletter",
    "newsletterUrl": "https://www.lennysnewsletter.com",
    "scrapedAt": "2026-10-15T12:27:47.420Z"
  }
]
```

**Newsletter**

```json
{
  "type": "newsletter",
  "id": 10845,
  "name": "Lenny's Newsletter",
  "url": "https://www.lennysnewsletter.com",
  "subdomain": "lenny",
  "customDomain": "www.lennysnewsletter.com",
  "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
  "language": "en",
  "subscriberCount": 1200000,
  "subscriberCountText": "Millions of subscribers",
  "paidSubscribersText": "Tens of thousands of paid subscribers",
  "isPaid": true,
  "monthlyPrice": 20,
  "annualPrice": 200,
  "currency": "USD",
  "leaderboardCategory": "Business",
  "leaderboardRank": 1,
  "leaderboardType": "paid",
  "hasPodcast": true,
  "rssFeedUrl": "https://www.lennysnewsletter.com/feed",
  "authorName": "Lenny Rachitsky",
  "authorHandle": "lenny",
  "authorProfileUrl": "https://substack.com/@lenny",
  "authorFollowerCount": 1384322,
  "authorLinks": {
    "twitter": "https://x.com/lennysan",
    "youtube": "https://www.youtube.com/c/LennysPodcast",
    "website": "https://www.lennyrachitsky.com/"
  },
  "stripeCountry": "US",
  "firstPostDate": "2019-06-14T15:00:48.833Z",
  "source": "url"
}
```

**Post** (with `includeContent: true`)

```json
{
  "type": "post",
  "id": 215243350,
  "title": "The Specter Of Neuralese",
  "url": "https://www.astralcodexten.com/p/the-specter-of-neuralese",
  "postType": "newsletter",
  "publishedAt": "2026-09-24T22:12:39.073Z",
  "audience": "everyone",
  "isPaywalled": false,
  "wordCount": 3037,
  "reactionCount": 359,
  "commentCount": 165,
  "restackCount": 28,
  "tags": [],
  "authors": [{ "id": 12009663, "name": "Scott Alexander", "handle": "astralcodexten" }],
  "newsletterName": "Astral Codex Ten",
  "newsletterUrl": "https://www.astralcodexten.com",
  "contentHtml": "<h4>I. At Long Last, We Have Created Neuralese Recurrence…</h4><p>…",
  "contentText": "I. At Long Last, We Have Created Neuralese Recurrence…\nIn the public debates after the release of AI 2027…",
  "isContentComplete": true
}
```

The dataset has two ready-made views in the Apify Console: **Newsletters** and **Posts**. Filter by `type` when exporting via API, e.g. `.../items?format=csv&fields=name,url,subscriberCount,authorLinks`.

### Ask in plain English

Using Apify AI, Claude, ChatGPT or another MCP client? Just describe what you need, the agent fills the input for you:

| You ask | Input the agent sends |
|---|---|
| "Get subscriber count and paid price of Lenny's Newsletter" | `{"startUrls": ["https://www.lennysnewsletter.com"]}` |
| "Find Substack newsletters about AI" | `{"searchQueries": ["AI"]}` |
| "Get the latest 10 posts with full text from Platformer" | `{"startUrls": ["https://www.platformer.news"], "scrapePosts": true, "maxPostsPerNewsletter": 10}` |

### Use with AI agents (MCP), LangChain, RAG & the API

This actor is a ready-made **tool for AI agents**: an LLM can call it, wait for the run and read clean JSON back.

**MCP server (Claude Desktop, Claude Code, Cursor, VS Code, any MCP client)** — add Apify's hosted MCP server with this actor enabled as a tool:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=torix33/substack-scraper",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}
```

**LangChain / LangGraph** (`pip install langchain-apify`, set `APIFY_API_TOKEN`):

```python
from langchain_apify import ApifyActorsTool, ApifyWrapper
from langchain_core.documents import Document

## 1) give the actor to an agent as a tool
tool = ApifyActorsTool("torix33/substack-scraper")

## 2) or load results as Documents for a RAG index
loader = ApifyWrapper().call_actor(
    actor_id="torix33/substack-scraper",
    run_input={"startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"], "searchQueries": ["marketing"], "categories": ["technology"], "leaderboardType": "paid", "maxNewslettersPerSource": 25, "includeNewsletterDetails": True, "enrichAuthors": True, "scrapePosts": True, "maxPostsPerNewsletter": 10, "postedAfter": "30 days", "includeContent": False},
    dataset_mapping_function=lambda item: Document(page_content=item["contentText"], metadata={"url": item.get("url")}),
)
docs = loader.load()
```

**REST API — one synchronous call, dataset items in the response:**

```bash
curl -X POST "https://api.apify.com/v2/acts/torix33~substack-scraper/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://www.lennysnewsletter.com", "https://substack.com/@noahpinion"], "searchQueries": ["marketing"], "categories": ["technology"], "leaderboardType": "paid", "maxNewslettersPerSource": 25, "includeNewsletterDetails": true, "enrichAuthors": true, "scrapePosts": true, "maxPostsPerNewsletter": 10, "postedAfter": "30 days", "includeContent": false}'
```

Also works with the Apify Python/JS clients, **Make, Zapier, n8n**, Google Sheets, webhooks and schedules. Export as JSON, CSV, Excel, XML or RSS.

**RAG tip:** With `includeContent: true` every post row carries the full article text (`contentText`) — embed it per post and keep `newsletterName`, `title`, `publishedAt` and `url` as metadata for cited answers.

### Speed

The actor reads Substack's own JSON endpoints over plain HTTP — no headless browser — so it is fast and cheap:

- \~700 records (120 newsletters + 590 posts) in **under a minute** on 256–512 MB memory.
- A full 180-post archive in ~4 seconds.
- When Substack rate-limits, the request is retried immediately through a new proxy IP — no waiting, no lost records.

### Limitations

- Paywalled posts return only the public preview, not the full text.
- Subscriber counts are what Substack shows publicly; some newsletters hide them.
- Only public newsletters and posts; no login.

### FAQ

**Do I need a Substack account or cookies?** No. Only publicly available data is collected.

**Can it get paywalled articles?** No. For paid posts you get all metadata plus the public preview text; `isContentComplete` tells you whether the text is the full article.

**Does it work with custom domains?** Yes — any newsletter hosted on Substack, whatever its domain.

**Are subscriber counts exact?** They are the numbers Substack shows publicly (e.g. "1,200,000" or "95K+"). Substack hides counts for some newsletters; those get `null` plus the textual tier in `subscriberCountText` when available.

**Do I need a proxy?** The actor uses Apify datacenter proxy by default (included in every Apify plan). Substack rate-limits cloud IPs, and the proxy lets the actor switch to a fresh IP immediately instead of waiting, which makes runs several times faster. You can turn it off for small runs.

**Email addresses?** Substack does not publish writer emails; the actor returns `supportEmail` when a newsletter sets one, plus author website and social links, which usually lead to contact details.

### Legal

This actor only extracts publicly available data. Make sure your use complies with Substack's Terms of Use and applicable data-protection law (e.g. GDPR) — especially when processing personal data such as author names and social profiles.

### Support

Found a bug or need an extra field? Open an issue on the **Issues** tab — feature requests are usually shipped within days.

# Actor input Schema

## `startUrls` (type: `array`):

Any mix of newsletter URLs (<code>https://name.substack.com</code> or a custom domain like <code>https://www.lennysnewsletter.com</code>), post URLs (<code>.../p/post-slug</code>), author profiles (<code>https://substack.com/@handle</code>) or a bare subdomain like <code>astralcodexten</code>.

## `searchQueries` (type: `array`):

Searches Substack writers (e.g. <code>marketing</code>, <code>AI startups</code>) and scrapes the newsletter of each match. Up to <b>Max newsletters per source</b> per keyword.

## `categories` (type: `array`):

Scrape the top newsletters of Substack categories. Slugs: technology, business, finance, culture, us-politics, world-politics, news, science, health, crypto, education, design, food, travel, sports, music, art, history, philosophy, fiction, literature, parenting, faith, climate, humor, comics, fashionandbeauty, film-and-tv, games, home-garden, international, health-politics.

## `leaderboardType` (type: `string`):

<b>Bestsellers</b> = top paid newsletters (max 25 per category). <b>Rising / all</b> = the broader list of up to ~525 newsletters per category.

## `maxNewslettersPerSource` (type: `integer`):

Limit of newsletters taken from each search keyword or category.

## `includeNewsletterDetails` (type: `boolean`):

One record per newsletter: subscriber count, paid plan prices, leaderboard rank, podcast/RSS feed, author name, bio and handle.

## `enrichAuthors` (type: `boolean`):

Adds the author's X/Twitter, LinkedIn, YouTube, website and other links plus follower count (one extra request per newsletter).

## `scrapePosts` (type: `boolean`):

Scrape posts of every newsletter (title, URL, date, likes, comments, restacks, word count, paywall status, tags, authors).

## `maxPostsPerNewsletter` (type: `integer`):

Newest posts first. <code>0</code> = no posts, <code>-1</code> = the whole archive.

## `postedAfter` (type: `string`):

Date like <code>2026-01-01</code> or a relative period like <code>30 days</code>, <code>2 weeks</code>, <code>1 year</code>. Empty = no date filter.

## `includeContent` (type: `boolean`):

Adds <code>contentHtml</code> and <code>contentText</code> to each post. Free posts get the full text; paywalled posts get the public preview. One extra request per free post.

## `maxResults` (type: `integer`):

Stop after this many records in total (newsletters + posts). <code>0</code> = no limit.

## `maxConcurrency` (type: `integer`):

Parallel requests. Substack's newsletter APIs are fast; requests to substack.com itself (search, leaderboards, profiles) are automatically paced.

## `proxyConfiguration` (type: `object`):

Substack rate-limits cloud IPs. The default Apify datacenter proxy lets the actor switch IP instantly when that happens, which makes runs several times faster. Turn it off only for small runs.

## Actor input object example

```json
{
  "startUrls": [
    "https://www.lennysnewsletter.com"
  ],
  "leaderboardType": "paid",
  "maxNewslettersPerSource": 25,
  "includeNewsletterDetails": true,
  "enrichAuthors": true,
  "scrapePosts": true,
  "maxPostsPerNewsletter": 20,
  "includeContent": false,
  "maxResults": 0,
  "maxConcurrency": 10,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `newsletters` (type: `string`):

No description

## `posts` (type: `string`):

No description

## `runStats` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://www.lennysnewsletter.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("torix33/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://www.lennysnewsletter.com"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("torix33/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://www.lennysnewsletter.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call torix33/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,torix33/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/iH0QCKUoCfvNOCZvJ/builds/ocGOf39xyc6uAjyrL/openapi.json