# Substack Scraper – Newsletters, Subscribers & Posts (`smoldesk/substack-scraper`) Actor

Scrape Substack newsletters without login: subscriber counts, paid tiers, prices, author socials and posts with likes, comments and full text. Discover newsletters by category.

- **URL**: https://apify.com/smoldesk/substack-scraper.md
- **Developed by:** [Smoldesk Studios](https://apify.com/smoldesk) (community)
- **Categories:** Social media, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 newsletters

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper – Subscriber Counts, Newsletters & Posts

Scrape **Substack newsletters** without login or API keys. Get each newsletter's **subscriber count**, **paid subscriber tier**, **subscription prices**, **author bio and X/Twitter**, plus **posts** with **likes, comments, restacks, word count** and optional **full text**.

**Discover newsletters by category**, too (Technology, Business, Finance, Crypto and 28 more), straight from Substack's leaderboards.

### What you can use it for

- **Sponsorship & influencer marketing**: find newsletters in your niche with their audience size, then shortlist the best fit for ads.
- **Lead generation**: build lists of creators with subscriber counts, paid status and X/Twitter handles.
- **Competitor & content research**: see which topics and formats get the most likes, comments and restacks.
- **Market research**: compare pricing of paid newsletters in a category.
- **AI / LLM datasets**: collect clean full-text posts for summarization, RAG or trend analysis.

### How to scrape Substack

1. **Create a free Apify account** (no credit card needed to start) and open this Actor.
2. **Pick the newsletters.** Paste newsletter links or names in **Newsletters**, or use **Discover by category** to find the top newsletters of a topic.
3. **Get the numbers.** Leave *Get subscriber counts & prices* on to see subscriber counts and paid-plan prices.
4. **Add posts (optional).** Turn on *Include posts*, set *Max posts per newsletter* and *Posts published after*. Add *Include full post text* if you want the article body for AI or analysis.
5. **Click Start.** Results appear in the **Output** tab while it runs.
6. **Download or connect.** Export as JSON, CSV, Excel or HTML, or read the dataset through the API, Make, Zapier, n8n, Google Sheets or a webhook.
7. **Automate it.** Open **Schedules** and run weekly with *Posts published after* set to last week to track new posts.

**Tips**

- Subscriber counts are only shown by newsletters that make them public; the rest return no count.
- Combine *Discover by category* with subscriber counts to build a lead list of newsletters in a niche.
- Paid posts return their free preview only.

### Output: newsletter

```json
{
  "type": "publication",
  "name": "Lenny's Newsletter",
  "url": "https://www.lennysnewsletter.com",
  "freeSubscriberCount": 1200000,
  "subscriberCountLabel": "1.2M+",
  "paidSubscribersLabel": "Tens of thousands of paid subscribers",
  "paidEnabled": true,
  "monthlyPrice": 20,
  "yearlyPrice": 200,
  "priceCurrency": "USD",
  "authorName": "Lenny Rachitsky",
  "authorBio": "Writing • Angel investing • Advising",
  "authorTwitter": "https://x.com/lennysan",
  "authorProfileUrl": "https://substack.com/@lenny",
  "language": "en",
  "hasPodcast": true,
  "firstPostDate": "2019-06-14T15:00:48.833Z"
}
```

### Output: post

```json
{
  "type": "post",
  "publicationName": "Lenny's Newsletter",
  "title": "How to tell if a startup has a moat",
  "url": "https://www.lennysnewsletter.com/p/how-to-tell-if-a-startup-has-a-moat",
  "publishedAt": "2026-10-13T12:45:41.160Z",
  "isPaid": true,
  "wordCount": 4153,
  "likes": 240,
  "comments": 0,
  "restacks": 5,
  "authors": [{"name": "Dan Hockenmaier", "handle": "danhock", "isGuest": true}],
  "tags": ["Career"],
  "contentText": "… (when Include full post text is on)"
}
```

### Input

| Field | Description |
|---|---|
| `publications` | Subdomains (`lenny`), `*.substack.com` URLs or custom domains |
| `categories` | Discover newsletters from Substack category leaderboards |
| `maxPublicationsPerCategory` | How many newsletters per category |
| `enrichPublications` | Subscriber counts, prices and author socials (recommended) |
| `includePosts` / `maxPostsPerPublication` / `postsSince` | Which posts to return |
| `includePostContent` | Full HTML + clean text (paid posts: free preview only) |

### Pricing

Pay per result, no monthly rental:

| Event | Price |
|---|---|
| Newsletter (with subscriber count, prices, author socials) | **$3.00 per 1,000 newsletters** |
| Post | **$1.00 per 1,000 posts** |

Example: 200 newsletters from a category with their last 10 posts each cost **$2.60**. Set **"Maximum cost per run"** in the run options to stay within budget. The Actor stops cleanly at your limit.

### Use with AI (LangChain / LlamaIndex / CrewAI)

Turn on `includePostContent` and each post's `contentText` is clean plain text, ready for summarisation, trend analysis or RAG. With LangChain (`pip install langchain-apify`):

```python
from langchain_apify import ApifyWrapper
from langchain_core.documents import Document

apify = ApifyWrapper()  # reads APIFY_API_TOKEN
loader = apify.call_actor(
    actor_id="smoldesk/substack-scraper",
    run_input={"publications": ["lenny"], "includePosts": True, "maxPostsPerPublication": 20, "includePostContent": True},
    dataset_mapping_function=lambda p: Document(
        page_content=p.get("contentText") or "",
        metadata={k: p.get(k) for k in ("type", "url", "title", "publicationName", "publishedAt", "likes")},
    ),
)
docs = [d for d in loader.load() if d.metadata["type"] == "post"]
```

Already have a run? Load its dataset with `ApifyDatasetLoader(dataset_id="<run's dataset ID>", dataset_mapping_function=...)`. **LlamaIndex**: `ApifyActor` from `llama-index-readers-apify`. **CrewAI / Claude / MCP clients**: add it as a tool via the Apify MCP server: `https://mcp.apify.com?actors=smoldesk/substack-scraper`.

### Limitations

- Subscriber counts are what Substack shows publicly (e.g. "1.2M+"). Some newsletters hide them.
- Paid posts only include the free preview text.
- Only public data is collected. No login, emails or private subscriber lists.

### FAQ

**Does Substack have an API?** Not an official public one. This Actor reads the same public pages and endpoints any visitor's browser loads, so you get structured data without an API key or login.

**Can I get subscriber emails?** No. Subscriber lists are private. You get the public subscriber count and the author's public profile and X/Twitter.

**Do I need proxies?** No. It works without a proxy.

**Is it legal?** This Actor only collects data that Substack publishes openly to every visitor. Respect copyright when reusing post content, and follow privacy laws when contacting creators.

### Feedback

Found a bug or missing a field? Open an issue in the **Issues** tab. If the Actor saved you time, a short review on this page helps other people find it.

### More Actors from Smoldesk Studios

- [Weibo Scraper](https://apify.com/smoldesk/weibo-scraper): keyword and hashtag search, posts, comments, profiles and the hot-search board from China's biggest microblog, no login
- [Google Hotels Scraper](https://apify.com/smoldesk/google-hotels-scraper): hotel prices for your dates in any currency, ratings, review aspects and booking-site prices, worldwide
- [Bilibili Scraper](https://apify.com/smoldesk/bilibili-scraper): videos, danmaku, comments and creator stats from China's YouTube (哔哩哔哩)
- [Google Trends Scraper](https://apify.com/smoldesk/google-trends-scraper): interest over time, by region and city, related queries and Trending now — no API key
- [Telegram Channel Scraper](https://apify.com/smoldesk/telegram-channel-scraper): public channel posts, views, reactions and subscribers
- [Shopify Store Analyzer](https://apify.com/smoldesk/shopify-store-analyzer): apps, contacts and products of any Shopify store
- [SEEK, JobStreet & JobsDB Scraper](https://apify.com/smoldesk/seek-jobs-scraper): jobs and salaries across Australia and Southeast Asia
- [Career Site Jobs Scraper](https://apify.com/smoldesk/career-site-jobs-scraper): live jobs from 13,000+ company career sites (Greenhouse, Lever, Workday, Ashby) with salary and seniority

# Actor input Schema

## `publications` (type: `array`):

Substack subdomains or newsletter URLs, e.g. lenny, pomp.substack.com or https://www.lennysnewsletter.com (custom domains work).

## `categories` (type: `array`):

Find newsletters from Substack's category leaderboards.

## `maxPublicationsPerCategory` (type: `integer`):

How many newsletters to take from each category.

## `enrichPublications` (type: `boolean`):

Visit each newsletter's homepage for subscriber count, paid tier label, plan prices and author socials.

## `includePosts` (type: `boolean`):

Return posts with likes, comments, restacks, word count and authors. Charged per post.

## `maxPostsPerPublication` (type: `integer`):

Newest first. 0 = all posts.

## `postsSince` (type: `string`):

Stop at posts older than this date (YYYY-MM-DD).

## `includePostContent` (type: `boolean`):

Add full HTML and clean text of each post (paid posts return the free preview only).

## `maxConcurrency` (type: `integer`):

How many newsletters to scrape at the same time.

## `proxyConfiguration` (type: `object`):

Usually not needed. Enable for very large runs.

## Actor input object example

```json
{
  "publications": [
    "lenny",
    "pomp"
  ],
  "maxPublicationsPerCategory": 50,
  "enrichPublications": true,
  "includePosts": true,
  "maxPostsPerPublication": 20,
  "includePostContent": false,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `publications` (type: `string`):

Newsletter profiles (type = publication).

## `posts` (type: `string`):

Posts (type = post).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "lenny",
        "pomp"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("smoldesk/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "publications": [
        "lenny",
        "pomp",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("smoldesk/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "lenny",
    "pomp"
  ]
}' |
apify call smoldesk/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,smoldesk/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9M3YGyTZjFHckWCVb/builds/QqwiAmiuIvXu8gYbS/openapi.json