# Substack Scraper: Posts, Newsletters & Leaderboards (`forgedata/substack-scraper`) Actor

Scrape Substack posts with full text, likes, comments and dates. Find newsletters by keyword with subscriber counts, author and bio, or pull Top overall and Top paid category leaderboards. No login, no browser. $1 per 1,000 results.

- **URL**: https://apify.com/forgedata/substack-scraper.md
- **Developed by:** [Data Forge](https://apify.com/forgedata) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper: Posts, Newsletters & Leaderboards

Get **Substack posts and newsletter data** in one actor: post titles, full text of free posts, likes, comments, restacks, word count and publish date, plus newsletter details like subscriber counts, author, author bio, paid plan and category rank. **$1 per 1,000 results.** No login, no cookies, no browser.

### Three ways to use it

- **Posts from these newsletters.** Paste newsletter addresses (`https://www.lennysnewsletter.com`, `astralcodexten.substack.com`, or just `astralcodexten`) and get every post, newest first, with optional full text.
- **Find newsletters by keyword.** Enter keywords like `recruiting`, `personal finance` or `artificial intelligence` and get every top newsletter whose name, description, author or author bio mentions them. Searches about 16,000 newsletters across Substack's category leaderboards, and tells you which keyword matched.
- **Top newsletters by category.** Pull Substack's leaderboards for Technology, Business, Finance, Culture and 30 more, either Top overall or Top paid, with each newsletter's rank.

Search and category modes can also return the **latest posts from each newsletter found**, so one run gives you a list of newsletters and what they are publishing.

### What you can use it for

- **Sponsorships and partnerships:** find newsletters in your niche, ranked, with subscriber counts and author details.
- **PR and influencer outreach:** build lists of writers who cover your topic, with their public bio and profile link.
- **Content and market research:** see what top writers publish, how often, and which posts get the most likes and comments.
- **AI and data teams:** clean, structured long-form text from free posts.
- **Monitoring:** schedule a daily run with **Posts since** to catch new posts from the newsletters you follow.

### What you get

**Posts**

| Field | Example |
|---|---|
| title / subtitle | How to price your product |
| url | link to the post |
| publication / publicationUrl | Lenny's Newsletter |
| authors / authorHandles | Lenny Rachitsky |
| postDate | 2026-10-21T13:02:00.000Z |
| audience / isPaywalled | everyone / false |
| reactions / commentCount / restacks | 412 / 38 / 25 |
| wordcount | 3,240 |
| postType / section / tags | newsletter, podcast or thread |
| content / contentLength | full text of free posts (when Include post text is on) |
| coverImage, description, podcastDurationSec, postId | |

**Newsletters**

| Field | Example |
|---|---|
| name / url / customDomain | Recruiting Brainfood |
| description | the newsletter's tagline |
| authorName / authorHandle / authorProfileUrl / authorBio | public author details |
| subscribersText / subscribersMin | Over 42,000 subscribers / 42000 |
| paidSubscriptionsEnabled / paidSubscribersText / bestsellerTier | |
| category / leaderboard / rank | Business / Top overall / 164 |
| matchedKeywords | recruiting |
| language, createdAt, logoUrl, freeBenefits, paidBenefits | |

Every row has a **type** field, `post` or `newsletter`, so you can split them in your spreadsheet. The dataset has a **Posts** view and a **Newsletters** view.

### How to use it

1. Pick **What to collect**.
2. For posts: add newsletters, set **Max posts per newsletter**, and optionally **Posts since**, **Free posts only** and **Include post text**.
3. For keyword search: add keywords, and optionally limit to a few categories for a faster run.
4. For categories: pick categories and Top overall or Top paid.
5. Optional filters for search and category modes: **Minimum subscribers** and **Language**.
6. Set **Max results**, run, and download as JSON, CSV or Excel, or use the API.

### Paid posts

For paid posts you get everything public: title, subtitle, date, likes, comments and the free preview Substack shows before the paywall. The actor never logs in, so it does not return paid content.

### Pricing

**$1 per 1,000 results** saved (posts and newsletters both count as one result), pay per result, and less on paid Apify plans (Starter $0.90, Scale $0.80, Business $0.70). You are never charged for rows removed by your filters or anything beyond your Max results.

### Is it legal?

This actor reads only public Substack pages and feeds that anyone can open without an account. It does not log in, it checks robots.txt before reading each site, and it does not collect comments, subscriber lists or private data. Posts are written by their authors, so follow copyright law if you republish them. This actor is not affiliated with Substack.

### Integrations and API

Run it from the Apify API, schedule it, or connect it to Make, Zapier, n8n, Google Sheets, Slack, Airtable and AI agents through the Apify MCP server.

### More from Data Forge

- [App Store Reviews Scraper](https://apify.com/forgedata/app-store-reviews-scraper): Apple App Store reviews from 100+ countries
- [Luma Events Scraper](https://apify.com/forgedata/luma-events-scraper): tech and startup events by city
- [Douban Reviews Scraper](https://apify.com/forgedata/douban-reviews-scraper): Chinese film, book and music reviews

# Actor input Schema

## `mode` (type: `string`):

Posts from the newsletters you list, newsletters matching your keywords, or the top newsletters in Substack's category leaderboards.

## `publications` (type: `array`):

Newsletter addresses, e.g. https://www.lennysnewsletter.com, astralcodexten.substack.com, or just the name before .substack.com (astralcodexten).

## `maxPostsPerPublication` (type: `integer`):

Newest posts first.

## `postsSince` (type: `string`):

Only posts published on or after this date.

## `freePostsOnly` (type: `boolean`):

Skip paid-subscriber posts.

## `includeContent` (type: `boolean`):

Add the full text of free posts. For paid posts only the free preview shown before the paywall is included.

## `searchQueries` (type: `array`):

Find newsletters whose name, description, author or author bio mention any of these words, e.g. artificial intelligence, personal finance, recruiting, real estate. Searches the top newsletters in Substack's category leaderboards (about 16,000).

## `searchInCategories` (type: `array`):

Optional. Limit keyword search to these categories (e.g. technology, business, finance) for a faster run. Empty means all categories.

## `categories` (type: `array`):

Substack categories: technology, business, finance, culture, us-politics, world-politics, news, science, health, sports, food, music, art, fashionandbeauty, faith, climate, literature, fiction, design, travel, parenting, philosophy, comics, crypto, history, humor, education, film-and-tv, home-garden, games, international.

## `leaderboard` (type: `string`):

Top overall or top by paid subscribers. Used by keyword search and category modes. Each category lists up to about 500 newsletters.

## `minSubscribers` (type: `integer`):

Only newsletters showing at least this many subscribers. 0 means no minimum.

## `language` (type: `string`):

Only newsletters in this language code, e.g. en, es, de, fr, pt. Leave empty for all.

## `includeLatestPosts` (type: `integer`):

For search and category modes: also collect this many recent posts from each newsletter found. 0 means newsletters only.

## `maxItems` (type: `integer`):

Stop after saving this many newsletters and posts in total. You pay only for results saved.

## Actor input object example

```json
{
  "mode": "posts",
  "publications": [
    "https://www.lennysnewsletter.com",
    "https://newsletter.pragmaticengineer.com"
  ],
  "maxPostsPerPublication": 50,
  "freePostsOnly": false,
  "includeContent": false,
  "searchQueries": [
    "artificial intelligence"
  ],
  "searchInCategories": [],
  "categories": [
    "technology"
  ],
  "leaderboard": "all",
  "minSubscribers": 0,
  "language": "",
  "includeLatestPosts": 0,
  "maxItems": 100
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "https://www.lennysnewsletter.com",
        "https://newsletter.pragmaticengineer.com"
    ],
    "searchQueries": [
        "artificial intelligence"
    ],
    "categories": [
        "technology"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("forgedata/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publications": [
        "https://www.lennysnewsletter.com",
        "https://newsletter.pragmaticengineer.com",
    ],
    "searchQueries": ["artificial intelligence"],
    "categories": ["technology"],
}

# Run the Actor and wait for it to finish
run = client.actor("forgedata/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "https://www.lennysnewsletter.com",
    "https://newsletter.pragmaticengineer.com"
  ],
  "searchQueries": [
    "artificial intelligence"
  ],
  "categories": [
    "technology"
  ]
}' |
apify call forgedata/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,forgedata/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/cuASkK8spm5yfboAs/builds/MFHjwRBVeHNgCtzhj/openapi.json