# Substack Scraper – Posts, Comments & Subscribers (`signalwharf/substack-scraper`) Actor

Substack scraper and Substack API alternative, no login. Scrape newsletter posts with full text, likes, comments and restacks, plus publication profiles: subscriber estimates, paid plan prices, bestseller tier and posting cadence. Custom domains, filters and new-post monitoring.

- **URL**: https://apify.com/signalwharf/substack-scraper.md
- **Developed by:** [Signal Wharf](https://apify.com/signalwharf) (community)
- **Categories:** Social media, News, Marketing
- **Features:** Limited permissions
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 posts

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper: posts, comments & subscriber data (Substack API alternative)

**Scrape any Substack newsletter without logging in or using an API key.** Each run gives you three things:

- **Posts** with the full text, likes, comments, restacks, a paid/free flag, tags, authors and outbound links
- **Publication profiles**: a subscriber estimate, paid plan prices, the Substack bestseller tier, posting cadence and engagement averages
- **Comments** with their replies (optional)

You can filter by date or keyword. A **monitoring mode** returns only the posts that are new since the last run. Full text costs nothing extra, and a monitoring run that finds nothing new isn't charged.

It works with `*.substack.com` addresses, custom domains (e.g. `www.lennysnewsletter.com`), and `substack.com/@handle` profile links.

### What can this Substack scraper do?

- 📰 **Full text costs the same as metadata.** Each post is one flat price.
- 📊 **Publication intelligence, not just posts.** You get subscriber counts (e.g. "Over 466,000 subscribers" → `466000`) and monthly, yearly and founding prices. You also get the bestseller tier, posts per week, the share of posts that are paid, and average likes, comments and restacks. That's everything you need to compare newsletters for sponsorships, research, or competitive analysis.
- 🔔 **Monitoring mode.** Schedule it every hour and receive only new posts. A run with nothing new costs you **$0**.
- 💬 **Optional comments**, with their reply threads (parent id and depth).
- 🔗 **Outbound links** from each post, for link and citation analysis.
- 🛡️ **Robust.** Polite rate limiting and retries. One broken publication never fails the run, and you get a per-publication `RUN_SUMMARY`.

### Substack scraper use cases

- **Newsletter sponsorships and media buying:** shortlist newsletters by audience size, price and engagement.
- **Market and competitor research:** see what newsletters in your niche publish, how often, and what gets engagement.
- **AI, RAG and LLM pipelines:** clean full text plus metadata for summarizers, vector stores and agents (works through the Apify API and MCP server).
- **Monitoring and alerts:** watch a list of publications and send new posts to Slack, email, Zapier, Make or n8n.
- **Academic and journalism research:** build datasets of essays, discussions and comment threads.

### How to scrape Substack

1. Add one or more **publications**: a subdomain (`noahpinion`), a Substack URL, a custom domain (`https://www.lennysnewsletter.com`), a post URL, or a `substack.com/@handle` profile.
2. Choose **max posts per publication**, and optionally a **date range**, **keywords**, **comments**, or **only new posts**.
3. Click **Start**. Posts, publication profiles and comments go to one dataset. Use `itemType` to tell them apart.
4. Export to **JSON, CSV, Excel**, or read the results through the Apify API. To monitor newsletters, schedule the Actor with **Only new posts** turned on.

### Input

| Field | Default | Description |
|---|---|---|
| `publications` | – | **Required.** Subdomains (`noahpinion`), URLs, custom domains, or `substack.com/@handle` links |
| `maxPostsPerPublication` | 50 | The most posts to save per publication, newest first |
| `dateFrom` / `dateTo` | – | Date range (`YYYY-MM-DD` or ISO, UTC) |
| `keywords` | – | Keep posts matching any keyword in the title, subtitle, tags or text |
| `includeFullText` | true | Full plain text plus outbound links (free preview only for paid posts) |
| `includeHtml` | false | Also return the raw HTML |
| `includePaidPosts` | true | Include subscriber-only posts (metadata plus the public preview) |
| `includePublicationProfile` | true | One profile item per publication |
| `includeComments` | false | Public comments with replies |
| `maxCommentsPerPost` | 50 | Cap on comments per post |
| `onlyNewPosts` | false | Monitoring mode |
| `stateStoreName` | `substack-scraper-state` | Where monitoring state is kept. Use one name per monitor |
| `resetMonitoringState` | false | Start monitoring from scratch |

```json
{
  "publications": ["noahpinion", "https://www.lennysnewsletter.com", "https://substack.com/@platformer"],
  "maxPostsPerPublication": 100,
  "dateFrom": "2026-10-01",
  "includeComments": true,
  "maxCommentsPerPost": 20
}
```

### Output

Every dataset item has an `itemType`: `post`, `publication`, or `comment`. The Console offers ready views: **Posts**, **Content**, **Publication profiles** and **Comments**.

**Post** (trimmed):

```json
{
  "itemType": "post",
  "publication": "noahpinion",
  "publicationName": "Noahpinion",
  "postId": 224380601,
  "url": "https://www.noahpinion.blog/p/india-is-quietly-doing-great",
  "title": "India is quietly doing great",
  "subtitle": "The world’s biggest country continues to defy the skeptics.",
  "postDate": "2026-11-11T08:51:56.921Z",
  "type": "newsletter",
  "isPaid": false,
  "authors": [{ "name": "Noah Smith", "handle": "noahpinion" }],
  "wordcount": 1596,
  "reactionCount": 404,
  "commentCount": 25,
  "restacks": 47,
  "bodyText": "My last post was about how for the first time, I’m becoming bearish on China’s ...",
  "isFullTextComplete": true,
  "links": ["https://ourworldindata.org/grapher/human-development-index?tab=line&country=IND~CHN"]
}
```

**Publication profile** (trimmed):

```json
{
  "itemType": "publication",
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "authorName": "Lenny Rachitsky",
  "subscriberCountText": "Over 1,200,000 subscribers",
  "subscriberCountApprox": 1200000,
  "paidSubscriberTierText": "Tens of thousands of paid subscribers",
  "bestsellerTier": 10000,
  "monthlyPrice": 20.0,
  "yearlyPrice": 200.0,
  "foundingMemberPrice": 400.0,
  "postsPerWeek": 0.69,
  "paidPostShare": 0.8,
  "avgWordcount": 3622,
  "avgReactions": 488.7,
  "avgComments": 11.5
}
```

**Comment**: `postId`, `postUrl`, `commentId`, `parentCommentId`, `depth`, `authorName`, `authorHandle`, `body`, `date`, `reactionCount` and `replyCount`.

The stats in a profile (cadence, averages, paid share) are calculated from the posts saved in that run. Scrape more posts for more reliable stats.

### How much does it cost to scrape Substack?

Pay per event, with no subscription:

| Event | Price |
|---|---|
| Post (full text included) | **$0.0015** ($1.50 per 1,000) |
| Publication profile | $0.003 |
| Comment (only if enabled) | $0.0003 |

Examples:

- 1 newsletter, last 100 posts with full text and a profile = **$0.153**
- 20 newsletters × 25 posts plus profiles = **$0.81**
- An hourly monitor on 10 newsletters with about 5 new posts a day = about **$0.02/day**. Runs with nothing new cost $0.

Set a **maximum cost per run** in the run options, and the Actor stops cleanly when it gets there.

### Limits and good to know

- **No login and no paywall bypass.** For subscriber-only posts you get the metadata plus the free preview Substack shows every visitor (`isFullTextComplete: false`).
- Some publications hide their subscriber count, so those fields are `null`. Substack shows subscriber counts as rounded public text, so treat them as estimates.
- Archive pages hold 25 posts. Full text needs one extra request per post, at about 0.8 s each by default.
- Substack Notes, chats and private member data aren't collected.
- Dates are in UTC.

### Is it legal to scrape Substack?

This Actor reads only information Substack publishes openly to every visitor, through the same public endpoints its website uses. It doesn't log in, bypass paywalls, or collect email addresses or private data. Comments include public display names and handles, which are personal data. Make sure your use complies with the GDPR and other privacy laws, copyright law, and Substack's terms. Don't republish authors' full articles without permission. If you're unsure, consult a lawyer. See [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

### FAQ

**Does it work with custom domains?** Yes. Enter the domain, a post URL, or the `substack.com/@handle` profile link.

**Can I get the whole archive?** Yes. Raise `maxPostsPerPublication`, or use `dateFrom`.

**How do I get alerts for new posts?** Turn on `onlyNewPosts`, schedule the Actor, and add a webhook or integration on *Run succeeded*.

**Is there an official Substack API?** Substack has no public developer API for reading other newsletters. This Actor works as a Substack API alternative: one call returns clean JSON for posts, comments and publication data.

**Does it scrape subscriber emails?** No. It collects only public information: never email lists or private subscriber data.

**Can I scrape Substack Notes?** Not yet. This Actor covers newsletter posts, comments and publication profiles.

**Can I use it from Python?** Yes. Use the Apify Python client or the REST API, then download the dataset.

**Something is missing or broken?** Open an issue on the Issues tab.

### Related signalwharf Actors

- [Telegram Channel Scraper](https://apify.com/signalwharf/telegram-channel-scraper): scrape public Telegram channel posts, views, reactions and media without login.
- [Substack Scraper](https://apify.com/signalwharf/substack-scraper): Substack posts with full text, comments, subscriber estimates and paid plan prices.
- [AI Brand Visibility Tracker](https://apify.com/signalwharf/ai-brand-visibility-tracker): track how often ChatGPT, Claude, Gemini and Perplexity mention your brand vs competitors.

# Actor input Schema

## `publications` (type: `array`):

Publication subdomains, Substack URLs, custom domains or profile links, e.g. <code>noahpinion</code>, <code>https://www.astralcodexten.com</code>, <code>lenny.substack.com</code>, <code>https://substack.com/@platformer</code>.

## `maxPostsPerPublication` (type: `integer`):

Maximum number of posts to save per publication (after filters). Newest first.

## `dateFrom` (type: `string`):

Only posts published on/after this date (YYYY-MM-DD or ISO, UTC). Scraping stops once older posts are reached.

## `dateTo` (type: `string`):

Only posts published on/before this date (whole day included, UTC).

## `keywords` (type: `array`):

Keep only posts whose title, subtitle, tags or text contain at least one keyword (case-insensitive).

## `includeFullText` (type: `boolean`):

Fetch each post's full text (plain text plus outbound links). For paid posts only the free preview is available. No extra charge.

## `includeHtml` (type: `boolean`):

Also include the raw post HTML (bigger items).

## `includePaidPosts` (type: `boolean`):

Include subscriber-only posts (metadata plus the public preview). Turn off to save only free posts.

## `includePublicationProfile` (type: `boolean`):

Add one profile item per publication: subscriber estimate, paid plan prices, bestseller tier, posting cadence and engagement averages.

## `includeComments` (type: `boolean`):

Also save public comments (with replies) for each post.

## `maxCommentsPerPost` (type: `integer`):

Maximum comments (including replies) saved per post.

## `onlyNewPosts` (type: `boolean`):

Return only posts published since the previous run for each publication. Ideal for scheduled runs. Runs with nothing new charge nothing.

## `stateStoreName` (type: `string`):

Named key-value store that remembers the last post per publication. Use a different name for each separate monitor.

## `resetMonitoringState` (type: `boolean`):

Forget the remembered last post for the given publications before scraping.

## `maxPagesPerPublication` (type: `integer`):

Safety cap; each archive page holds 25 posts.

## `requestDelaySecs` (type: `number`):

Polite delay between HTTP requests.

## `maxRetries` (type: `integer`):

Retries for 429/5xx responses and network errors.

## `proxyConfiguration` (type: `object`):

Optional. Usually not needed.

## Actor input object example

```json
{
  "publications": [
    "noahpinion",
    "https://www.astralcodexten.com"
  ],
  "maxPostsPerPublication": 20,
  "includeFullText": true,
  "includeHtml": false,
  "includePaidPosts": true,
  "includePublicationProfile": true,
  "includeComments": false,
  "maxCommentsPerPost": 50,
  "onlyNewPosts": false,
  "stateStoreName": "substack-scraper-state",
  "resetMonitoringState": false,
  "maxPagesPerPublication": 400,
  "requestDelaySecs": 0.8,
  "maxRetries": 4,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `posts` (type: `string`):

All dataset items (posts, publication profiles, comments).

## `publications` (type: `string`):

Publication profile items.

## `runSummary` (type: `string`):

Per-publication status, counts and error codes.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "noahpinion",
        "https://www.astralcodexten.com"
    ],
    "maxPostsPerPublication": 20,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("signalwharf/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publications": [
        "noahpinion",
        "https://www.astralcodexten.com",
    ],
    "maxPostsPerPublication": 20,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("signalwharf/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "noahpinion",
    "https://www.astralcodexten.com"
  ],
  "maxPostsPerPublication": 20,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call signalwharf/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,signalwharf/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Hr0ZqDo3W3bEvxkFV/builds/FKjoTVS3z3v1IusqI/openapi.json