# Substack Posts Scraper Pro (`red.cars/substack-posts-scraper-pro`) Actor

Extract posts from any Substack publication (custom domain or substack.com) as clean JSON: title, subtitle, publish date, paywall status, reaction and comment counts, word count and cover image, plus a full-body-text mode. Server-side search with exact pagination. No API key needed.

- **URL**: https://apify.com/red.cars/substack-posts-scraper-pro.md
- **Developed by:** [AutomateLab](https://apify.com/red.cars) (community)
- **Categories:** News, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 post rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Posts Scraper Pro

Extract posts from any Substack publication as clean JSON: title, subtitle, publish date, paywall status, reaction and comment counts, word count and cover image. Works with custom-domain publications (newsletter.pragmaticengineer.com, noahpinion.blog) and substack.com publications alike. Optionally fetch complete body text per post. No API key needed.

### Why this actor

The most-used Substack actor on the Store has no public reviews yet, and the best-rated one sits at 2.6 stars. This actor covers what buyers actually ask for:

- **Search within a publication** with server-side filtering, so non-matching posts are never returned or billed
- **Custom-domain support**: works on any Substack publication, whether it lives on substack.com or its own domain
- **Engagement on every row**: reaction totals, comment counts and word counts on archive rows
- **Exact pagination**: results stop precisely at your max count
- **Full body text mode** for single posts, with paywall status flagged

### Input

| Field | Type | Description |
| --- | --- | --- |
| `mode` | select | `posts` (archive metadata rows, default) or `post` (full text for specific slugs) |
| `publicationUrl` | text | Publication address, e.g. `https://newsletter.pragmaticengineer.com` or `https://heathercoxrichardson.substack.com` |
| `searchQuery` | text | Optional keyword filter applied within the publication (posts mode) |
| `postSlugs` | list | Post slugs for `post` mode, e.g. `the-pulse-new-trend-of-building-internal` |
| `maxItems` | number | Hard cap on dataset items (1-1000, default 10) |

### Output

One JSON row per post:

| Field | Description |
| --- | --- |
| `type` | `post` (archive) or `post_full` (single mode) |
| `publication` | Publication host |
| `postId` | Numeric post ID |
| `title`, `subtitle`, `slug`, `url` | Post identity and link |
| `publishDate` | ISO publish date |
| `audience` | `everyone` or `only_paid` |
| `isPaid` | True when the post is for paid subscribers |
| `isPaywalled` | Single mode only: full body is paywalled |
| `reactionsTotal`, `commentCount`, `wordCount` | Engagement and size metrics |
| `section`, `coverImage`, `description` | Section name, cover image URL, text preview |
| `bodyText` | Single mode only: complete body text as plain text |
| `podcastDuration` | Podcast episode length in seconds, when present |

### Pricing

Pay per event:

- **$0.005 per actor start**
- **$0.002 per post row** saved to the dataset

Filtering happens before rows are saved, so you are never charged for rows you did not ask for. A 10-post test run costs about $0.025.

### Use cases

- **Content intelligence**: track publishing cadence, topics and engagement for any publication or competitor newsletter
- **Creator research**: measure reaction and comment volume across a publication's archive before sponsoring or partnering
- **LLM training data**: clean full-text posts with titles, dates and metadata
- **Lead generation**: find high-engagement publications in a niche, with contact-ready publication URLs

### FAQ

**Does it work with custom-domain publications?**
Yes. Pass the domain as the publication URL. A small number of custom domains do not answer archive requests; the actor reports that clearly instead of returning empty results.

**Can I get the full text of every post in a publication?**
Run posts mode first to get slugs, then re-run in post mode with the slugs you need.

**Is scraping Substack allowed?**
Only publicly accessible content is returned. Paid posts are flagged as paid and are not unlocked.

**Does it need cookies, keys or a logged-in account?**
No. It runs keyless.

# Actor input Schema

## `mode` (type: `string`):

posts: one metadata row per post from a publication's archive. post: fetch full records (complete body text) for specific slugs.

## `publicationUrl` (type: `string`):

The publication's address, e.g. https://newsletter.pragmaticengineer.com or https://heathercoxrichardson.substack.com. Required in posts mode; custom domains supported.

## `searchQuery` (type: `string`):

Optional keyword filter applied server-side before rows are fetched, so non-matching posts are never returned. Empty means all posts.

## `postSlugs` (type: `array`):

Post slugs for single mode, e.g. the-pulse-new-trend-of-building-internal. Ignored in posts mode.

## `maxItems` (type: `integer`):

Hard cap on dataset items saved. Pagination stops exactly at this count; extra rows are never fetched or billed.

## Actor input object example

```json
{
  "mode": "posts",
  "publicationUrl": "https://newsletter.pragmaticengineer.com",
  "postSlugs": [
    "the-pulse-new-trend-of-building-internal"
  ],
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

URL to the dataset items: one clean JSON row per post, with title, subtitle, publish date, paywall status, reaction and comment counts, word count and cover image; single mode adds the full body text.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publicationUrl": "https://newsletter.pragmaticengineer.com",
    "searchQuery": "",
    "postSlugs": [
        "the-pulse-new-trend-of-building-internal"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("red.cars/substack-posts-scraper-pro").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publicationUrl": "https://newsletter.pragmaticengineer.com",
    "searchQuery": "",
    "postSlugs": ["the-pulse-new-trend-of-building-internal"],
}

# Run the Actor and wait for it to finish
run = client.actor("red.cars/substack-posts-scraper-pro").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publicationUrl": "https://newsletter.pragmaticengineer.com",
  "searchQuery": "",
  "postSlugs": [
    "the-pulse-new-trend-of-building-internal"
  ]
}' |
apify call red.cars/substack-posts-scraper-pro --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,red.cars/substack-posts-scraper-pro"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/gZwE5Wv6ZmtmbCXQ0/builds/GWWtdbqhuLUr6fuVU/openapi.json