# Substack Posts Scraper (`leorochasantos/substack-posts-scraper`) Actor

Scrape posts from any Substack publication (subdomain or custom domain): full HTML and plain text for public posts, paywall signal and teaser for paid posts. Flat, LLM-ready JSON. $0.0002 per post.

- **URL**: https://apify.com/leorochasantos/substack-posts-scraper.md
- **Developed by:** [Leonardo Santos](https://apify.com/leorochasantos) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.26 / 1,000 post results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Substack Posts Scraper

Fetch the posts of any Substack publication — a `.substack.com` subdomain or a
custom domain like `www.slowboring.com` — as clean, flat, LLM/agent-ready JSON.
Free posts come back with full HTML and plain text; paywalled posts come back
with the paywall signal plus whatever public teaser the publication provides.
One item per post, newest-first, ready to drop straight into a pipeline.

### Why this actor?

- 💵 **$0.0002 per post** — undercuts the field: the category leader charges a
  $0.005 start fee before you get a single item, and the next-cheapest credible
  competitor is still 50% more per item. No start fee here — you only pay for
  posts delivered.
- 🎯 **Reliability-first** — built and monitored on a first-party JSON API
  (`/api/v1/posts`), not brittle HTML scraping. Contract-checked by an
  automated watchdog with smoke tests on every run.
- 🌐 **Works with custom domains** — subdomains and mapped custom domains
  (`www.slowboring.com`, `astralcodexten.com`) are handled identically.
- 🔒 **No personal data** — public post content only. This actor never returns
  subscriber lists, author emails, or any contact data.
- ⏱️ **Incremental crawls built in** — `publishedAfter` stops pagination as
  soon as it reaches older posts, so scheduled re-runs only pay for new
  content.
- 🤖 **`body_text` ready for LLM pipelines** — tag-stripped, entity-decoded
  plain text alongside the raw `body_html`, so you can feed posts straight
  into an LLM without writing your own HTML cleaner.

### Input

```json
{
  "publications": ["astralcodexten", "www.slowboring.com"],
  "maxPostsPerPublication": 50,
  "publishedAfter": "2026-01-01",
  "includeBodyHtml": true
}
````

`publications` accepts a bare name (`astralcodexten` → `astralcodexten.substack.com`),
a subdomain, a custom domain, or a post/homepage URL — up to 50 publications
per run, deduplicated. See the input schema for `proxyConfiguration`.

### Output

One item per post:

```json
{
  "input": "astralcodexten",
  "publication_host": "astralcodexten.substack.com",
  "post_id": 148213567,
  "slug": "links-for-july",
  "url": "https://astralcodexten.substack.com/p/links-for-july",
  "title": "Links For July",
  "subtitle": "Highlights from the comments, some AI news, and more",
  "post_date": "2026-07-14T13:05:00.000Z",
  "type": "newsletter",
  "audience": "everyone",
  "is_paywalled": false,
  "body_html": "<p>Here's a bunch of stuff I found interesting this month...</p>",
  "body_text": "Here's a bunch of stuff I found interesting this month...",
  "word_count": 3120,
  "description": "Highlights from the comments, some AI news, and more",
  "cover_image": "https://substackcdn.com/image/fetch/astralcodexten/links-for-july-cover.jpg",
  "comment_count": 412,
  "reaction_count": 987,
  "error": null,
  "scraped_at": "2026-07-17T00:00:00.000Z"
}
```

A paywalled post comes back the same shape with `is_paywalled: true`,
`audience: "only_paid"`, `body_html: null`, and `body_text` set to whatever
teaser the publication exposes (may be empty — not every publication ships
one).

### Pricing

Pay per result: **one `post-result` event at $0.0002 per delivered post
item** — free and paywalled posts are both charged, since metadata plus the
paywall signal is a delivered result either way. Error items (bad input,
publication not found, no posts match) are always **free**. This is a **Pay
per event + usage** actor — you also pay Apify platform usage for the run's
compute, same as our other actors.

### Limitations

- Up to 50 publications per run, up to 2,000 posts per publication.
- **Paywalled post bodies are not included.** You get the paywall signal
  (`is_paywalled`, `audience`) and whatever public teaser the publication
  provides — never the paid content itself.
- No comments and no Notes in v1 — post metadata and content only.

### Use cases

- 🗞️ **Content monitoring** — track new posts from publications you follow or
  compete with, with `publishedAfter` for cheap incremental runs.
- 🤖 **LLM pipelines / RAG** — flat, pre-cleaned `body_text` for summarization,
  search indexing, or agent tool use.
- 📊 **Research & analysis** — post cadence, paywall mix, and engagement
  signals (`comment_count`, `reaction_count`) across a set of publications.

### FAQ

**Can I get the full text of paywalled posts?** No — this actor never
attempts to circumvent paywalls. Paid posts return the paywall signal and any
public teaser only.

**Does this work on custom domains, not just `*.substack.com`?** Yes, custom
domains are handled identically to subdomains — just pass the domain as the
publication.

**Do I pay for posts that fail?** No — error items (invalid input,
publication not found, no matching posts) are always free.

***

*This Actor is an independent tool, not affiliated with or endorsed by
Substack Inc.*

# Actor input Schema

## `publications` (type: `array`):

Substack publications to scrape — a bare name (astralcodexten), a subdomain (astralcodexten.substack.com), a custom domain (www.slowboring.com), or any post/homepage URL. One item per post. Up to 50 publications per run, deduplicated.

## `maxPostsPerPublication` (type: `integer`):

Newest-first cap per publication (default 50, max 2000).

## `publishedAfter` (type: `string`):

Optional ISO date (e.g. 2026-01-01). Only posts published after this date are returned; pagination stops early, so incremental runs stay cheap.

## `includeBodyHtml` (type: `boolean`):

Ship the full post HTML in body\_html (default true). Plain-text body\_text is always included. Turn off for slimmer datasets.

## `proxyConfiguration` (type: `object`):

Proxy to use for outgoing requests.

## Actor input object example

```json
{
  "publications": [
    "astralcodexten",
    "www.slowboring.com"
  ],
  "maxPostsPerPublication": 50,
  "includeBodyHtml": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "astralcodexten",
        "www.slowboring.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("leorochasantos/substack-posts-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "publications": [
        "astralcodexten",
        "www.slowboring.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("leorochasantos/substack-posts-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "astralcodexten",
    "www.slowboring.com"
  ]
}' |
apify call leorochasantos/substack-posts-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=leorochasantos/substack-posts-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Substack Posts Scraper",
        "description": "Scrape posts from any Substack publication (subdomain or custom domain): full HTML and plain text for public posts, paywall signal and teaser for paid posts. Flat, LLM-ready JSON. $0.0002 per post.",
        "version": "0.1",
        "x-build-id": "qbUtLRymUL7Xv8FQY"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/leorochasantos~substack-posts-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-leorochasantos-substack-posts-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/leorochasantos~substack-posts-scraper/runs": {
            "post": {
                "operationId": "runs-sync-leorochasantos-substack-posts-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/leorochasantos~substack-posts-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-leorochasantos-substack-posts-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "publications"
                ],
                "properties": {
                    "publications": {
                        "title": "Publications",
                        "type": "array",
                        "description": "Substack publications to scrape — a bare name (astralcodexten), a subdomain (astralcodexten.substack.com), a custom domain (www.slowboring.com), or any post/homepage URL. One item per post. Up to 50 publications per run, deduplicated.",
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxPostsPerPublication": {
                        "title": "Max posts per publication",
                        "minimum": 1,
                        "maximum": 2000,
                        "type": "integer",
                        "description": "Newest-first cap per publication (default 50, max 2000).",
                        "default": 50
                    },
                    "publishedAfter": {
                        "title": "Published after",
                        "type": "string",
                        "description": "Optional ISO date (e.g. 2026-01-01). Only posts published after this date are returned; pagination stops early, so incremental runs stay cheap."
                    },
                    "includeBodyHtml": {
                        "title": "Include body HTML",
                        "type": "boolean",
                        "description": "Ship the full post HTML in body_html (default true). Plain-text body_text is always included. Turn off for slimmer datasets.",
                        "default": true
                    },
                    "proxyConfiguration": {
                        "title": "Proxy configuration",
                        "type": "object",
                        "description": "Proxy to use for outgoing requests.",
                        "default": {
                            "useApifyProxy": true
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```