# Substack Posts Scraper (`chorelet/substack-posts-scraper`) Actor

Scrape posts from any Substack publication, including custom domains: title, date, likes, comments, restacks, authors, paid/free flag, optional full text — newest first, with date filter and CSV/JSON export. No login.

- **URL**: https://apify.com/chorelet/substack-posts-scraper.md
- **Developed by:** [Ilia Muravev](https://apify.com/chorelet) (community)
- **Categories:** News, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 posts

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Posts Scraper

Get the posts of any Substack publication — on `*.substack.com` or a custom domain — as JSON, CSV or Excel, or straight into your own tools via API. No login, no cookies: it reads the same public archive that powers the publication's own "Archive" page.

Paste publication URLs, subdomains or author handles, set how many posts you need (or a date like "90 days"), optionally turn on **Include post content** to get the text of every post.

### What you get

| Field | Description |
|---|---|
| `publication`, `publicationName`, `publicationAuthor` | Host (e.g. `www.lennysnewsletter.com`), name and author of the publication |
| `id`, `slug`, `url`, `title`, `subtitle`, `description` | The post and its listing summary |
| `date`, `type` | Publication time (UTC); `newsletter`, `podcast`, `thread` or `video` |
| `audience`, `isPaid`, `wordCount` | Free vs paywalled, length |
| `reactions`, `reactionCount`, `commentCount`, `restacks` | Engagement as shown publicly |
| `authors`, `tags`, `section`, `coverImage` | Bylines with handles, tags, section |
| `podcastUrl`, `podcastDuration` | For podcast posts |
| `previewText` | First lines of the post as Substack lists them |
| `bodyText`, `bodyHtml` | Only with **Include post content**: full text of free posts; for paid posts the public preview an anonymous reader sees |

Posts come **newest first** per publication. A per-publication summary (name, author, posts fetched, errors) is saved to the key-value store as `SUMMARY`.

### Input

- **Publications** — any mix of `lenny`, `astralcodexten.substack.com`, `https://www.astralcodexten.com`, a post URL, or an author handle `@lenny` / `https://substack.com/@lenny` (resolved to the author's primary publication).
- **Max posts per publication** — newest first, read in pages of 25.
- **Only posts newer than** — `2026-01-31` or `7 days` / `2 weeks` / `3 months` / `1 year`. Scraping stops at the first older post, so a daily run with `1 day` costs almost nothing.
- **Include post content** — one extra request per post, charged as a separate event.
- **Proxy** — off by default.

### Limits and notes

- Paywalled posts return exactly what Substack shows to non-subscribers: metadata plus the public preview. The Actor cannot and does not bypass paywalls.
- Comments and Notes are not included.
- A subdomain that was moved to a custom domain (e.g. `lenny.substack.com`) is followed automatically. If a subdomain turns out to be a personal profile rather than a publication, the run says so and continues with the rest.
- Public data only; the Actor stores nothing beyond the dataset of your run.

### Pricing

Pay per post — plus a small per-post fee when you ask for the full content. Runs that return zero posts cost nothing.

### Typical uses

Newsletter monitoring and competitive research, feeding posts into an LLM for summaries or topic tracking, building a reading digest of many publications, archiving your own publication, engagement benchmarks (likes, comments, restacks per post) over time.

# Actor input Schema

## `publications` (type: `array`):

Substack subdomains or URLs, including custom domains: `lennysnewsletter`, `astralcodexten.substack.com`, `https://www.astralcodexten.com`, or any post URL of the publication.

## `maxPostsPerPublication` (type: `integer`):

Newest posts first. The archive is read in pages of 25.

## `postedAfter` (type: `string`):

Absolute date `2026-01-31` or relative `7 days`, `2 weeks`, `3 months`, `1 year`. Scraping stops at the first older post. Leave empty for no date limit.

## `includeContent` (type: `boolean`):

Fetch the body of every post (`bodyText` and `bodyHtml`). Free posts come in full; for paid posts you get the public preview only, exactly as an anonymous reader sees it. One extra request per post, charged as a separate event.

## `proxyConfiguration` (type: `object`):

Off by default — Substack's public API answers without a proxy. Enable Apify datacenter proxies only for very large runs.

## Actor input object example

```json
{
  "publications": [
    "lennysnewsletter",
    "https://www.astralcodexten.com"
  ],
  "maxPostsPerPublication": 50,
  "postedAfter": "90 days",
  "includeContent": false,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `posts` (type: `string`):

All scraped posts — items of the default dataset. Use ?format=csv or xlsx on this URL for spreadsheets.

## `summary` (type: `string`):

Publication name, subscriber phrase and number of posts fetched per publication, plus errors.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "lennysnewsletter",
        "https://www.astralcodexten.com"
    ],
    "maxPostsPerPublication": 50,
    "postedAfter": "90 days",
    "includeContent": false,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("chorelet/substack-posts-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publications": [
        "lennysnewsletter",
        "https://www.astralcodexten.com",
    ],
    "maxPostsPerPublication": 50,
    "postedAfter": "90 days",
    "includeContent": False,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("chorelet/substack-posts-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "lennysnewsletter",
    "https://www.astralcodexten.com"
  ],
  "maxPostsPerPublication": 50,
  "postedAfter": "90 days",
  "includeContent": false,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call chorelet/substack-posts-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,chorelet/substack-posts-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/dSMs9GkOyUs5Gb9fn/builds/2juUfHxf8J5RgtY4A/openapi.json