# Substack Scraper (`yegnata/substack-scraper`) Actor

Scrape posts from any Substack newsletter, including custom domains: title, date, authors, likes, comments, word count and the full text of free posts. Pay only per post.

- **URL**: https://apify.com/yegnata/substack-scraper.md
- **Developed by:** [eshetu elias](https://apify.com/yegnata) (community)
- **Categories:** News, AI
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 posts

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper

Scrape posts from **any Substack newsletter**, including ones on custom domains. Get the title, subtitle, date, authors, likes, comments, restacks, word count, cover image and the **full text of free posts**, newest first, back to the very first issue.

### What you get

| publication | date | title | authors | likes | comments | words | paid |
|---|---|---|---|---|---|---|---|
| astralcodexten | 2026-10-30 | Your Book Review: The History And Adventures Of An Atom | Scott Alexander | 77 | 48 | 5,789 | no |
| astralcodexten | 2026-10-29 | Links For October 2026 | Scott Alexander | 163 | 423 | 4,112 | no |
| astralcodexten | 2026-10-28 | My California Ballot, 2026 | Scott Alexander | 151 | 573 | 5,439 | no |

Every post also has `postId`, `subtitle`, `url`, `type` (newsletter, podcast, thread), `audience`, `isPaywalled`, `restacks`, `coverImage`, `podcastUrl`, `teaser`, `content` (free posts) and `scrapedAt`.

### Use cases

- **Content and competitor research:** see what top newsletters in your niche publish and what gets the most likes and comments.
- **AI and search:** build a clean text corpus from free posts for RAG, summaries or topic tracking.
- **Creator analytics:** track publishing frequency, length and engagement over time.
- **Monitoring:** get every new post from the newsletters you follow in one feed.

### How much does it cost?

You pay **$3.00 per 1,000 posts**, nothing else. Set *Maximum results* to cap your spend; the run stops when it reaches your limit. Publications that don't exist cost nothing.

### How to use it

1. Add publications to **Publications**: `astralcodexten`, `astralcodexten.substack.com`, or a custom domain like `https://www.lennysnewsletter.com`. A link to any post works too.
2. Optionally set **Max posts per publication**, **Posted within (days)** or **Free posts only**.
3. Click **Start** and download as JSON, CSV or Excel, or pull results by API.

### Input example

```json
{
  "publications": ["astralcodexten", "https://www.lennysnewsletter.com"],
  "maxPostsPerPublication": 100,
  "postedWithinDays": 90,
  "includeContent": true,
  "maxItems": 500
}
```

### Paid posts

Paid (subscriber-only) posts are included with their metadata and Substack's own short teaser, and are marked `isPaywalled: true`. Their full text is **never** returned; this Actor doesn't get around paywalls. Turn on **Free posts only** to skip them.

### Speed

About 600 posts a minute without full text, and about 50 a minute with full text (one extra request per free post).

### FAQ

**Is this legal?** The Actor reads each publication's public archive, the same data its /archive page shows to every visitor, and full text only for posts the author made free.

**A publication returned nothing.** It's listed under `FAILED_PUBLICATIONS` in the run's key-value store. Check that the address opens a Substack in your browser.

**Something broke.** Open an issue on the Issues tab; fixes usually ship within 24 hours.

# Actor input Schema

## `publications` (type: `array`):

Substack names or links: "astralcodexten", "astralcodexten.substack.com", or a custom domain like "https://www.lennysnewsletter.com". Any post link from the publication works too.

## `maxPostsPerPublication` (type: `integer`):

Newest posts first. Leave empty to go back as far as Maximum results allows.

## `postedWithinDays` (type: `integer`):

Keep only posts from the last N days. Leave empty for all.

## `includeContent` (type: `boolean`):

Add the full plain-text body of free posts. Paid posts always get only Substack's short teaser.

## `freeOnly` (type: `boolean`):

Skip paid (subscriber-only) posts entirely.

## `maxItems` (type: `integer`):

Stop after this many posts in total. You are charged per post, so this caps your cost.

## `proxyConfiguration` (type: `object`):

Not needed for normal use.

## Actor input object example

```json
{
  "publications": [
    "astralcodexten",
    "https://www.lennysnewsletter.com"
  ],
  "includeContent": true,
  "freeOnly": false,
  "maxItems": 100,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `posts` (type: `string`):

Every post saved in this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "astralcodexten",
        "https://www.lennysnewsletter.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("yegnata/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "publications": [
        "astralcodexten",
        "https://www.lennysnewsletter.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("yegnata/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "astralcodexten",
    "https://www.lennysnewsletter.com"
  ]
}' |
apify call yegnata/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,yegnata/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RzZHAdXpCPJNtqxn7/builds/Y68ldgvWveo0nbXmm/openapi.json