# Substack Scraper — Posts, Archives & Author Info, $0.9/1k (`leadharbor/substack-scraper`) Actor

Scrape Substack newsletters: post metadata, titles, dates, likes, author info from public archives. $0.0009/post, no start fee, no login.

- **URL**: https://apify.com/leadharbor/substack-scraper.md
- **Developed by:** [Mohammed Samaan](https://apify.com/leadharbor) (community)
- **Categories:** Social media, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.90 / 1,000 post scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper — Posts, Archives & Author Info

Give it Substack publications. Each public post comes back as one row: **title, subtitle, date, audience (free or paid), reactions, comment count, authors, cover image and word count**. Built for newsletter research, competitor tracking, content calendars and creator lists.

It reads the publication's public archive JSON endpoint (`/api/v1/archive`) with plain HTTP. No browser, no proxy, no login, no cookies.

**Price: $0.9 per 1,000 posts** ($0.0009 each). **No start fee.** Publications that cannot be read (not found, not a Substack, blocked, timeout) return one `error` row and are **not charged**.

### What you get per post

| Field | Meaning |
|---|---|
| `publication` | Host that was read (e.g. `newsletter.pragmaticengineer.com`, or `name.substack.com` for a bare name) |
| `id`, `slug`, `url` | Post id, URL slug, canonical post URL |
| `title`, `subtitle` | As shown in the archive |
| `post_date` | Publish time, ISO 8601 |
| `type` | Post type reported by Substack (e.g. `newsletter`, `podcast`, `thread`) |
| `audience` | `everyone` (free) or `only_paid` (paywalled) and other values Substack reports, passed through unchanged |
| `reactions` | Reaction (like) count |
| `comments` | Comment count |
| `authors` | List of author names |
| `cover_image` | Cover image URL, or null |
| `wordcount` | Word count if Substack reports one, else null |
| `error` | null on a post row; the reason on an error row |

### What it does NOT do

- **No post body text.** Paywalled posts are listed (title, date, audience, counts) but their content is never fetched, and free-post bodies are not included either. This Actor returns archive metadata only.
- **No subscriber counts, no subscriber lists, no emails, no comments text.** Only what the public archive endpoint returns.
- **No login and no private publications.** If a publication is private or hides its archive, you get an error row.
- Posts are returned newest first; there is no date filter or keyword search.
- Substack may rate-limit or block requests. If that happens you get an error row for that publication (not charged). Posts already returned stay in the dataset.

### Input

| Field | What it does |
|---|---|
| `publications` | One per line. A custom domain (`newsletter.pragmaticengineer.com`), a Substack name (`astralcodexten` becomes `astralcodexten.substack.com`) or a URL (only the host is used). Duplicates are removed. Up to 100 per run. |
| `maxPosts` | Max posts per publication, 1–5,000 (default 50). Newest first. |

If the same publication is given under two addresses (for example its `.substack.com` name and its custom domain), the second is skipped with an uncharged error row so you are not billed twice.

### Example

Input:

```json
{ "publications": ["newsletter.pragmaticengineer.com"], "maxPosts": 50 }
```

Output row:

```json
{
  "publication": "newsletter.pragmaticengineer.com",
  "id": 218367831,
  "title": "The Pulse: Firebase’s global outage & poor response",
  "subtitle": "Also: OpenAI’s platform play that has similarities to AWS, more data on companies moving to open models, and more",
  "slug": "the-pulse-firebases-global-outage",
  "url": "https://newsletter.pragmaticengineer.com/p/the-pulse-firebases-global-outage",
  "post_date": "2026-10-01T16:43:19.985Z",
  "type": "newsletter",
  "audience": "only_paid",
  "reactions": 53,
  "comments": 3,
  "authors": ["Gergely Orosz"],
  "cover_image": "https://substackcdn.com/image/fetch/...",
  "wordcount": 2889,
  "error": null
}
```

Error row (not charged):

```json
{ "publication": "nonexistent-pub-zzqx9.substack.com", "error": "Publication not found (HTTP 404)." }
```

### Use it from code

```bash
curl -X POST "https://api.apify.com/v2/acts/leadharbor~substack-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"publications":["astralcodexten"],"maxPosts":20}'
```

Python, with the [apify-client](https://pypi.org/project/apify-client/) package:

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")
run = client.actor("leadharbor/substack-scraper").call(
    run_input={"publications": ["astralcodexten"], "maxPosts": 20}
)
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row["post_date"], row["title"], row["reactions"])
```

### Use it from an AI agent

This Actor is pay-per-event and works with the [Apify MCP server](https://mcp.apify.com), so AI agents can call it directly with a list of publications.

### FAQ

**Am I charged for publications that fail?**
No. Only post rows are charged. Error rows are free.

**Why is `wordcount` sometimes null?**
Substack does not report it for every post; the field is passed through only when present.

**Can I get the full text of a post?**
No. This Actor does not fetch post bodies, paywalled or free.

**Are the counts live?**
They are what the archive endpoint reports at the moment of the run.

**How fast is it?**
Posts are fetched 12 per request, one publication after another, so 1,000 posts is roughly 80 requests.

### Related actors

- [Reddit Brand & Keyword Monitor](https://apify.com/leadharbor/reddit-brand-keyword-monitor)
- [YouTube Video Scraper](https://apify.com/leadharbor/youtube-video-scraper)
- [Website Contact Extractor](https://apify.com/leadharbor/website-contact-extractor)

### Changelog

- **1.0.0 (6 Oct 2026):** first release. Archive metadata only, pay-per-event $0.0009 per post.

# Actor input Schema

## `publications` (type: `array`):

One publication per line: a custom domain (newsletter.pragmaticengineer.com), a Substack name (astralcodexten) or a URL (https://example.substack.com/p/some-post). Duplicates are removed. Up to 100 per run.

## `maxPosts` (type: `integer`):

Newest posts are returned first. Hard cap 5,000 per publication.

## Actor input object example

```json
{
  "publications": [
    "newsletter.pragmaticengineer.com",
    "astralcodexten"
  ],
  "maxPosts": 50
}
```

# Actor output Schema

## `results` (type: `string`):

One row per post: title, subtitle, date, audience, reactions, comments, authors, cover image, word count. Publications that could not be read come back as an error row.

## `resultsUi` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "newsletter.pragmaticengineer.com",
        "astralcodexten"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("leadharbor/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "publications": [
        "newsletter.pragmaticengineer.com",
        "astralcodexten",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("leadharbor/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "newsletter.pragmaticengineer.com",
    "astralcodexten"
  ]
}' |
apify call leadharbor/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,leadharbor/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/umbcYpR2dUzHJlYQ3/builds/Dm7h6Q41qaOOl6Qkg/openapi.json