# Substack Scraper - Posts, Subscribers & Paid Plan Prices (`cirkit/substack-newsletter-scraper`) Actor

Scrape any Substack newsletter archive: post title, subtitle, date, full text, paywall state, likes, restacks and comment counts, plus the publication's subscriber size, paid plan prices and bestseller tier. Custom domains supported. No login, one flat price.

- **URL**: https://apify.com/cirkit/substack-newsletter-scraper.md
- **Developed by:** [Crikit](https://apify.com/cirkit) (community)
- **Categories:** News, Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.70 / 1,000 post rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

Scrape any Substack newsletter and get its full post archive plus the numbers that describe the newsletter itself: how many subscribers it has, what it charges for a paid subscription, whether it is a bestseller, and how each post actually performed. Point it at a publication URL, a custom domain or a bare handle and it returns clean, structured rows ready for a spreadsheet, a database or an LLM pipeline.

No login, no cookies, no API key. One flat price per post returned.

### What is Substack Scraper?

Substack Scraper is an Apify Actor that turns a Substack publication into structured data. Give it `https://sherwood.substack.com`, `www.slowboring.com` or just `sherwood`, and every post in the archive comes back as a row with its title, subtitle, publish date, word count, paywall state, likes, restacks and comment count, alongside the publication's subscriber size and paid plan prices.

- **Walk the complete archive, not the first page.** Substack serves a short first page (23 rows) and then full 50-row pages after it. Scrapers that stop on a short page truncate an archive of hundreds down to 23 posts. This one advances by the exact row count Substack returned and stops only when a page comes back genuinely empty, so a 900-post archive returns 900 posts.
- **Get newsletter economics, not just post metadata.** Every row carries the publication's subscriber count, its monthly, annual and founding plan prices in dollars, its bestseller tier and its paid-subscriber band. That is the data you need to rank newsletters, size a sponsorship, or research a market.
- **Search inside a publication before you pay for it.** A keyword is passed to Substack's own archive search, so only matching posts are fetched and billed. Twenty posts about one topic costs twenty rows, not the whole archive.
- **Custom domains work.** `www.thefp.com` and `www.slowboring.com` are Substack publications on their own domains and both are scraped exactly like a `*.substack.com` host.
- **Filter before you are charged.** Post type, free versus paywalled and a publish-date window are applied before rows are written, so you only pay for posts you actually asked for.
- **Full article text on demand.** Turn on the body option to get each post's complete text, as plain text or as HTML with links and images intact.

### What data does Substack Scraper extract?

Every row is one post, stamped with its publication's own stats.

#### Post fields

| Field | Description |
|---|---|
| `title`, `subtitle`, `description` | Post headline, deck and meta description |
| `postUrl`, `slug`, `postId` | Canonical URL (custom domain aware), slug and numeric id |
| `postDate` | Publish timestamp, ISO 8601 |
| `postType` | `newsletter`, `podcast`, `thread` or `video` |
| `audience`, `isPaywalled`, `isFreeToRead` | Raw Substack audience value plus plain booleans |
| `wordCount` | Word count as Substack counts it |
| `reactionCount`, `reactions` | Total likes plus the emoji breakdown |
| `restackCount` | Times the post was restacked on Substack Notes |
| `commentCount` | Comment count |
| `authorName`, `authorHandle`, `authors` | Bylines, including multi-author posts, with profile URLs |
| `excerpt` | Substack's own truncated preview text |
| `bodyText`, `bodyHtml` | Full article text and HTML, when the body options are on |
| `coverImageUrl` | Post cover image |
| `podcastUrl`, `podcastDurationSeconds`, `hasVoiceover` | Audio fields for podcast posts and voiceovers |
| `sectionName`, `language`, `isGeoblocked` | Section, language and geoblock flag |

#### Newsletter fields

| Field | Description |
|---|---|
| `publicationName`, `publicationUrl`, `publicationHost` | Newsletter identity, including the custom domain |
| `publicationSubdomain`, `publicationCustomDomain`, `publicationId` | Both hostnames and the numeric publication id |
| `subscriberCount`, `subscriberCountLabel` | Exact subscriber count where Substack publishes it, plus the displayed band such as `200K+` |
| `paidSubscriberLabel` | Substack's own paid-subscriber band, such as `Tens of thousands of paid subscribers` |
| `monthlyPriceUsd`, `annualPriceUsd`, `foundingPriceUsd` | Live paid plan prices in dollars |
| `acceptsPayments`, `isInviteOnly` | Whether the newsletter sells subscriptions and whether it is invite only |
| `bestsellerTier` | Substack's bestseller badge tier |
| `publicationAuthorName`, `publicationAuthorHandle` | The newsletter's owner |
| `publicationLaunchedAt`, `publicationFirstPostAt` | When the newsletter was created and first published |
| `hasPodcast`, `publicationLanguage`, `publicationLogoUrl`, `publicationDescription` | Podcast flag, language, logo and tagline |
| `scrapedAt`, `sourceUrl`, `resultPosition`, `sortedBy`, `searchQuery` | Provenance for every row |

### How to scrape Substack posts

1. Open the Actor and paste one or more publications into **Publications**. A URL (`https://sherwood.substack.com`), a custom domain (`www.slowboring.com`), a handle (`sherwood`) or any post URL all work.
2. Set **Max results**. Use `0` to walk every publication's entire archive.
3. Optional: type a keyword into **Search within the publication** to fetch only matching posts.
4. Optional: choose **Sort by** `Top posts first` to get the publication's best performing posts instead of the newest.
5. Optional: narrow by **Post type**, **Paywall**, **Published after** and **Published before**.
6. Optional: turn on **Include full post text** or **Include full post HTML** for complete articles.
7. Click **Start** and export the results as JSON, CSV or Excel, or pull them from the API.

### Input example

```json
{
  "publications": ["www.slowboring.com", "sherwood"],
  "maxItems": 500,
  "maxPostsPerPublication": 250,
  "sortBy": "new",
  "postType": "newsletter",
  "audience": "free",
  "publishedAfter": "2026-01-01",
  "includeBodyText": true,
  "includePublicationStats": true
}
```

### Output example

```json
{
  "postId": 214556400,
  "postUrl": "https://www.slowboring.com/p/the-case-for-more-housing",
  "title": "The case for more housing",
  "subtitle": "Supply matters more than the discourse admits",
  "postDate": "2026-09-10T10:02:11.000Z",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "isFreeToRead": true,
  "wordCount": 2761,
  "reactionCount": 412,
  "reactions": { "❤": 412 },
  "restackCount": 37,
  "commentCount": 289,
  "authorName": "Matthew Yglesias",
  "authorHandle": "matthewyglesias",
  "publicationName": "Slow Boring",
  "publicationHost": "www.slowboring.com",
  "subscriberCount": 200000,
  "subscriberCountLabel": "200K+",
  "paidSubscriberLabel": "Tens of thousands of paid subscribers",
  "monthlyPriceUsd": 9,
  "annualPriceUsd": 80,
  "foundingPriceUsd": 500,
  "bestsellerTier": 10000,
  "hasPodcast": true,
  "publicationLaunchedAt": "2020-11-05T16:20:32.177Z",
  "scrapedAt": "2026-09-10T13:58:04.221Z"
}
```

### How much does it cost to scrape Substack?

One flat price of **$0.70 per 1,000 posts**. There is no start fee, no per-publication fee, no separate charge for the newsletter stats, and no surcharge for full article text.

| Posts returned | Cost |
|---|---|
| 100 | $0.07 |
| 1,000 | $0.70 |
| 10,000 | $7.00 |
| 100,000 | $70.00 |

Posts removed by a filter are never charged, and neither is a publication that turns out to be unreachable.

Two things to compare against when you price this out. Most Substack scrapers charge more per post the moment you ask for the article body, typically around twice the metadata rate, and several add a per-run start fee on top. Here the body is included in the same flat $0.70 per 1,000, and there is no start fee, so a 5,000-post run with full text costs $3.50 and nothing else.

### What can you use Substack data for?

- **Find newsletters to sponsor.** Rank publications in a niche by subscriber band, paid tier and posting cadence, then read the engagement on their recent posts.
- **Track a competitor's newsletter.** Watch their publishing frequency, which posts they put behind the paywall, and what their audience actually reacts to.
- **Research newsletter pricing.** Pull monthly, annual and founding plan prices across a whole category to see where your own price should sit.
- **Build a content dataset for an LLM.** Export full article text with the body option and feed a retrieval index or a fine-tuning corpus.
- **Monitor a topic across publications.** Run a keyword search over dozens of newsletters and get every post that mentions it.
- **Analyse what performs.** Compare likes, restacks and comments against word count, post type and paywall state.
- **Source writers and experts.** Author names, handles and profile URLs come with every post.

### Tips and limits

- Substack caps one archive page at 50 posts. The Actor pages through automatically, so this only affects speed, not completeness.
- The archive's first page returns fewer rows than the ones after it. That is normal Substack behaviour and is handled; do not read a 23-row first page as the end of an archive.
- `subscriberCount` is only populated when a publication chooses to display an exact number. `subscriberCountLabel` carries the band Substack shows publicly, and it is far more widely available.
- Paywalled posts return their title, metrics and preview text. Their full body is not readable without a paid subscription, so `bodyText` is the public portion.
- `substack.com/@someone` is a reader profile, not a publication. Use the publication's own host instead.
- Turning on full bodies adds one request per post, so a large archive takes noticeably longer.

### Integrations

Connect Substack Scraper to Google Sheets, Airtable, Slack, Zapier, Make, GitHub or almost any HTTP API through Apify integrations, or trigger runs and read results from the Apify API and the JavaScript and Python clients. Schedule it daily to keep a newsletter dataset current.

### Related Actors

- Google News Scraper for headlines across publishers
- YouTube Video Search Scraper for video content in the same niche
- Bluesky Profile and Posts Scraper for social conversation around a newsletter
- Telegram Channel Posts Scraper for channel-based publishing

### Substack Scraper FAQ

#### Is scraping Substack legal?

Substack Scraper reads only pages and API responses that Substack serves publicly to any visitor, with no login and no cookies. Collecting public data is broadly lawful, and courts in the United States have repeatedly declined to treat scraping public web pages as unauthorised access. You are responsible for how you use what you collect: respect copyright in the article text, and if you handle personal data such as author names, comply with GDPR, CCPA and the other rules that apply to you. If in doubt, take legal advice for your specific use case.

#### Do I need a Substack account or a paid subscription?

No. The Actor never logs in and never asks for a cookie. It sees exactly what an anonymous visitor sees, which includes every post's metadata and metrics, and the public part of paywalled posts.

#### Can it scrape the full text of paywalled posts?

No. Paid-only content is not served to anonymous readers, so the Actor returns the post's title, date, metrics and public preview and leaves the paid body out. Everything a logged-out reader can see is returned.

#### How many posts can it return from one publication?

The whole archive. Set **Max results** to `0` and the Actor pages through every post the publication has ever published, in order, without gaps.

#### Does it work on newsletters with their own domain?

Yes. Publications on custom domains such as `www.thefp.com` and `www.slowboring.com` are handled identically to `*.substack.com` hosts, and both hostnames are reported on every row.

#### How do I get only posts about one topic?

Put a keyword in **Search within the publication**. Substack filters the archive server side, so unmatched posts are never fetched and never charged.

#### How much does it cost?

$0.70 per 1,000 posts, flat, with no start fee and no extra charge for full article text. A 250-post archive costs about 18 cents.

### Support

Found a publication that does not scrape correctly, or a field that comes back empty when it should not? Open an issue on the Actor's Issues tab with the publication URL and the input you used, and it will be looked at.

# Actor input Schema

## `publications` (type: `array`):

Substack publications to scrape. Accepts a publication URL (https://sherwood.substack.com), a custom domain (www.slowboring.com), a bare handle (sherwood), or any post URL, in which case its publication is scraped.

## `maxItems` (type: `integer`):

Hard cap on rows returned across every publication in this run. Set 0 for no limit and the full archive is walked.

## `maxPostsPerPublication` (type: `integer`):

Cap per publication, useful when several publications share one Max results budget. Set 0 for no per-publication cap.

## `sortBy` (type: `string`):

Order Substack returns the archive in. Newest walks the archive back in time; Top returns the publication's best performing posts first.

## `searchQuery` (type: `string`):

Optional keyword. Substack filters the archive server side, so only matching posts are fetched and charged. Leave empty to walk the whole archive.

## `postType` (type: `string`):

Keep only one kind of post. Newsletter is a written article, podcast is an audio episode, thread is a discussion post and video is a video post.

## `audience` (type: `string`):

Keep only free posts (readable by anyone) or only paywalled posts (paid or founding subscribers only).

## `publishedAfter` (type: `string`):

Keep only posts published on or after this date, as 2026-01-31 or a full ISO timestamp.

## `publishedBefore` (type: `string`):

Keep only posts published on or before this date, as 2026-06-30 or a full ISO timestamp.

## `includeBodyText` (type: `boolean`):

Fetch each post's full article text as plain text. Costs one extra request per post, so the run is slower. Off by default because the excerpt is enough for most jobs.

## `includeBodyHtml` (type: `boolean`):

Fetch each post's full article as HTML, keeping links, images and formatting. Costs one extra request per post and makes rows much larger.

## `includePublicationStats` (type: `boolean`):

Stamp every row with the publication's subscriber size, monthly and annual paid plan prices, bestseller tier, launch date and podcast flag. One extra request per publication, not per post.

## `proxyConfiguration` (type: `object`):

Proxy settings. The Apify default works: Substack answers datacenter IPs normally and needs no residential traffic.

## Actor input object example

```json
{
  "publications": [
    "https://sherwood.substack.com"
  ],
  "maxItems": 25,
  "maxPostsPerPublication": 0,
  "sortBy": "new",
  "postType": "any",
  "audience": "any",
  "includeBodyText": false,
  "includeBodyHtml": false,
  "includePublicationStats": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `posts` (type: `string`):

Every post row from this run, as the Overview table: title, date, type, paywall state, likes, restacks, comments and the publication it came from.

## `newsletters` (type: `string`):

The same rows through the Newsletters view: publication name, subscriber size, paid plan prices, bestseller tier and launch date.

## `postsCsv` (type: `string`):

All rows and all fields as CSV, ready for Excel or Google Sheets.

## `consoleRun` (type: `string`):

The run's log, input and output in the Apify Console.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "https://sherwood.substack.com"
    ],
    "maxItems": 25,
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("cirkit/substack-newsletter-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publications": ["https://sherwood.substack.com"],
    "maxItems": 25,
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("cirkit/substack-newsletter-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "https://sherwood.substack.com"
  ],
  "maxItems": 25,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call cirkit/substack-newsletter-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cirkit/substack-newsletter-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/xHT8gGpfFCqUfEv9T/builds/KPUZlqFFgz5c9iKsB/openapi.json