# Substack Scraper ($0.99/1K Results) (`scrapers-hub/substack-scraper`) Actor

\[𝘾𝙝𝙚𝙖𝙥𝙚𝙨𝙩] Substack scraper to extract newsletters, authors, posts, subscriber insights, and metadata from Substack ✍️📊 Ideal for content research, creator discovery, and newsletter analysis. Fast and scalable.

- **URL**: https://apify.com/scrapers-hub/substack-scraper.md
- **Developed by:** [Scrapers Hub](https://apify.com/scrapers-hub) (community)
- **Categories:** Developer tools, News, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### Substack Scraper ✍️

**Substack Scraper** collects newsletters, posts, and writer profiles from Substack and turns them into a clean, downloadable dataset — titles, full article text, authors, publication details, subscriber numbers, engagement counts, and reader comments. Whether you are researching a niche, finding writers to work with, or tracking what your competitors publish, this Substack Scraper gathers it all in one run.

No coding and no Substack account needed. Paste a link, press Run, and download the results as a spreadsheet or JSON file.

***

### 🚀 Why Choose This Substack Scraper?

| Feature | What It Means for You |
| --- | --- |
| ✅ **Three Kinds of Link** | Works as a **Substack search scraper**, a **Substack publication scraper**, and a **Substack profile scraper** — it recognises each link automatically |
| ✅ **Full Article Text** | The **Substack post scraper** brings back complete article bodies, not just headlines |
| ✅ **Audience Numbers** | See **Substack subscriber counts**, follower counts, and bestseller tiers for writers and publications |
| ✅ **Comments Included** | Optionally collect reader replies under each post |
| ✅ **Any Domain** | Handles both `something.substack.com` and newsletters on their own custom domains |
| ✅ **Spreadsheet-Ready** | Clear, well-labelled columns let you **export Substack data to CSV**, Excel, or JSON instantly |

***

### 🔑 Key Features of the Substack Scraper

- 🔎 **Search Any Topic** — paste a Substack search link and the **Substack search scraper** collects every matching result
- 📰 **Whole Newsletters** — point the **Substack publication scraper** at any newsletter and it works through the archive
- 👤 **Writer Profiles** — the **Substack profile scraper** returns bio, audience size, social links, and their main publication
- 📄 **Complete Article Bodies** — full text and formatted versions of each post, so you can read and analyse offline
- 💬 **Reader Comments** — switch on replies to collect the discussion under each article
- 📊 **Engagement Data** — likes, comments, and restack counts for every post
- 💰 **Free or Paid** — each post is marked as free or subscriber-only, so you know what sits behind a paywall
- 🎧 **Podcast Support** — audio links and episode durations for newsletters that publish podcasts
- 🏷️ **Tags and Topics** — the labels each publication applies to its own posts
- 📈 **Sort Your Way** — choose the most popular results or the most recent ones
- 🌐 **Custom Domains Handled** — newsletters that moved to their own web address are recognised automatically
- 📝 **Word Counts** — article length is recorded, which is handy for **newsletter research** and content benchmarking
- 🎯 **Set Your Own Size** — collect a handful of posts for a quick look, or a whole archive for deep analysis
- 🔁 **Repeatable Runs** — save your settings and **scrape Substack posts** again whenever you want a fresh snapshot
- 🧾 **Consistent Columns** — every run uses the same layout, so your **Substack data extraction** stacks neatly over time

***

### 📝 Input

Everything can be set up with the simple form in the Apify Console. If you prefer to enter settings directly, this is what a full configuration looks like:

```json
{
  "startUrls": [
    "https://substack.com/search/climate%20tech",
    "https://www.thedispatch.com",
    "https://example.substack.com",
    "https://substack.com/@janedoe"
  ],
  "result_type": "top",
  "limit": 100,
  "enrich_data": true,
  "get_replies": true,
  "maxConcurrency": 5
}
```

#### Input Settings

| Setting | Type | Default | What It Does |
| --- | --- | --- | --- |
| `startUrls` | list | — | The links to collect. Each can be a search page, a newsletter, or a writer's profile — the Substack Scraper works out which is which. |
| `result_type` | text | `top` | How results are ordered: `top` for the most popular and relevant, or `new` for the most recent. Applies to searches and newsletter archives. |
| `limit` | number | — | The most records to collect for each link you supply. |
| `enrich_data` | yes/no | `true` | Fetch each post in full so the complete article text is included. Slower, but far richer. Leave it on unless you only need headlines. |
| `get_replies` | yes/no | `false` | Also collect the comments under each post, attached to the post's row. |
| `maxConcurrency` | number | — | How many pages to work on at once. Lower it if you want gentler, slower runs. |

💡 **Tip:** you can mix all three link types in one run. Add a search, two newsletters, and a writer's profile together, and the Substack Scraper handles each correctly.

***

### 📦 Output

The Substack Scraper writes one row per record. Each row is a **post**, a **publication**, or a **profile**, marked by the `type` column, which keeps results simple to separate and **export Substack data to CSV**. Here is one post row:

```json
[
  {
    "id": "148392017",
    "type": "post",
    "title": "Why grid storage is the real bottleneck",
    "subtitle": "Batteries are cheap now. Connecting them is not.",
    "description": "Batteries are cheap now. Connecting them is not.",
    "published_at": "2026-03-02T11:00:00.000Z",
    "url": "https://climatecurrent.substack.com/p/why-grid-storage-is-the-real-bottleneck",
    "author_name": "Jane Doe",
    "author_handle": "janedoe",
    "publication_name": "Climate Current",
    "image_url": "https://substackcdn.com/image/fetch/cover.jpg",
    "post_type": "newsletter",
    "audience": "everyone",
    "is_paid": false,
    "reaction_count": 412,
    "comment_count": 58,
    "restack_count": 76,
    "wordcount": 2140,
    "language": "en",
    "slug": "why-grid-storage-is-the-real-bottleneck",
    "tags": ["Energy", "Policy"],
    "podcast_url": null,
    "body_text": "Battery cell prices have fallen by more than 80% in a decade...",
    "replies": [
      {
        "body_text": "The interconnection queue point is underrated.",
        "author_name": "Sam Patel"
      }
    ]
  }
]
```

#### Post Columns

| Column | Type | What It Contains |
| --- | --- | --- |
| `id` | text | Substack's own identifier for the post |
| `type` | text | `post`, `publication`, or `profile` |
| `title` | text | The article headline |
| `subtitle` | text | The standfirst shown beneath the headline |
| `description` | text | The summary used in previews and social shares |
| `published_at` | text | When the post was published |
| `url` | text | A clickable link to the article |
| `author_name` | text | Who wrote it |
| `author_handle` | text | Their Substack handle |
| `publication_name` | text | The newsletter it appeared in |
| `image_url` | text | The article's cover image |
| `published_bylines` | list | Every contributor, for co-written posts |
| `post_type` | text | Whether it is a newsletter, podcast, or thread |
| `audience` | text | Who can read it, such as everyone or paying subscribers only |
| `is_paid` | yes/no | `true` when the post sits behind a paywall |
| `reaction_count` | number | How many likes it received |
| `comment_count` | number | How many comments it received |
| `restack_count` | number | How many times readers reshared it |
| `wordcount` | number | How long the article is |
| `language` | text | The language it was written in |
| `tags` / `post_tags` | list | Topic labels applied by the publication |
| `podcast_url` | text | The audio file, for podcast posts |
| `podcast_duration` | number | Episode length in seconds |
| `body_text` | text | The complete article as plain readable text |
| `body_html` | text | The same article with its original formatting |
| `replies` | list | Reader comments, when reply collection is switched on |
| `updated_at` | text | When the post was last edited |

#### Publication and Profile Columns

| Column | Type | What It Contains |
| --- | --- | --- |
| `type` | text | `publication` for newsletters, `profile` for writers |
| `title` | text | The newsletter name, or the writer's name |
| `description` | text | The newsletter tagline, or the writer's bio |
| `url` | text | A clickable link to the newsletter or profile |
| `author_name` | text | Who runs it |
| `author_bio` | text | Their written biography |
| `subscriber_count` | number | How many subscribers they have |
| `subscriber_count_string` | text | The same figure as displayed, e.g. `12K` |
| `follower_count` | number | How many people follow the writer |
| `free_subscriber_count` | number | How many subscribers are on the free tier |
| `bestseller_tier` | number | Substack's bestseller badge level, when awarded |
| `publication_type` | text | The kind of publication it is |
| `tier` | text | Which paid tiers it offers |
| `has_podcast` | yes/no | Whether it publishes audio episodes |
| `custom_domain` | text | Its own web address, when it has moved off substack.com |
| `subdomain` | text | Its `something.substack.com` name |
| `first_post_date` | text | When it started publishing |
| `twitter_screen_name` | text | The writer's X/Twitter handle |
| `user_links` | list | Other links the writer lists on their profile |
| `image_url` | text | The logo, cover photo, or profile picture |

***

### 🚀 How to Use the Substack Scraper (Step by Step)

1. **Sign in** — go to [console.apify.com](https://console.apify.com) and open the **Actors** tab.
2. **Open the actor** — search for **Substack Scraper** and click its card.
3. **Add your links** — paste search pages, newsletter addresses, writer profiles, or a mixture of all three.
4. **Choose your order** — the most popular results, or the most recent.
5. **Set your size** — decide how many records to collect per link. Start small to review the output.
6. **Choose your depth** — leave full article text on for complete posts, or switch it off for faster headline-only runs.
7. **Add comments** — switch on replies if you want reader discussion included.
8. **Press Run and download** — open the **OUTPUT** tab and save as CSV, Excel, JSON, XML, or RSS.

No coding required — the Substack Scraper gives you a complete dataset in minutes.

***

### 🎯 What Makes This Substack Scraper Different

- 🧠 **It Recognises Your Links** — you do not have to say what each link is; the Substack Scraper works out whether it is a search, a newsletter, or a profile
- 📄 **Whole Articles, Not Snippets** — full body text means you can genuinely read and analyse content, which most tools skip
- 🌐 **Custom Domains Handled** — newsletters that moved to their own address are still collected properly, where many tools simply fail
- 📊 **Audience Data Attached** — subscriber and follower numbers arrive with each writer, so your Substack data extraction supports real comparisons
- 💰 **Paywall Clarity** — every post states whether it is free or paid, so you always know what you are looking at
- 🎧 **Podcasts Are Not Forgotten** — audio links and durations come through for newsletters that publish episodes
- 🧵 **Comments in Context** — replies stay attached to their post rather than arriving as a disconnected list
- 🎚️ **Speed You Control** — turn full text on or off, and set how many pages run at once, to balance depth against time
- 📬 **Built for Newsletters** — as a **Substack newsletter scraper** it understands archives, paywall tiers, and podcast posts, not just generic web pages

***

### 💡 Best Use Cases for the Substack Scraper

- 📚 **Competitive Content Analysis** — **scrape Substack posts** from rival newsletters to see what topics and formats they use
- 🤝 **Writer and Creator Discovery** — find newsletters in your niche and check their real audience size before reaching out
- 📢 **Sponsorship Research** — compare **Substack subscriber counts** and engagement to judge whether a placement is worth its price
- 📈 **Topic and Trend Tracking** — follow how a subject is covered across many publications over time
- ✍️ **Content Planning** — use **newsletter research** to spot which angles attract the most reactions and restacks
- 🎙️ **Podcast Discovery** — the **Substack newsletter scraper** surfaces which publications also publish audio
- 🔬 **Media and Academic Research** — build a corpus of full articles with dates, authors, and engagement attached
- 💬 **Audience Insight** — collect comments to hear what readers actually say, not just what writers publish
- 🗄️ **Archive Building** — point the **Substack post scraper** at your own newsletter to keep a complete offline backup of everything you have written
- 📥 **Dashboards and Reports** — **export Substack data to CSV** and load it straight into your reporting tool

***

### 🔧 Good to Know

A few practical notes before you run the Substack Scraper:

- **What you can start from**
  - A Substack search page
  - A newsletter address, on `substack.com` or its own custom domain
  - A writer's profile page
- **Download formats**
  - CSV, Excel, JSON, XML, and RSS
- **Please note**
  - ❌ Subscriber-only articles cannot be read in full — paid posts return their public preview, and the `is_paid` column tells you which ones
  - ❌ Private subscriber lists and email addresses are never collected
  - ℹ️ Full article text makes runs noticeably slower, since each post is fetched individually
  - ℹ️ Comment collection adds further time, so combine it with a sensible limit
  - ℹ️ Very large archives take a while, so start with a small limit to confirm the output looks right

***

### 🤝 Support & Feature Requests

Need a hand, or have an idea for the **Substack Scraper**?

- 💡 **Feature Requests** — want date filtering, scheduled monitoring, or cross-newsletter comparison reports? Just ask.
- 📧 **Contact** — email us at <scrapershub@gmail.com>.

Your feedback shapes what we build next and helps keep this the most useful Substack Scraper on the Apify Store.

***

*Start today with the most complete Substack Scraper available — posts, newsletters, writers, and comments in one run.*

### Related Scrapers

If the Substack Scraper is useful to you, these tools from Scrapers Hub collect listings, profiles, posts, and contact details from other platforms:

- [Reddit Scraper](https://apify.com/scrapers-hub/reddit-scraper-enterprise)
- [Bluesky Scraper](https://apify.com/scrapers-hub/bluesky-scraper-enterprise-grade)
- [Pinterest Scraper](https://apify.com/scrapers-hub/pinterest-scraper-all-in-one)
- [Product Hunt Scraper with Founders & Emails](https://apify.com/scrapers-hub/product-hunt-scraper-with-founders-emails)
- [Y Combinator Scraper with Founders & Emails](https://apify.com/scrapers-hub/y-combinator-scraper-with-founders-emails)
- [Craigslist Scraper](https://apify.com/scrapers-hub/craigslist-scraper)
- [Alibaba Scraper](https://apify.com/scrapers-hub/alibaba-pro-search-scraper)
- [Mercari JP Scraper](https://apify.com/scrapers-hub/mercari-jp-search-scraper)
- [Avito Scraper](https://apify.com/scrapers-hub/avito-kvartiry-scraper)

### Disclaimer

This Substack Scraper collects information only from **publicly available pages**. It does not access paywalled article bodies, private subscriber lists, email addresses, or any password-protected content. Articles remain the property of their authors, and you are responsible for respecting copyright, Substack's terms of service, and applicable data privacy laws when using what you collect.

For data removal requests, please contact **<scrapershub@gmail.com>**. Always use these tools responsibly, ethically, and for legitimate purposes only.

# Actor input Schema

## `startUrls` (type: `array`):

A list of Substack URLs to scrape. Each URL can be a search page (https://substack.com/search/<query>), a user profile (https://substack.com/@handle), a publication subdomain (https://name.substack.com), or a publication custom domain (https://www.example.com).

## `result_type` (type: `string`):

How to sort results. 'top' returns the most popular/relevant items; 'new' returns the most recent. Applies to search pages and publication archives.

## `limit` (type: `integer`):

Maximum number of records to collect per start URL.

## `enrich_data` (type: `boolean`):

When enabled, each post is fetched individually to include its full body\_text and body\_html. Slower but richer. Enabled by default so every post carries its full body.

## `get_replies` (type: `boolean`):

When enabled, fetch the comments/replies for each post and attach them as a 'replies' array.

## `maxConcurrency` (type: `integer`):

Maximum number of parallel HTTP requests.

## Actor input object example

```json
{
  "startUrls": [
    "https://substack.com/search/movie?searching=top",
    "https://unstackit.substack.com",
    "https://substack.com/@vanessathe",
    "https://www.lennysnewsletter.com"
  ],
  "result_type": "top",
  "limit": 100,
  "enrich_data": true,
  "get_replies": false,
  "maxConcurrency": 8
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://substack.com/search/movie?searching=top",
        "https://unstackit.substack.com",
        "https://substack.com/@vanessathe",
        "https://www.lennysnewsletter.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapers-hub/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        "https://substack.com/search/movie?searching=top",
        "https://unstackit.substack.com",
        "https://substack.com/@vanessathe",
        "https://www.lennysnewsletter.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("scrapers-hub/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://substack.com/search/movie?searching=top",
    "https://unstackit.substack.com",
    "https://substack.com/@vanessathe",
    "https://www.lennysnewsletter.com"
  ]
}' |
apify call scrapers-hub/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=scrapers-hub/substack-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/R3Je7a3suOUzThhUh/builds/DLcFthS1pQBxvt517/openapi.json