# Substack Scraper: Posts, Likes, Comments, Full Text (`fernandosord/substack-scraper-posts`) Actor

Scrape any Substack newsletter, custom domains included: title, date, authors, likes, comments, restacks, tags and subscriber count. Optional full text as plain text and Markdown, comment threads, and keyword search across Substack. No login.

- **URL**: https://apify.com/fernandosord/substack-scraper-posts.md
- **Developed by:** [Plainsource](https://apify.com/fernandosord) (community)
- **Categories:** Social media, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 posts

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

This Substack scraper downloads posts from any Substack newsletter, including the ones on their own domain, with title, date, authors, likes, comments, restacks, tags and the publication's subscriber count. Turn on full text to get each article as plain text and Markdown, and turn on comments to get the discussion under each post. No Substack account, cookie or API key needed.

- **Paste any kind of Substack link.** A name like `noahpinion`, a `substack.com` URL, a custom domain like `www.lennysnewsletter.com`, an author profile like `@noahpinion`, or a single post URL all work.
- **Get the whole archive.** Posts come newest first and the Actor keeps paging until it hits your limit or a date like `2026-01-31` or `30 days`.
- **Hidden sections included.** Many newsletters keep podcast notes or guest series out of their main archive. This Actor reads those sections too and merges them by date.
- **Search all of Substack** by keyword to collect posts from newsletters you don't know yet.

### What data you get

| Field | What it holds |
|---|---|
| `title`, `subtitle`, `url`, `publishedAt` | The post and when it went out |
| `authors`, `authorNames` | Every byline with name, handle and profile link, guests included |
| `likes`, `commentCount`, `restacks` | Engagement counts |
| `audience`, `isPaid` | Whether the post is free or for paid subscribers |
| `wordCount` | Length of the full post, also for paid posts |
| `tags`, `section`, `coverImage`, `podcast` | Tags, section name, cover image URL, and audio URL for podcast episodes |
| `publicationName`, `publicationUrl`, `publicationSubscribers` | The newsletter, its home page and its public subscriber count ("Over 461,000 subscribers", plus `publicationSubscribersMin` as a number) |
| `text`, `markdown`, `textIsPreview` | Full article as plain text and Markdown (with **Include full text**) |
| `comments` | Comment threads with author, date, text, likes and nested replies (with **Include comments**) |

### How to use it

1. Add one or more publications or post URLs, or a search query.
2. Set **Max posts per publication**.
3. Optional: set **Only posts newer than**, and turn on **Include full text** or **Include comments**.
4. Run it and export to JSON, CSV or Excel, or call it from the API, Make, Zapier, n8n or an AI agent through the Apify MCP server.

Full text and comments each cost one extra request per post, so a run with both on takes a few seconds per post instead of a few seconds per 50 posts.

#### Example input

```json
{
  "publications": ["noahpinion", "www.lennysnewsletter.com"],
  "maxPostsPerPublication": 3,
  "onlyPostsNewerThan": "30 days",
  "includeFullText": true,
  "includeComments": true,
  "maxCommentsPerPost": 5
}
```

#### Example output (one post, text and comments shortened)

```json
{
  "publicationName": "Noahpinion",
  "publicationUrl": "https://www.noahpinion.blog",
  "publicationSubdomain": "noahpinion",
  "publicationSubscribers": "Over 461,000 subscribers",
  "publicationSubscribersMin": 461000,
  "postId": 222264141,
  "title": "Roundup #90: In which Noah joins a Japanese VC firm",
  "url": "https://www.noahpinion.blog/p/roundup-90-in-which-noah-joins-a",
  "publishedAt": "2026-10-27T07:17:34.767Z",
  "updatedAt": "2026-10-27T07:18:58.872Z",
  "type": "newsletter",
  "audience": "everyone",
  "isPaid": false,
  "authors": [
    { "name": "Noah Smith", "handle": "noahpinion", "id": 8243895, "profileUrl": "https://substack.com/@noahpinion", "isGuest": false }
  ],
  "authorNames": "Noah Smith",
  "wordCount": 3167,
  "likes": 278,
  "commentCount": 34,
  "restacks": 17,
  "coverImage": "https://substackcdn.com/image/fetch/$s_!KCl-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5b5caa2f-6bdf-48cc-9785-963744ee4cae_1006x600.jpeg",
  "tags": [],
  "language": "en",
  "text": "I have a fun announcement today, which is that I have a new side gig. I’ll be joining the Japanese VC firm Coral Capital as a Venture Partner! ...",
  "markdown": "![](https://substackcdn.com/image/fetch/...)\n\nI have a fun announcement today, which is that I have a new side gig. ...",
  "textWordCount": 3177,
  "textIsPreview": false,
  "comments": [
    {
      "id": 371442049,
      "parentId": null,
      "author": "Fallingknife",
      "date": "2026-10-27T09:13:44.880Z",
      "text": "Dying or mean reverting? ...",
      "likes": 21,
      "replies": [
        { "id": 371957546, "parentId": 371442049, "author": "cp6", "date": "2026-10-27T20:47:34.043Z", "text": "I agree. ...", "likes": 7, "replies": [] }
      ]
    }
  ],
  "commentsFetched": 5,
  "source": "noahpinion"
}
```

### How much does it cost?

You pay per post saved. Comments are stored inside their post, so they never add items. Inputs that fail (a wrong link, a deleted post, a site that is not on Substack) cost nothing and are listed in the `PROBLEMS` record of the run's key-value store.

Each post costs $0.0015, plus $0.0005 once per run. Full text and comments are included at no extra cost. 1,000 posts cost about $1.50, and Apify's free plan credit of $5 a month covers about 3,300 posts.

### Use cases

- **Newsletter research:** compare how often a set of newsletters post, how long the posts are and which ones get the most likes and restacks.
- **Content monitoring:** run it on a schedule with **Only posts newer than** set to `1 day` to collect new posts from the newsletters you follow.
- **AI and search datasets:** collect free articles as Markdown for summaries, RAG pipelines or topic analysis.
- **Audience research:** read the comment threads to see what readers ask and argue about.

### FAQ

**Can it get paid posts?**
It gets every paid post's title, date, authors, word count, likes, comments count and restacks. The text is only the free preview that Substack shows to non-subscribers, and those posts have `textIsPreview: true`. Substack also hides the comments on paid posts from non-subscribers, so `commentsNote` says so instead.

**Does it work with custom domains?**
Yes. Give the custom domain (`www.thefp.com`) or the `substack.com` subdomain, which redirects to it. If a domain is not a Substack site, the input is listed in `PROBLEMS` with that reason.

**What about rate limits?**
Substack starts answering "Too Many Requests" when one IP asks too fast. The Actor spaces its requests about a second apart, and when Substack still says no it waits longer and retries, up to six times per request. If you scrape thousands of posts with full text, turn on Apify Proxy to spread the load.

**How many posts can I get from one newsletter?**
The whole public archive. Set **Max posts per publication** to a number larger than the archive.

**Do I need a Substack account?**
No. The Actor reads the same public data your browser loads when you open a newsletter's archive.

**Is this legal?**
It only reads what newsletters publish publicly. You are responsible for how you use the data, especially personal data in comments, under laws like GDPR.

### Other Actors by this developer

- [Telegram Channel Scraper](https://apify.com/fernandosord/telegram-channel-scraper): posts, views and reactions from public Telegram channels, no login.
- [Google Trends Scraper](https://apify.com/fernandosord/google-trends-reliable): interest over time, by region and related searches, built to survive Google's rate limits.

# Actor input Schema

## `publications` (type: `array`):

Any mix of: a subdomain (noahpinion), a publication URL (https://noahpinion.substack.com), a custom domain (www.lennysnewsletter.com), an author profile (@noahpinion or substack.com/@noahpinion) or a single post URL (https://www.noahpinion.blog/p/some-post).

## `searchQueries` (type: `array`):

Optional. Keywords to search across every Substack publication, using Substack's own search. Each query returns posts from many newsletters.

## `maxPostsPerPublication` (type: `integer`):

Newest posts first. The Actor stops at this number for each publication. Set it high (for example 5000) to get the whole archive.

## `maxPostsPerSearch` (type: `integer`):

How many posts to keep for each search query, in Substack's relevance order.

## `onlyPostsNewerThan` (type: `string`):

Optional. A date (2026-01-31) or a relative period (7 days, 3 months). Older posts are skipped.

## `includeAllSections` (type: `boolean`):

Some newsletters keep sections (podcast notes, guest series, link roundups) out of their main archive. On by default, so you get every post the publication has. Turn off for the main newsletter only.

## `includeFullText` (type: `boolean`):

Adds the article as plain text and as Markdown. Costs one extra request per post, so runs take longer. Paid posts only give the free preview, marked with textIsPreview.

## `includeComments` (type: `boolean`):

Adds the comment thread (author, date, text, likes, replies) inside each post item. Comments do not add extra items or extra cost per item. Comments on paid posts are hidden by Substack for non-subscribers.

## `maxCommentsPerPost` (type: `integer`):

Cap on comments (replies included) saved inside each post when comments are on. Best comments come first.

## `proxyConfiguration` (type: `object`):

Optional. Substack's public API works without a proxy. Turn on Apify Proxy if you scrape thousands of posts and start seeing HTTP 429 retries in the log.

## Actor input object example

```json
{
  "publications": [
    "noahpinion"
  ],
  "maxPostsPerPublication": 20,
  "maxPostsPerSearch": 50,
  "includeAllSections": true,
  "includeFullText": false,
  "includeComments": false,
  "maxCommentsPerPost": 200,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

One item per post, newest first for each publication.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "noahpinion"
    ],
    "maxPostsPerPublication": 20,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("fernandosord/substack-scraper-posts").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publications": ["noahpinion"],
    "maxPostsPerPublication": 20,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("fernandosord/substack-scraper-posts").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "noahpinion"
  ],
  "maxPostsPerPublication": 20,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call fernandosord/substack-scraper-posts --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fernandosord/substack-scraper-posts"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/eCHnoPOQT1U910a00/builds/YSLAjKSzRheSkBAzL/openapi.json