# Substack Scraper – Posts, Newsletters & Stats (`anxuanng/substack-scraper`) Actor

Scrape posts from any Substack newsletter: title, date, free or paid, authors, likes, comments, restacks, word count and the full text of free posts. Custom domains, keyword search for publications and an only-new mode for monitoring. Pay per post.

- **URL**: https://apify.com/anxuanng/substack-scraper.md
- **Developed by:** [Xuan An Nguyen](https://apify.com/anxuanng) (community)
- **Categories:** Social media, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 post scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Substack Scraper – Posts, Newsletters & Stats

Scrape posts from any Substack newsletter, including those on custom domains: title, subtitle, publish date,
**free or paid**, authors, **likes, comments, restacks**, word count, tags, cover image and the **full text of free
posts**. Scrape many publications in one run, or find publications by keyword. An only-new mode makes it a cheap
daily monitor.

### What data can I get?

| Field | Example |
|---|---|
| `publication`, `publicationName`, `publicationSubscribers` | `www.lennysnewsletter.com`, name and free-subscriber count (when found through search) |
| `title`, `subtitle`, `url` | post title, subtitle and canonical link |
| `publishedAt`, `updatedAt` | ISO dates |
| `audience`, `isPaid` | `free` / `paid` / `founding`, `true` |
| `authors` | `["Lenny Rachitsky"]` (names as printed on the post) |
| `likes`, `comments`, `restacks` | `371`, `18`, `9` |
| `wordCount`, `tags`, `language`, `type` | `933`, `["Career"]`, `en`, `newsletter` / `podcast` / `thread` |
| `coverImage`, `podcastUrl` | image and audio links |
| `bodyText`, `bodyTruncated` | full text of free posts; the public preview of paid posts, flagged `true` |

### How much does it cost?

**$2 per 1,000 posts.** You pay only for posts delivered. Scraping the last 50 posts of 10 newsletters costs $1.

### How do I use it?

1. Add publications, one per line: `lenny`, `noahpinion.substack.com`, `https://www.lennysnewsletter.com`, or a post URL.
2. Or type a topic in **Find publications by keyword** (e.g. `AI agents`) to discover newsletters and scrape them.
3. Set how many recent posts per publication, a date window, and free/paid.
4. Run, then download JSON, CSV or Excel, or call the API.

```json
{
  "publications": ["lenny", "https://www.noahpinion.blog"],
  "searchQuery": "product management",
  "maxPublications": 10,
  "maxPostsPerPublication": 50,
  "postedAfter": "2026-01-01",
  "audience": "all",
  "includeBody": true
}
```

### How do I monitor newsletters for new posts?

Turn on **Only new posts** and schedule the Actor. Each run looks at the newest posts of each publication (up to
"Max posts per publication") and returns only the ones you haven't received yet, so you never pay twice for a post.

### Use cases

- **Content research:** which topics get the most likes and comments in your niche.
- **Competitor and creator tracking:** every new post from a list of newsletters, in one sheet or feed.
- **Sponsorship and partnerships:** find newsletters by topic with their subscriber counts.
- **AI and search pipelines:** clean full text of free posts for summarisation or RAG.

### FAQ

**Can it read paid posts?** No. It never logs in or bypasses the paywall. Paid posts come with their public preview
and `bodyTruncated: true`.

**Is it legal?** It reads the same public JSON that Substack's own web app loads for any visitor. It does not collect
subscriber lists, commenter names or emails.

**Do I need a proxy?** No. It runs at 256 MB without a proxy.

**A custom-domain newsletter isn't found.** Paste any post URL from it; the Actor reads the host from the link.

### More scrapers by the same developer

- [ATS Jobs Scraper](https://apify.com/anxuanng/ats-jobs-scraper): Job postings straight from company career boards on Greenhouse, Lever, Ashby, Workday, Workable, Recruitee and SmartRecruiters in one schema, with salary ranges and an only-new mode.
- [Chotot Scraper](https://apify.com/anxuanng/chotot-scraper): Scrape Chợ Tốt (chotot.com), Vietnam's largest classifieds site: cars, motorbikes, real estate, phones, laptops and more, with prices in VND, location and all item attributes. Pay per listing.
- [EU Company Registry Scraper](https://apify.com/anxuanng/company-registry-scraper): Companies from the official open registries of France (SIRENE), Norway (Brønnøysund) and Finland (PRH): legal form, status, activity code, address, workforce and French revenue. New-registration monitoring.
- [Events Scraper](https://apify.com/anxuanng/events-scraper): Upcoming events in any city from Eventbrite, Meetup and Luma in one schema: time in UTC, venue, online or in person, organizer, price and attendance.
- [Hacker News Scraper](https://apify.com/anxuanng/hacker-news-scraper): Hacker News front page feeds, keyword search over stories and comments, and Who Is Hiring job posts parsed into company, role, location, remote, salary and stack. Only-new mode for alerts.
- [Prediction Markets Scraper](https://apify.com/anxuanng/prediction-markets-scraper): Kalshi and Polymarket markets in one schema: yes/no probabilities, bid/ask, volume, liquidity, close time and results, with keyword and category filters and an only-changed mode for monitoring.
- [Remote Jobs Scraper](https://apify.com/anxuanng/remote-jobs-scraper): Remote job listings from Himalayas, Remote OK, We Work Remotely, Jobicy and Arbeitnow in one schema, deduplicated across boards, with salaries and an only-new mode.
- [SEEK, JobStreet & JobsDB Jobs Scraper](https://apify.com/anxuanng/seek-jobs-scraper): Job listings from SEEK (Australia, New Zealand), JobStreet (Malaysia, Singapore, Philippines, Indonesia) and JobsDB (Hong Kong, Thailand) in one schema, with salary, work type, classification and an only-new mode.
- [Telegram Channel Scraper](https://apify.com/anxuanng/telegram-channel-scraper): Scrape public Telegram channels: posts, views, reactions, media, links, forwards, plus channel stats. Incremental mode for daily monitoring. Pay per post.
- [Vietnam Jobs Scraper](https://apify.com/anxuanng/vietnam-jobs-scraper): Scrape job postings from VietnamWorks, TopCV and ITviec in one unified schema. Extract title, company, salary in VND/USD, location, level, skills, deadline and full description. Pay per job, JSON/CSV export, API and MCP access.

# Actor input Schema

## `publications` (type: `array`):

Substack newsletters to scrape, one per line: a name ('lenny'), a subdomain ('noahpinion.substack.com'), a custom domain ('www.lennysnewsletter.com') or any post URL.

## `searchQuery` (type: `string`):

Optional. Search Substack for publications about this topic (e.g. 'product management', 'AI agents') and scrape their posts too.

## `maxPublications` (type: `integer`):

How many publications to take from the keyword search.

## `maxPostsPerPublication` (type: `integer`):

Newest posts first. Stops earlier if 'Posted after' is reached.

## `postedAfter` (type: `string`):

Only posts published on or after this date (YYYY-MM-DD). Leave empty for no limit.

## `audience` (type: `string`):

Keep all posts, only free posts, or only paid (subscriber-only) posts.

## `includeBody` (type: `boolean`):

Add the full text of free posts as plain text (paid posts get the public preview, marked bodyTruncated). Turn off for metadata only; it is faster.

## `onlyNewPosts` (type: `boolean`):

Return only posts you have not received in earlier runs of this Actor. Turn on for scheduled monitoring so you never pay twice for the same post.

## `maxItems` (type: `integer`):

Stop after this many posts across all publications. You are only charged per post delivered.

## Actor input object example

```json
{
  "publications": [
    "lenny",
    "https://www.noahpinion.blog"
  ],
  "maxPublications": 10,
  "maxPostsPerPublication": 50,
  "audience": "all",
  "includeBody": true,
  "onlyNewPosts": false,
  "maxItems": 1000
}
```

# Actor output Schema

## `listings` (type: `string`):

All post rows as JSON; append ?format=csv for CSV or ?format=xlsx for Excel.

## `runSummary` (type: `string`):

Counts of pushed/skipped/failed rows and whether the budget limit was hit.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "lenny",
        "https://www.noahpinion.blog"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("anxuanng/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "publications": [
        "lenny",
        "https://www.noahpinion.blog",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("anxuanng/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "lenny",
    "https://www.noahpinion.blog"
  ]
}' |
apify call anxuanng/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,anxuanng/substack-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/By9ilw0gleO6DGHyF/builds/fRLBdhvbNcXWc5o4s/openapi.json