# Substack Scraper (`tortuga/substack-scraper`) Actor
Scrape Substack publications: post archive with titles, dates, likes, comments, paywall status and full text of free posts, plus publication details and comments. No login.
- **URL**: https://apify.com/tortuga/substack-scraper.md
- **Developed by:** [Trevor Ortega](https://apify.com/tortuga) (community)
- **Categories:** News, Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
from $2.00 / 1,000 posts
This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event
## What's an Apify Actor?
Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Substack Scraper
Scrape Substack publications: post archive with titles, dates, likes, comments, paywall status and full text of free posts, plus publication details and comments. No login.
Substack Scraper reads the public JSON endpoints behind every Substack site (`.substack.com` subdomains and custom domains alike) and turns them into a clean dataset. Give it a newsletter name or URL and get the whole **post archive** newest first: title, subtitle, publish date, post type (newsletter, podcast, video, thread), likes, comment count, restacks, word count, tags, cover image, audio/video links, public bylines and, for free posts, the full `bodyHtml`/`bodyText`. Paywalled posts are marked `isPaywalled: true` with the public preview only. Switch to **publication** mode for the newsletter's description, authors, subscriber count, paid plans, sections and total post count, or to **comments** mode for comment threads with handles, likes and dates. Download JSON, CSV or Excel, or call it from Python, Node, n8n, Make or Google Sheets. You pay only for the items you get; no account or API key is needed.
### What data does Substack Scraper extract?
Every item has a `type` field: `post`, `publication` or `comment`.
#### Post items (`type: "post"`, mode `posts`)
| Field | Description |
|---|---|
| `id`, `slug`, `url`, `canonicalUrl` | Substack post ID, slug and public URL (custom domain when the publication has one) |
| `title`, `subtitle`, `description` | Headline, dek and SEO description |
| `publication` | `{id, name, subdomain, url}` of the newsletter |
| `authors` | Public bylines: `[{name, handle, url, isGuest}]` |
| `publishedAt`, `updatedAt` | ISO 8601 timestamps (`updatedAt` needs **Include post body**) |
| `postType` | `newsletter`, `podcast`, `video` or `thread` |
| `isPaywalled`, `audience` | `true` when `audience` is not `everyone` (`only_paid`, `founding`) |
| `likes`, `reactions` | Reaction count and the per-emoji breakdown (`{"❤": 191}`) |
| `commentsCount`, `childCommentsCount`, `restacks` | Engagement counters |
| `wordCount` | Word count as reported by Substack (also for paywalled posts) |
| `coverImageUrl` | Cover / social image |
| `audioUrl`, `audioDurationSec` | Podcast episode MP3 and duration |
| `hasVideo`, `videoUrl`, `videoUploadId`, `videoPlaybackId`, `videoDurationSec` | Video posts: public MP4 source link and duration |
| `hasVoiceover` | Whether the post has an audio narration |
| `tags`, `section`, `language` | Post tags, section `{name, slug}` and language code |
| `previewText` | Short public teaser (when the author shows one) |
| `bodyHtml`, `bodyText`, `bodyIsPreview` | Full body of free posts (HTML and plain text); for paywalled posts only the public preview, flagged with `bodyIsPreview: true` |
| `scrapedAt` | When the item was collected |
#### Publication items (`type: "publication"`, mode `publication`)
| Field | Description |
|---|---|
| `id`, `name`, `subdomain`, `customDomain`, `url`, `substackUrl` | Identity and both URLs |
| `description`, `longDescription` | Tagline and the longer About text |
| `logoUrl`, `coverImageUrl` | Branding images |
| `authors`, `primaryAuthor` | Contributors `[{name, handle, role, isOwner, url}]` and the owner's name |
| `subscriberCountText`, `freeSubscriberCount`, `freeSubscriberCountApprox` | Public subscriber figures (`Over 1,200,000 subscribers`, `1200000`, `1.2M+`) when the author shows them |
| `paidSubscriberCountText`, `bestsellerTier` | `Tens of thousands of paid subscribers`, bestseller badge tier (100/1000/10000) |
| `isPaid`, `paymentsState`, `plans` | Paid plans `[{id, name, interval, amount, currency}]` |
| `postsCount` | Total posts in the archive |
| `sections`, `hasPodcast`, `podcastTitle`, `podcastFeedUrl` | Sections and podcast info |
| `language`, `publicationType`, `copyright`, `firstPostAt`, `createdAt` | Metadata |
| `socialLinks` | Links the publication lists on its homepage `[{title, url, group}]` |
#### Comment items (`type: "comment"`, mode `comments` or **Include comments**)
`postId`, `id`, `parentId`, `depth`, `body`, `author` (`{name, handle, url}`), `likes`, `restacks`, `repliesCount`, `createdAt`, `editedAt`, `isDeleted`, `isPinned`, `scrapedAt`. Replies are flattened depth-first; `parentId` rebuilds the thread.
### How to use Substack Scraper
1. Paste one or more **Publications**: a subdomain (`lenny`), `lenny.substack.com`, a custom domain (`https://www.astralcodexten.com`) or a single post URL.
2. Pick a **Mode**: `posts` (default), `publication` or `comments`.
3. Set **Max posts per publication** and **Max items** to cap the run and the cost. Turn off **Include post body** for a cheap metadata-only pass, or turn on **Include comments** to get threads with each post.
4. Optionally set **Published from** to stop at a date (the archive is read newest first).
5. Click **Start**. Results appear in the **Dataset** tab; export from there or use the API.
### Input example
```json
{
"publications": ["astralcodexten", "https://www.lennysnewsletter.com"],
"mode": "posts",
"maxPostsPerPublication": 200,
"maxItems": 1000,
"includePostBody": true,
"includeComments": false,
"dateFrom": "2026-01-01"
}
```
```json
{ "mode": "publication", "publications": ["lenny", "newsletter.pragmaticengineer.com", "thefp"] }
```
```json
{ "mode": "comments", "publications": ["https://www.astralcodexten.com/p/god-help-us-lets-try-to-learn-about"], "maxCommentsPerPost": 200 }
```
### Output example
```json
{
"type": "post",
"id": 207542232,
"slug": "god-help-us-lets-try-to-learn-about",
"url": "https://www.astralcodexten.com/p/god-help-us-lets-try-to-learn-about",
"title": "God Help Us, Let’s Try To Learn About Mechanistic Interpretability Techniques",
"subtitle": "...",
"publication": { "id": 89120, "name": "Astral Codex Ten", "subdomain": "astralcodexten", "url": "https://www.astralcodexten.com" },
"authors": [{ "name": "Scott Alexander", "handle": "astralcodexten", "url": "https://substack.com/@astralcodexten", "isGuest": false }],
"publishedAt": "2026-09-08T12:04:21.658Z",
"updatedAt": "2026-09-08T14:10:02.114Z",
"postType": "newsletter",
"isPaywalled": false,
"audience": "everyone",
"likes": 191,
"reactions": { "❤": 191 },
"commentsCount": 110,
"restacks": 13,
"wordCount": 4695,
"coverImageUrl": "https://substackcdn.com/image/fetch/...",
"audioUrl": null,
"hasVideo": false,
"tags": [],
"section": null,
"bodyHtml": "...
...
",
"bodyText": "...",
"bodyIsPreview": false,
"scrapedAt": "2026-09-08T22:30:00Z"
}
```
```json
{
"type": "publication",
"id": 10845,
"name": "Lenny's Newsletter",
"subdomain": "lenny",
"customDomain": "www.lennysnewsletter.com",
"url": "https://www.lennysnewsletter.com",
"description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
"authors": [{ "name": "Lenny Rachitsky", "handle": "lenny", "role": "admin", "isOwner": true, "url": "https://substack.com/@lenny" }],
"subscriberCountText": "Over 1,200,000 subscribers",
"freeSubscriberCount": 1200000,
"paidSubscriberCountText": "Tens of thousands of paid subscribers",
"isPaid": true,
"plans": [{ "id": "monthly15", "name": "$15 a month", "interval": "month", "amount": 15.0, "currency": "USD" }],
"postsCount": 812,
"sections": [{ "name": "How I AI", "slug": "how-i-ai" }],
"hasPodcast": true
}
```
### How to scrape a Substack newsletter's full archive
Use **Mode** `posts` with the publication name or URL and raise **Max posts per publication** (and **Max items**) above the number of posts; the Actor pages through the archive 50 posts at a time until it runs out. Leave **Include post body** on to get the full text of every free post in one run, or turn it off to list thousands of posts for a fraction of the requests. **Published from** lets you re-run on a schedule and only pick up posts since your last export.
### Does it work without a Substack account?
Yes. Everything comes from the public endpoints that Substack's own website calls when you browse logged out. No login, cookies or API key are involved, which also means paid-subscriber-only content is never accessed: paywalled posts come with their metadata (title, date, likes, comment count, word count) and the public preview, marked `isPaywalled: true` and `bodyIsPreview: true`.
### Can I scrape custom-domain Substacks like lennysnewsletter.com?
Yes. Paste the custom domain URL, the `.substack.com` subdomain or just the subdomain name; Substack redirects the API to the custom domain and the Actor follows it, so URLs in the output always use the publication's real domain.
### How to get Substack comments
Set **Mode** `comments` (or keep `posts` and switch on **Include comments**) and set **Max comments per post**. Each comment item carries the public commenter name and handle, the text, likes, reply count, dates and `parentId` so you can rebuild threads. Paste a single post URL as the publication to get the comments of just that post.
### How to get subscriber counts and prices of a Substack
Use **Mode** `publication`. The item includes the subscriber figures the author makes public (`Over 1,200,000 subscribers`, paid tier text, bestseller badge), monthly/annual plan prices, sections, podcast info and the total number of posts. Publications that hide their subscriber count return `null` there.
### What does it cost?
Pay per item: a post or a publication item costs a fraction of a cent, a comment a quarter of that. There is no start fee and no subscription; Apify's free plan is enough to export a few thousand posts.
### Integrations and API
Use the run in Zapier, Make, n8n, Google Sheets, or call it from Python/Node with the Apify client. See the **API** tab for ready-made snippets, or schedule the Actor to keep an archive or comment feed fresh.
### Is it legal to scrape Substack?
This Actor collects only publicly available data that Substack shows to anyone without an account: post archives, free post text, public previews, publication details and public comments. It does not log in and never reads paid-subscriber content. Author and commenter names and handles are the public bylines Substack prints on every page; no emails, avatars or private profile data are collected. You are responsible for how you use the data and for complying with Substack's terms and applicable law.
### Support
Found a bug or need a field added? Open an issue in the **Issues** tab; it is usually answered within a day.
# Actor input Schema
## `publications` (type: `array`):
Substack publications to scrape, one per line: a subdomain (`lenny`), `lenny.substack.com`, a custom domain (`https://www.astralcodexten.com`) or a single post URL (`https://www.astralcodexten.com/p/some-post`, which scrapes only that post).
## `mode` (type: `string`):
`posts`: every post in the archive (newest first) with metadata and, optionally, body and comments. `publication`: one item per publication with description, authors, subscriber count, plans and post count. `comments`: only comments of the newest posts.
## `maxPostsPerPublication` (type: `integer`):
How many posts to read from each publication's archive, newest first (also bounds the posts whose comments are fetched).
## `maxItems` (type: `integer`):
Overall cap on dataset items for the run (posts + comments, or publications). Keeps cost predictable.
## `includePostBody` (type: `boolean`):
Fetch each post to add `bodyHtml`, `bodyText` and `updatedAt`. Free posts give the full text; paywalled posts give only the public preview (if the author shows one). One extra request per post.
## `includeComments` (type: `boolean`):
In `posts` mode also save `comment` items for each post (one extra request per post with comments).
## `maxCommentsPerPost` (type: `integer`):
Cap on comments (including replies) saved per post when comments are fetched.
## `dateFrom` (type: `string`):
Only posts published on or after this ISO date, e.g. `2026-01-01` or `2026-01-01T00:00:00Z`. The archive is read newest first and stops at the first older post.
## `proxyConfiguration` (type: `object`):
Apify Proxy is recommended for larger runs so requests are spread over several IPs.
## Actor input object example
```json
{
"publications": [
"astralcodexten",
"https://www.lennysnewsletter.com"
],
"mode": "posts",
"maxPostsPerPublication": 100,
"maxItems": 500,
"includePostBody": true,
"includeComments": false,
"maxCommentsPerPost": 50,
"proxyConfiguration": {
"useApifyProxy": true
}
}
```
# Actor output Schema
## `results` (type: `string`):
No description
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"publications": [
"astralcodexten",
"https://www.lennysnewsletter.com"
],
"proxyConfiguration": {
"useApifyProxy": true
}
};
// Run the Actor and wait for it to finish
const run = await client.actor("tortuga/substack-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"publications": [
"astralcodexten",
"https://www.lennysnewsletter.com",
],
"proxyConfiguration": { "useApifyProxy": True },
}
# Run the Actor and wait for it to finish
run = client.actor("tortuga/substack-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"publications": [
"astralcodexten",
"https://www.lennysnewsletter.com"
],
"proxyConfiguration": {
"useApifyProxy": true
}
}' |
apify call tortuga/substack-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,tortuga/substack-scraper"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/yE3dsOhErlTO2Z5ZU/builds/vafeHoDTxU0sbtxeI/openapi.json