# Substack Scraper - Posts, Substack Notes & Newsletter Archive (`kamerozkan/substack-scraper-posts-notes`) Actor

Substack scraper for posts and Substack Notes: paste publication URLs or profile handles and get every post with title, date, paywall flag, likes, restacks, comments, tags, authors and the full text of free posts, plus Notes with text, likes, restacks and replies. No login, no API key.

- **URL**: https://apify.com/kamerozkan/substack-scraper-posts-notes.md
- **Developed by:** [Kamer Ozkan](https://apify.com/kamerozkan) (community)
- **Categories:** News, Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.96 / 1,000 post or notes

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Scraper - Posts, Substack Notes & Newsletter Archive

A **Substack scraper** for posts and **Substack Notes**: paste publication URLs or profile handles and get every post of a newsletter archive (title, subtitle, date, paywall flag, likes, restacks, comments, word count, tags, authors, cover image, podcast link and, for free posts, the full article text) plus the Notes of any profile (text, date, likes, restacks, replies, attached posts and links). It reads the public JSON endpoints that every Substack publication serves, so it needs no login, no API key and no proxy, and it never bypasses a paywall.

### What does Substack Scraper do?

Give it newsletters (`https://www.lennysnewsletter.com/`, `noahpinion`, any post URL, or a profile URL that is resolved to the author's main publication) and it walks the archive newest first, 50 posts per request, with an optional date cut-off so scheduled runs only read what is new. Give it profile handles (`lenny`, `@noahpinion`, `https://substack.com/@lenny`) and it reads their **Substack Notes** feed with the same metrics. Both kinds land in one dataset with a `kind` field (`post` or `note`); the **Posts** and **Notes** views show each kind as a clean table, and CSV, Excel, JSON and the API export work as usual.

Typical runs: export a whole newsletter archive with full text for research or a RAG index, monitor competitor newsletters and their engagement, collect the Notes of the creators you follow, find a publication's most liked posts, or build a topic feed across many newsletters.

### Who is it for?

- **Newsletter operators and content marketers**: competitor archives with likes, restacks and comments per post; top posts by engagement; which topics and tags perform.
- **Researchers, journalists and analysts**: searchable archives of expert newsletters as plain text, with dates, authors and paywall status.
- **AI and RAG builders**: the full text of free posts as clean plain text, one row per post, ready for embeddings.
- **Creators and ghostwriters**: the Notes of a profile with engagement, to see what works on Substack Notes.
- **Growth and sales teams**: a scheduled feed of new posts and Notes from a watchlist of publications and people.

### What data can you get?

Posts (`kind` = `post`):

| Field | Example |
|---|---|
| `id`, `title`, `subtitle`, `slug`, `url` | 215694124, All of the Lenny & Friends Summit talks are now online!, Plus, some reflections and takeaways from the day, ..., https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit |
| `postType`, `publishedAt` | newsletter (or podcast, thread, video), 2026-09-29T13:15:57.512Z |
| `audience`, `isPaywalled` | everyone / only_paid / founding, false |
| `likes`, `restacks`, `comments`, `wordCount` | 291, 5, 7, 1276 |
| `tags`, `sectionName`, `language` | \["Career"], null, en |
| `authors` | \[{"name": "Lenny Rachitsky", "handle": "lenny", "url": "https://substack.com/@lenny", "isGuest": false}] |
| `description`, `coverImage`, `podcastUrl`, `podcastDurationSeconds` | the post description, cover image URL, audio URL and length for podcast posts |
| `publicationId`, `publicationName`, `publicationUrl` | 10845, Lenny's Newsletter, https://www.lennysnewsletter.com |
| `bodyText`, `bodyTruncated` | the full article as plain text (free posts, with **Include the full text** on); paywalled posts carry the public preview and `bodyTruncated` = true |

Notes (`kind` = `note`):

| Field | Example |
|---|---|
| `id`, `url` | 216329331, https://substack.com/@lenny/note/c-216329331 |
| `text`, `publishedAt`, `editedAt` | the Note as plain text, 2026-02-18T18:18:50.292Z, null |
| `likes`, `restacks`, `replies` | 36, 1, 6 |
| `isRestack`, `attachments` | true when the Note only shares something; \[{"type": "publication", "title": "Marketing Ideas", "url": "https://www.marketingideas.com"}] (types: post, link, image, publication, note) |
| `authorName`, `authorHandle`, `authorUrl`, `authorId` | Lenny Rachitsky, lenny, https://substack.com/@lenny, 1849774 |
| `publicationName`, `publicationUrl` | Lenny's Newsletter, https://www.lennysnewsletter.com |

Authors are the publishers' public bylines. Commenters, subscribers and other reader data are never collected.

### How much does it cost?

You pay per item: **$1.20 per 1,000 posts or Notes** ($0.0012 per item) on the Free plan, with Store discounts on paid Apify plans: Bronze $1.08, Silver $1.02, Gold and above $0.96 per 1,000 items. Platform usage is included; the full text of free posts costs nothing extra.

Worked examples:

- The archive of one newsletter, 300 posts with metadata: 300 x $0.0012 = **$0.36**.
- The same archive with the full text of its 180 free posts (**Only free posts** on): 180 x $0.0012 = **$0.22**.
- A daily watchlist of 20 publications and 20 profiles with **Published after** set to yesterday, about 15 new posts and 40 new Notes a day: 55 x $0.0012 = **$0.07 per day**.

Publications and profiles that cannot be read (private, deleted, mistyped) are listed in the run's OUTPUT record and are **never charged**. Posts skipped by a filter are not charged either. Set **Max items** or a maximum cost per run to cap spending.

### How to use it

1. Click **Try for free**.
2. Paste newsletters into **Publications** (URL, subdomain or post URL, one per line) and profile handles into **Profiles (Substack Notes)**. Either list can stay empty.
3. Optional: turn on **Include the full text of free posts**, **Only free posts**, choose **Top posts first**, or set **Published after** for a scheduled feed.
4. Click **Start** and download the table as CSV, Excel, JSON or through the API. Switch to the **Posts** or **Notes** view for a table of one kind.

### Input example

```json
{
  "publications": ["https://www.lennysnewsletter.com/", "noahpinion"],
  "profiles": ["lenny"],
  "maxPostsPerPublication": 50,
  "maxNotesPerProfile": 50,
  "includeFullText": false,
  "onlyFreePosts": false,
  "sortPosts": "new",
  "maxItems": 1000
}
```

### Output example

Two unedited rows from a live run (long strings shortened):

```json
{
  "kind": "post",
  "id": 215694124,
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "slug": "all-of-the-lenny-and-friends-summit",
  "postType": "newsletter",
  "publishedAt": "2026-09-29T13:15:57.512Z",
  "audience": "everyone",
  "isPaywalled": false,
  "description": "Plus, some reflections and takeaways from the day",
  "likes": 291,
  "restacks": 5,
  "comments": 7,
  "wordCount": 1276,
  "tags": [],
  "sectionName": null,
  "authors": [{"name": "Lenny Rachitsky", "handle": "lenny", "url": "https://substack.com/@lenny", "isGuest": false}],
  "language": "en",
  "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-...jpeg",
  "podcastUrl": null,
  "podcastDurationSeconds": null,
  "publicationId": 10845,
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "bodyText": null,
  "bodyTruncated": null
}
{
  "kind": "note",
  "id": 216329331,
  "url": "https://substack.com/@lenny/note/c-216329331",
  "text": "I'm thrilled to welcome The Skip with @Nikhyl Singhal to Lenny's Podcast Network!\nI've been a massive fan of Nikhyl's for many years. He spoke at my inaugural summit and was an early guest on my podcast. ...",
  "publishedAt": "2026-02-18T18:18:50.292Z",
  "editedAt": null,
  "likes": 36,
  "restacks": 1,
  "replies": 6,
  "isRestack": false,
  "attachments": [{"type": "link", "title": "- YouTube", "url": "https://www.youtube.com/watch?v=38FFgtSXMew&t=1s"}],
  "authorName": "Lenny Rachitsky",
  "authorHandle": "lenny",
  "authorUrl": "https://substack.com/@lenny",
  "authorId": 1849774,
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "language": "en"
}
```

### Integrations and API

Run it from your code with the Apify API or the `apify-client` package, schedule it in Apify (a daily run with **Published after** gives you only the new posts and Notes), or connect it to Make, n8n, Zapier, Google Sheets or a webhook.

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("kamerozkan/substack-scraper-posts-notes").call(run_input={
    "publications": ["https://www.lennysnewsletter.com/", "noahpinion"],
    "profiles": ["lenny"],
    "includeFullText": True,
    "onlyFreePosts": True,
    "maxItems": 200,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["kind"], item.get("title") or item.get("text"), item["publishedAt"], item["likes"])
```

### FAQ

**Does it return the text of paid posts?** No. The full text is read for free posts only. Paywalled posts return their public metadata and the preview that Substack shows to everyone, with `bodyTruncated` = true. Use **Only free posts** to leave them out entirely.

**How do I get only new posts on a schedule?** Set **Published after** to the date of your last run (or use a date one day back for a daily schedule). With **Newest first** the scraper stops reading the archive as soon as it reaches older posts, so each run costs only the new items.

**Can I scrape Notes of a profile without a publication?** Yes. Notes come from the profile feed; `publicationName` is then empty.

**What counts as a Note?** Every entry of the profile's Notes feed: original Notes with text, and restacks (shares of a post or publication without text), flagged with `isRestack` = true and the shared item in `attachments`.

**Which URLs are accepted?** Publication home pages on substack.com subdomains or custom domains, any post URL (the publication is derived from it), bare subdomains, and profile URLs or handles. Private or deleted publications end up in the OUTPUT record as failed and are not charged.

**How fast is it?** About one request per 50 posts and one per page of Notes, with a short pause between requests to stay polite; a 300-post archive takes a few seconds, full text adds about half a second per free post.

**Is it legal to collect this data?** It collects publicly available posts, Notes and publisher bylines only, respects paywalls and never collects reader or subscriber data. You are responsible for how you use it, including copyright and privacy laws like GDPR.

**Something is not working?** Open an issue on the Issues tab with your run link and we will fix it fast.

### Support

Questions, a publication that answers oddly, or a field you need (for example comments of a post, podcast transcripts or a publication's recommendations)? Open an issue on the **Issues** tab of this actor with your run link. Issues are read every working day and fixes ship within a few days; feature requests that other users share get built.

# Actor input Schema

## `publications` (type: `array`):

Newsletters whose posts you want, one per line: a publication URL (https://www.lennysnewsletter.com/), a subdomain (noahpinion = noahpinion.substack.com), any post URL of the publication, or a profile URL (https://substack.com/@lenny, resolved to the author's main publication).

## `profiles` (type: `array`):

Substack profiles whose Notes you want, one per line: a handle (lenny), @handle or a profile URL (https://substack.com/@lenny).

## `maxPostsPerPublication` (type: `integer`):

Newest first. 50 posts are one request; the whole archive of a large newsletter is a few hundred posts.

## `maxNotesPerProfile` (type: `integer`):

Newest first. Restacks (a Note that only shares a post or publication) count too and are flagged with isRestack.

## `includeFullText` (type: `boolean`):

One extra request per free post adds bodyText (the whole article as plain text). Paywalled posts get the public preview only and bodyTruncated = true; the scraper never bypasses a paywall.

## `onlyFreePosts` (type: `boolean`):

Skip paywalled posts (audience 'only_paid' or 'founding'). Skipped posts are not charged.

## `sortPosts` (type: `string`):

new = newest first (the archive order), top = most liked first.

## `publishedAfter` (type: `string`):

Return only posts and Notes published on or after this date, for example 2026-01-01. With 'Newest first' the scraper stops reading as soon as it reaches older posts, so scheduled runs stay cheap.

## `maxItems` (type: `integer`):

Stop after this many posts and Notes in total. You are charged per item, so this also caps the cost of the run.

## `proxyConfiguration` (type: `object`):

Not needed: Substack's public JSON endpoints answer plain requests. Enable Apify Proxy only if a run reports 403 or 429 errors.

## Actor input object example

```json
{
  "publications": [
    "https://www.lennysnewsletter.com/",
    "noahpinion"
  ],
  "profiles": [
    "lenny"
  ],
  "maxPostsPerPublication": 20,
  "maxNotesPerProfile": 20,
  "includeFullText": false,
  "onlyFreePosts": false,
  "sortPosts": "new",
  "maxItems": 100
}
```

# Actor output Schema

## `results` (type: `string`):

One row per result in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publications": [
        "https://www.lennysnewsletter.com/",
        "noahpinion"
    ],
    "profiles": [
        "lenny"
    ],
    "maxPostsPerPublication": 20,
    "maxNotesPerProfile": 20,
    "maxItems": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("kamerozkan/substack-scraper-posts-notes").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "publications": [
        "https://www.lennysnewsletter.com/",
        "noahpinion",
    ],
    "profiles": ["lenny"],
    "maxPostsPerPublication": 20,
    "maxNotesPerProfile": 20,
    "maxItems": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("kamerozkan/substack-scraper-posts-notes").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publications": [
    "https://www.lennysnewsletter.com/",
    "noahpinion"
  ],
  "profiles": [
    "lenny"
  ],
  "maxPostsPerPublication": 20,
  "maxNotesPerProfile": 20,
  "maxItems": 100
}' |
apify call kamerozkan/substack-scraper-posts-notes --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,kamerozkan/substack-scraper-posts-notes"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/2cQ9aBKApM4Syaag6/builds/Q4weVXGQ6bzToyF6D/openapi.json