# Substack Newsletter Scraper (`romy/substack-newsletter-scraper`) Actor

Substack Newsletter Scraper extracts newsletter metadata and recent posts from Substack the leading platform for independent writers and paid newsletters. Search by keyword or browse top newsletters by category. No login required

- **URL**: https://apify.com/romy/substack-newsletter-scraper.md
- **Developed by:** [Romy](https://apify.com/romy) (community)
- **Categories:** News, Developer tools, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $30.00 / 1,000 publication scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### What does Substack Newsletter Scraper do?

**Substack Newsletter Scraper** extracts newsletter metadata and recent posts from [Substack](https://substack.com) — the leading platform for independent writers and paid newsletters. Search by keyword (e.g. "finance", "AI", "politics") or browse top newsletters by category. No login required. Runs on the Apify platform with scheduling, proxy rotation, and API access built in.

### Why use Substack Newsletter Scraper?

- **Market research**: Find the top newsletters in any niche with subscriber counts and growth signals
- **Content intelligence**: Track what topics top writers are covering and how audiences engage
- **Lead generation**: Identify influential newsletter authors for sponsorships or partnerships
- **Competitive analysis**: Monitor competitor newsletters and their paid vs free content strategy
- **Investment / media research**: Discover emerging media brands before they go mainstream

### How to use Substack Newsletter Scraper

1. Open the actor on Apify and click **Try for free**
2. Enter a **Search Query** (e.g. `finance`) or a **Category ID** (e.g. `153` for Finance, `4` for Technology)
3. Set **Max Publications** (how many newsletters to scrape) and **Max Posts per Publication**
4. Click **Start** — results appear in the **Output** tab in seconds

#### Common category IDs

| ID | Category |
|----|----------|
| 4 | Technology |
| 62 | Business |
| 153 | Finance |
| 96 | Culture |
| 76739 | U.S. Politics |
| 76740 | World Politics |
| 76741 | Health Politics |
| 103 | News |
| 134 | Science |
| 118 | Crypto |
| 18 | History |
| 94 | Sports |
| 11 | Music |
| 355 | Health & Wellness |
| 339 | Literature |
| 284 | Fiction |
| 13645 | Food & Drink |
| 109 | Travel |
| 114 | Philosophy |
| 34 | Education |
| 15414 | Climate & Environment |
| 15417 | Art & Illustration |
| 49715 | Fashion & Beauty |
| 61 | Design |
| 387 | Comics |
| 223 | Faith & Spirituality |
| 1796 | Parenting |
| 49692 | Humor |
| 51282 | International |
| 76782 | Film & TV |
| 76866 | Home & Garden |

### Input

```json
{
  "query": "artificial intelligence",
  "maxPublications": 50,
  "maxPostsPerPublication": 5
}
```

Or browse by category:

```json
{
  "categoryId": 153,
  "maxPublications": 100,
  "maxPostsPerPublication": 10
}
```

### Output

Each item in the dataset represents one newsletter with its recent posts embedded:

```json
{
  "id": 5076800,
  "name": "AI In Finance",
  "subdomain": "finstoryai",
  "base_url": "https://finstoryai.substack.com",
  "hero_text": "Helping CFOs turn numbers into stories...",
  "language": "en",
  "created_at": "2025-05-20T17:04:26.053Z",
  "payments_state": "enabled",
  "subscriber_count": "Over 12,000 subscribers",
  "subscriber_magnitude": 10000,
  "paid_subscriber_magnitude": 1000,
  "author_badge": {"type": "bestseller", "tier": 1000},
  "posts": [
    {
      "id": 208547842,
      "title": "Anthropic just released Claude Opus 5",
      "slug": "anthropic-just-released-claude-opus",
      "canonical_url": "https://finstoryai.substack.com/p/anthropic-just-released-claude-opus",
      "post_date": "2026-07-31T12:15:22.317Z",
      "audience": "everyone",
      "wordcount": 1137,
      "reaction_count": 56,
      "comment_count": 1,
      "restacks": 9,
      "tags": ["CFOs", "AI"],
      "authors": [{"name": "AI In Finance", "handle": "finstoryai"}]
    }
  ]
}
```

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

### Data table

| Field | Description |
|-------|-------------|
| `name` | Newsletter name |
| `subdomain` | Substack subdomain |
| `base_url` | Full newsletter URL |
| `subscriber_count` | Human-readable subscriber count string |
| `subscriber_magnitude` | Subscriber order of magnitude (100, 1000, 10000…) |
| `payments_state` | `enabled` = has paid tier |
| `author_badge` | Bestseller badge tier |
| `posts[].title` | Post title |
| `posts[].post_date` | Publication date (ISO 8601) |
| `posts[].audience` | `everyone` = free post, `only_paid` = paywalled |
| `posts[].reaction_count` | Likes/hearts count |
| `posts[].restacks` | Number of restacks (reposts) |

### Pricing

- **$0.05** per run start
- **$0.03** per newsletter scraped

Scraping 100 newsletters costs approximately **$3.05**. The free Apify tier gives you enough credits to test before subscribing.

### Tips

- Use `categoryId` instead of `query` to get the **top-ranked** paid newsletters in a category — these are sorted by subscriber count
- Set `maxPostsPerPublication: 0` to only collect newsletter metadata (faster, cheaper)
- Combine with scheduling to track growth: run weekly and compare subscriber magnitudes over time
- Substack does not rate-limit aggressively — proxy is optional for normal volumes

### FAQ & Disclaimers

**Is scraping Substack legal?** This actor only accesses publicly available data through Substack's official mobile API — the same data visible to anyone without logging in. No paywalled content is accessed.

**Why no full article text?** Full article body requires authentication. This actor returns metadata and previews only.

**Known limitations:** Subscriber count is a magnitude string (e.g. "Over 12,000"), not an exact number — Substack does not expose exact counts publicly.

**Support:** Open an issue in the Issues tab on this actor's page. Custom solutions available on request.

# Actor input Schema

## `query` (type: `string`):

Keyword to search for newsletters (e.g. 'finance', 'AI', 'politics'). Use this OR categoryId.

## `categoryId` (type: `string`):

Substack category ID to browse top paid newsletters. Use this OR query.

96=Culture, 4=Technology, 62=Business, 76739=U.S. Politics, 153=Finance, 13645=Food & Drink, podcast=Podcasts, 94=Sports, 15417=Art & Illustration, 76740=World Politics, 76741=Health Politics, 103=News, 49715=Fashion & Beauty, 11=Music, 223=Faith & Spirituality, 15414=Climate & Environment, 134=Science, 339=Literature, 284=Fiction, 355=Health & Wellness, 61=Design, 109=Travel, 1796=Parenting, 114=Philosophy, 387=Comics, 51282=International, 118=Crypto, 18=History, 49692=Humor, 34=Education, 76782=Film & TV, 76866=Home & Garden, bestseller=New Bestsellers

## `maxPublications` (type: `integer`):

Maximum number of newsletters to scrape.

## `maxPostsPerPublication` (type: `integer`):

Number of recent posts to fetch per newsletter. Set to 0 to skip posts.

## `proxyConfiguration` (type: `object`):

Proxy settings. Substack rarely blocks, but proxies help with rate limits.

## Actor input object example

```json
{
  "maxPublications": 25,
  "maxPostsPerPublication": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("romy/substack-newsletter-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("romy/substack-newsletter-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call romy/substack-newsletter-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,romy/substack-newsletter-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/3zl1dHyyiae4ksED3/builds/bG7cPd3a6uVxQtvgM/openapi.json