# Substack Publication Archive Scraper (`apt_marble/substack-publication-archive-scraper`) Actor

Export a Substack publication archive with post titles, dates, URLs, audience labels, reactions, comments and word counts.

- **URL**: https://apify.com/apt_marble/substack-publication-archive-scraper.md
- **Developed by:** [Hamza](https://apify.com/apt_marble) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.60 / 1,000 result records

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Publication Archive Scraper

Collect public posts from a Substack publication archive, including custom-domain publications. Export titles, dates, authors, engagement, sections, previews and every nonempty field returned with archive results. No individual-post detail pages are fetched.

### Start a run

Enter the publication's HTTPS root, such as `https://on.substack.com`, choose Maximum results and Maximum pages, then start.

```json
{
  "publicationUrl": "https://on.substack.com",
  "maxItems": 100,
  "maxPages": 20
}
```

| Input | Meaning |
| --- | --- |
| Publication URL | Public HTTPS publication root. Custom domains work when they serve a Substack archive. Post links, unrelated websites and private-network addresses are not valid publication roots. |
| Maximum results | Maximum unique posts to save, from 1 to 10,000. Default 100. |
| Maximum pages | Maximum archive requests, from 1 to 100. Default 20. |

An invalid URL or unavailable publication ends successfully with a clear explanation and the working example `https://on.substack.com`. Check the run summary if no posts were saved. An empty archive is also a valid successful outcome. A genuine service outage before any results or a storage failure can still fail the run.

### What the results contain

| Fields | Contents |
| --- | --- |
| `id`, `url`, `slug` | Post identifier and canonical post link. |
| `title`, `subtitle`, `description` | Returned headings and description. A description identical to the subtitle is omitted. |
| `postDate`, `audience`, `type`, `wordcount` | Publication date, audience, content type and word count. |
| `reactionCount`, `reactions`, `commentCount`, `childCommentCount`, `restacks` | Engagement totals and reaction breakdowns. |
| `authors` | Complete published-author information, including biography, avatar, publication links and returned account status. |
| `coverImage`, `coverImagePalette` | Cover media and its returned color palette. |
| `sectionId`, `sectionSlug`, `sectionName` | Section information. |
| `truncatedBodyText`, `bodyHtml`, `bodyJson` | Preview text or body fields only when the archive actually provides them. |
| `postTags`, `postCountryBlocks`, `audience`, permission fields | Tags, visibility and access metadata. |
| Podcast, video and voiceover fields | Available media URLs, upload objects, duration, episode information and flags. |
| Other archive fields | Every other nonempty field returned by the archive, including new fields. |
| `scrapedAt` | Collection time in UTC. |

Field names use readable camelCase, including nested author/media objects. Original record IDs keep their returned numeric or string type. Null values and empty strings are omitted; zero counts, false flags and arrays remain intact. Missing data is never estimated. Internal retrieval URLs are not added to datasets, summaries or logs.

Arrays and nested objects remain structured in JSON. CSV/Excel exports may represent them as JSON strings. Full article bodies are often absent from archive responses; this actor does not fetch post pages to fill those gaps or unlock paid content.

### Pagination and progress

Posts are saved individually as they arrive, with a result event for each saved unique post. Duplicate IDs within the run do not create another result. Progress lines show received and saved counts per page. Short pages are not assumed to be the end: the next offset advances by the actual number returned.

The run stops at Maximum results, Maximum pages, an empty page or a repeated page with no new records. A later-page error retains already-saved posts and ends successfully with a partial-result message. The run summary includes counts, completed pages, request count and the outcome.

# Actor input Schema

## `publicationUrl` (type: `string`):

Public HTTPS publication root, including custom domains. Working example: https://on.substack.com. Do not enter an individual post URL.

## `maxItems` (type: `integer`):

Global record cap. A cap is not a promise that this many records exist.

## `maxPages` (type: `integer`):

Maximum archive requests. The actor stops at this cap or earlier if results are exhausted.

## Actor input object example

```json
{
  "publicationUrl": "https://on.substack.com/",
  "maxItems": 100,
  "maxPages": 20
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "publicationUrl": "https://on.substack.com/"
};

// Run the Actor and wait for it to finish
const run = await client.actor("apt_marble/substack-publication-archive-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "publicationUrl": "https://on.substack.com/" }

# Run the Actor and wait for it to finish
run = client.actor("apt_marble/substack-publication-archive-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "publicationUrl": "https://on.substack.com/"
}' |
apify call apt_marble/substack-publication-archive-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,apt_marble/substack-publication-archive-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/GqmCHuXgi0ZZua1xU/builds/RJgVia4k1Qe8T2WeW/openapi.json