Substack Scraper - Posts, Metrics & New-Post Alerts

For newsletter researchers, competitor analysts and content curators who want a publication's public post list as structured data. Give it one or more Substack publication URLs (including custom domains) and get one row per post: title, subtitle, date, free/paid audience, word count, reactions, comment count and author. No login, cookies or proxies. Turn on new-post alerts (onlyNewSinceLastRun) and run it on a schedule to receive only posts you have not been given before.

Pay-per-result: $0.0015 per post — 1,000 posts = $1.50. You are charged only for rows that land in your dataset.

Quick start

1. Open the Input tab and use the prefill below, or replace the URL with the publications you follow.
2. Click Start.
3. Open the dataset and export it as CSV, Excel, JSON or XML.

{
  "publicationUrls": [
    "https://www.lennysnewsletter.com"
  ],
  "maxPagesPerSearch": 2,
  "maxItems": 20,
  "onlyNewSinceLastRun": false,
  "stateStoreName": "substack-scraper-seen-posts"
}

Pass publication home URLs in publicationUrls (up to 50). The Actor stays on the origin you supply and reads the public /api/v1/archive JSON, falling back to the public /feed only when the first archive request returns 404. maxPagesPerSearch defaults to 2 (range 1-10) and maxItems defaults to 100 (hard maximum 5,000), so maxItems doubles as your cost cap.

Example output

One flat row per post. This row is produced by the Actor's parser from a captured Lenny's Newsletter archive response:

{
  "id": "215694124",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "postDate": "2026-09-29T13:15:57.512Z",
  "audience": "free",
  "wordcount": 1276,
  "reactions": 271,
  "commentCount": 6,
  "author": "Lenny Rachitsky"
}

Every row also carries searchTerm, page, position, sourceUrl and scrapedAt. Any value Substack does not publish is null; nothing is guessed.

Use cases

• Newsletter research: export a publication's recent catalogue with audience, word count and engagement to see what topics and formats work.
• Competitor monitoring: watch rival newsletters on a schedule and get only newly published posts, with posting cadence, authors, reactions and comments.
• Content curation: build a weekly digest or reading list from the newsletters you follow, straight into a sheet, Slack or email.
• Sponsorship and partnership scouting: compare free-versus-paid mix and engagement across publications before you reach out.

New-post alerts

1. Add your publication URLs, then set onlyNewSinceLastRun to true.
2. Click Schedule on the run page (or create one under Schedules in the Apify Console). Daily works well for most newsletters.
3. The first run is a baseline: it delivers (and charges for) the posts currently listed and remembers their ids per publication. Every later run delivers only ids it has not delivered before.
4. Add a webhook or integration (Slack, email via Zapier/Make, Google Sheets) on run success to forward the new rows.

A run with nothing new still succeeds with an empty dataset and costs nothing.

Get new Substack post alerts in Discord, Telegram, Slack, Teams, Google Chat, your phone (ntfy/Pushover) or any app (webhook)

Want a Substack new post alert in Discord, Telegram or on your phone instead of checking a dataset? Add one or more destinations in the Actor's Notifications (optional) section and each run sends the posts it delivered. It is a monitor that notifies you: it only reads public publication archives, and it never subscribes, comments or logs in to Substack.

Channel	Fill in	What you get
Discord	discordWebhookUrl	One card per post, 10 per message
Telegram	telegramBotToken + telegramChatId	Up to 10 posts per message, one line each
Slack	slackWebhookUrl	Block Kit message, up to 10 posts per message
Mattermost, Rocket.Chat	slackCompatibleWebhookUrl	Slack-style incoming webhook: a Markdown list (these apps do not render Slack Block Kit)
Microsoft Teams	teamsWebhookUrl	Adaptive Card from a Teams Workflows webhook, up to 10 posts per card
Google Chat	googleChatWebhookUrl	Card in a space, up to 10 posts per message
ntfy (phone push, no account)	ntfyTopic (optional ntfyServer, ntfyToken)	A push per post, up to 10, then one "...and N more"
Pushover (phone push)	pushoverUserKey + pushoverAppToken	A push per post, up to 10, then one "...and N more"
Any app: Zapier, Make, n8n, Pipedream, IFTTT, Home Assistant	webhookUrl	One JSON POST per run with every delivered row

Fill in as many as you like; each channel works on its own and a failing one never stops the others.

Setup, channel by channel

• Discord (1 minute): Server Settings → Integrations → Webhooks → New Webhook → pick the channel → Copy Webhook URL.
• Telegram (2 minutes): chat with @BotFather, send /newbot, paste the token into telegramBotToken. Add the bot to the chat or channel (administrator for a channel; for a chat with yourself press Start on the bot). telegramChatId is a numeric id (123456789, or -1001234567890 for a group or channel) or a public @channelusername.
• Slack: at api.slack.com/apps create an app (From scratch) → Incoming Webhooks → switch it on → Add New Webhook to Workspace → pick the channel → copy the https://hooks.slack.com/services/... URL.
• Mattermost or Rocket.Chat: create an incoming webhook in the app and paste its public https:// URL into slackCompatibleWebhookUrl. It must be reachable from the internet: localhost, private IP addresses and .local / .internal names are rejected.
• Microsoft Teams: in the channel open ... → Workflows → Post to a channel when a webhook request is received → finish the wizard and copy the URL (a host ending in .logic.azure.com or .powerplatform.com). The old "Office 365 connector" webhooks are retired and not supported.
• Google Chat: in the space open Apps & integrations → Webhooks → Add webhook, name it and copy the https://chat.googleapis.com/v1/spaces/... URL (needs a Google Workspace account that allows webhooks).
• ntfy (free, no account): install the ntfy app (or open ntfy.sh), subscribe to a topic with a long random name, and put the same name in ntfyTopic. Anyone who knows the topic can read it, so make it unguessable. Own server or a protected topic: set ntfyServer (public https:// address) and ntfyToken.
• Pushover: copy your user key from pushover.net into pushoverUserKey, create an application at pushover.net/apps/build and put its API token in pushoverAppToken. Pushover itself is a separate paid app after its trial: check their pricing.
• Any app (webhook): paste the catch-hook URL from Zapier, Make, n8n, Pipedream, IFTTT or Home Assistant into webhookUrl. It must be a public https:// address.

The recipe: a Substack new-post alert in five steps

1. Add the publication URLs you follow to publicationUrls.
2. Turn on onlyNewSinceLastRun.
3. Add your destination(s) from the table above.
4. Run it once. The first run is the baseline and sends the posts currently listed (up to maxNotifications).
5. Create a Schedule (daily works for most newsletters). From then on you only hear about posts it has not delivered before.

{
  "publicationUrls": [
    "https://www.lennysnewsletter.com"
  ],
  "maxItems": 100,
  "onlyNewSinceLastRun": true,
  "discordWebhookUrl": "https://discord.com/api/webhooks/<id>/<token>",
  "ntfyTopic": "your-long-random-topic-name",
  "maxNotifications": 25
}

The generic webhook (webhookUrl)

Each run sends one POST per 100 delivered rows (so usually one) with this body, followed by whatever your tool does with it:

{
  "source": "apify",
  "actor": "substack-scraper",
  "runId": "HG7ML7M8z78YcAPEB",
  "count": 2,
  "total": 2,
  "part": 1,
  "parts": 1,
  "items": [
    {
      "...": "the full dataset rows"
    }
  ]
}

count is the number of rows in this POST, total the number delivered in the run, and part / parts number the POSTs when a run delivers more than 100 rows. maxNotifications does not cap the webhook: your tool gets every delivered row. Your endpoint should answer with a 2xx status; redirects are not followed and a failure is reported in the run status, never retried in a loop.

What each message contains

Each post is announced as New post: <title> — <author> with the link, the subtitle, publication date, free or paid audience, word count, reactions and comment count when Substack publishes them, and the publication's host name. Anything Substack does not publish (for example the counts on the feed fallback) is left out. Discord gets a card per post, Telegram one line per post, Slack Block Kit sections, Teams an Adaptive Card, Google Chat a card, and ntfy/Pushover a push that opens the post. Link previews are off, and titles are escaped so a title can never inject formatting or mentions.

What to expect

• Only delivered posts are sent. A run that finds nothing new sends nothing, so there is no "no new posts" spam. Rows the run could not deliver (for example because your spending limit was reached) are not sent.
• Without onlyNewSinceLastRun, every delivered row is sent, so a scheduled plain run would repeat the same posts: use alert mode for alerts (the message then says "New post"; a plain run says "Post").
• maxNotifications (default 25, max 100) caps how many posts go to chat and phone channels per run (the webhook is not capped). The rest are still in the dataset and the status message says so. ntfy and Pushover send at most 10 individual pushes, then one "...and N more" summary.
• Cost: notifications are free. You pay only the usual price for delivered posts, exactly as without alerts. There is no extra charge for any channel.
• Failures never fail the run. A deleted webhook or an unreachable service leaves the run successful and the dataset complete, and the status message says what happened, for example Notified: Discord 12, Slack 12, ntfy 10 (+summary). Teams failed (HTTP 404) - check the webhook URL. The logs show the channel name and HTTP status only, never a URL or token.
• Your URLs, tokens and ntfy topic are stored as secret inputs. Anyone holding a webhook URL can post to that channel, so treat it like a password.
• It is a monitor that notifies you. It never subscribes, comments or logs in to Substack. The messages go only to the destinations you set up.
• Nothing changes by default. With no destination set, a run behaves exactly as before and the dataset rows are identical.

Pricing

Pay-per-event: $0.0015 per delivered post — 1,000 posts = $1.50. Rows skipped by robots.txt, blocks, failed routes, or already-seen alert history are not delivered and are not charged. A run that returns nothing is not charged. Price changes follow Apify's notice rules; the Pricing tab always shows the rate you pay.

Notifications to Discord, Telegram, Slack, Teams, Google Chat, ntfy, Pushover and webhooks are free: you pay only for delivered posts, never for messages sent.

Limitations

• Metadata only, never post bodies. The Actor does not fetch individual posts, so article text is not included. Paywalled posts appear as metadata rows (audience is paid), but their bodies are never requested, and private or subscriber feeds are never touched.
• Robots.txt only. Every path is checked against the publication's robots.txt first; a disallowed path is skipped, not bypassed. Requests are spaced about three seconds apart with an identifying User-Agent. No login, CAPTCHA solving or proxies.
• Pages, not the full archive. Each archive page returns 12 posts, so the default 2 pages gives about 24 posts per publication and the maximum of 10 pages about 120. Older posts beyond that are not retrieved.
• Feed fallback has fewer fields. If a publication's archive endpoint returns 404, the Actor uses its public /feed (a single response). Feed rows have audience, wordcount, reactions and commentCount set to null because the feed does not publish them.
• Public publications on Substack's standard routes. A publication that exposes neither the archive JSON nor a public feed produces no rows and is reported in the run status.
• Engagement counts are a snapshot taken at run time, as published by Substack.
• Layout changes. If Substack changes its public routes, the run reports a clear error instead of returning bad rows.

Use it from your code, Make, Zapier or n8n

Built for automated pipelines: small runs, stable field names, one row per item, and you pay only for rows actually delivered.

One API call that runs the Actor and returns the data (replace $APIFY_TOKEN with your token from Apify Console → Settings → Integrations):

curl -s -X POST "https://api.apify.com/v2/acts/lukehunter~substack-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"publicationUrls": ["https://www.lennysnewsletter.com"], "maxPagesPerSearch": 2, "maxItems": 20}'

JavaScript (npm i apify-client):

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('lukehunter~substack-scraper').call({
  "publicationUrls": [
    "https://www.lennysnewsletter.com"
  ],
  "maxPagesPerSearch": 2,
  "maxItems": 20
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

Python (pip install apify-client):

import os
from apify_client import ApifyClient
client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('lukehunter~substack-scraper').call(run_input={"publicationUrls": ["https://www.lennysnewsletter.com"], "maxPagesPerSearch": 2, "maxItems": 20})
items = client.dataset(run['defaultDatasetId']).list_items().items

Schedules, webhooks and no-code tools: save your input as a Task, add a Schedule (e.g. hourly or daily), and attach a webhook or one of the Integrations tab's connectors (Make, Zapier, n8n, Google Sheets, Slack) to push each run's results into your own system.

Only new items each run: set "onlyNewSinceLastRun": true on a scheduled Task and every run after the first delivers, and charges for, only items it hasn't delivered before for that same input. That makes it a ready-made change feed.

Reliability: an automated test runs this Actor's parser against the live site every day, and failures are fixed fast.

Feedback: an honest rating on the Store page, good or bad, helps others decide if this Actor fits their needs. Missing a field or found a problem? Open an issue on the Actor page.

FAQ

How much does it cost? $0.0015 per delivered post, so 1,000 posts cost $1.50. maxItems is a hard cap on spend.

Can I monitor a newsletter for new posts on a schedule? Yes. Turn on onlyNewSinceLastRun and attach an Apify Schedule. The first run is a baseline; later runs deliver only unseen posts.

Does it work with custom domains? Yes. Use the publication's home URL, whether it is name.substack.com or its own domain; the Actor only builds the public archive and feed routes on that origin.

Does it read paywalled content? No. It lists paid posts as metadata only and never requests their bodies.

What export formats are available? Anything the Apify dataset supports: CSV, Excel, JSON, XML, RSS and HTML table, plus the API.

Can I get Substack post alerts in Discord, Telegram, Slack or on my phone? Yes. Set discordWebhookUrl, telegramBotToken + telegramChatId, slackWebhookUrl, teamsWebhookUrl or googleChatWebhookUrl, or use ntfyTopic (free, no account) or pushoverUserKey + pushoverAppToken for phone pushes. Mattermost and Rocket.Chat work through slackCompatibleWebhookUrl. With onlyNewSinceLastRun on and a schedule, each run sends only the posts it has not delivered before. Setup steps for each are in the alerts section above. Notifications cost nothing extra.

Can I get Substack post alerts by email? Not natively. There are two routes. (1) Set webhookUrl to a Zapier, Make or n8n catch hook and add an email or Gmail step: the webhook delivers every row as JSON, so the email can list the posts. (2) Apify's own run integrations and run notifications can email you when a run finishes, but they only send a link to the run or dataset, not the posts themselves. For instant alerts without building anything, Telegram, ntfy or Pushover reach your phone directly.

Can I get Substack post alerts in WhatsApp? Not natively, and we do not use unofficial WhatsApp APIs, which risk getting your number banned. The official route is the WhatsApp Business Platform (Cloud API) with its own business account, message templates and pricing: point webhookUrl at a Zapier, Make or n8n flow that calls it. For free phone alerts use Telegram or ntfy instead.

More from this publisher

• Vinted Scraper & New Listing Alerts — public Vinted search listings with new-item alerts.
• Vinted Cross-Country Prices — compare public Vinted asking prices across countries.
• Reed Jobs Scraper — public UK job search listings.
• Jobindex Jobs Scraper — public Denmark job search listings.
• Pracuj.pl Jobs Scraper — public Poland job search listings.