Substack Posts Monitor

Post list for Substack publications, read from each publication's public archive list: title, subtitle, URL, publish date, free or paid (audience), like count, comment count, restack count, word count, authors and tags. With monitoring on, later runs return only the posts that were not there before.

Paste publications as https://name.substack.com, the publication's own domain (https://www.lennysnewsletter.com), a post link, or just the name before .substack.com.

No login, no cookies, no subscription.

What you get

One row per post, newest first:

Column	Meaning
publication, publicationId	The host that answered (after a redirect to a custom domain, the custom domain) and Substack's numeric ID of the publication
title, subtitle, postUrl, slug, postId	The post
postDate	Publish date and time (UTC, ISO 8601)
audience, isPaywalled	everyone for a free post, only_paid (or another value Substack gives) for a paid one; isPaywalled is true when audience is not everyone
reactionCount, commentCount, restackCount	Likes, comments and restacks at the moment of the read
wordCount	Word count of the full post as Substack reports it
authors, authorHandles, tags, sectionName, type, language	Bylines, tags, section, post type (newsletter, podcast, ...) and language
previewText	The short preview text in the public list (the opening lines)
coverImage, podcastDurationSeconds	Cover image URL; duration for audio posts
input, scrapedAt, status	What you typed, when it was read, ok for a post row

Monitoring columns (filled only in monitoring mode): change (first-check or new-post), isFirstCheck, previousCheckedAt.

A value that is not in the list is null, never 0.

The rows are the posts the publication's own archive list returns. If a publication keeps some post types out of that list, they are not in the rows.

Input

• publications - one per line. https://name.substack.com, name.substack.com, name, a custom domain, or a post link from the publication. A name.substack.com address that redirects to a custom domain is followed. Profile links (substack.com/@name) are not publications and come back as an invalid-input row.
• maxPostsPerPublication - newest first, stop after this many posts (default 50, up to 5,000).
• postedAfter - optional date YYYY-MM-DD (UTC). Older posts are not returned and reading stops when the archive reaches them.
• monitoringMode, resetMonitoringState - see below.
• useProxy - requests go directly to each publication; a residential proxy is tried once per refused request, at most three times in a run, only when a site answers HTTP 403 or 429. After a refusal the run stops sending requests and lists what it did not read.

Monitoring mode

Turn monitoringMode on and schedule the Actor. It remembers, per publication, the posts it returned and, on later runs:

• a publication with no new post returns no post row; it still costs one check (see "What is charged"), and if no publication had a new post you get one free no-new-posts row saying how many were compared;
• a new post comes back once, as change: "new-post";
• a publication it has never seen returns its newest posts once (first-check, up to maxPostsPerPublication). Posts older than the oldest post returned on that first run are treated as already there and are never returned as new - also when the first run was cut short by a charge limit or a failed page. To start over, run once with resetMonitoringState.

On later runs new posts are not cut by maxPostsPerPublication. A change in likes, comments or restacks alone does not return a post again; the counts are simply reported in every returned row. A post that is published with a date earlier than the oldest post of the first run is not reported as new.

Publications are remembered one by one (by Substack's publication ID), so adding a publication to the list does not return the others again. Do not put the same publication in two schedules that run at the same moment: the platform's key-value store has no atomic update, so two runs finishing together can overwrite each other's record and return the same post twice.

What is charged

• post-returned - one per post row.
• publication-checked - monitoring mode only: one per publication whose archive was read and has at least one post, whether or not it had a new post.

There is no start fee and no monthly fee. Rows that explain why nothing was returned are free: no-such-publication, not-a-substack-archive, robots-disallowed, invalid-input, duplicate-input, unreadable, blocked, no-posts, no-new-posts, incomplete, not-read, budget-reached. A publication that could not be read is not charged a check.

If you set a maximum total charge for a run, the Actor stops reading before it would exceed it, returns the posts that fit, and adds a free budget-reached row and a not-read row naming the publications it did not read. In monitoring mode a publication is read only if the limit has room for its check plus one post row.

How it reads a publication

1. GET https://<publication>/robots.txt. If the rules for all crawlers (User-agent: *) disallow /api/v1/archive, the archive is not requested and you get a free robots-disallowed row.
2. GET https://<publication>/api/v1/archive?sort=new&offset=N&limit=M (at most 50 per request), newest first, until the number you asked for, the postedAfter date, an empty page, or - in monitoring mode - the posts it already returned. At most 150 archive requests per publication.

Requests carry an honest User-Agent (neverempty-substack-posts-monitor) and are sent one at a time with a pause between them. A refused request (HTTP 403 or 429) is sent once more through a residential proxy when useProxy is on, and is otherwise not retried; after a refusal the run stops. Individual post pages, comments, private feeds and anything behind a login are not requested.

Rows that are not posts

status	When
no-such-publication	The address answered HTTP 404 for the archive: a misspelled name, a removed publication, or a site that is not on Substack
not-a-substack-archive	The address answered, but not with a Substack post list
robots-disallowed	The publication's robots.txt disallows the archive request
blocked	HTTP 403 or 429, or a verification page. The run stops sending requests
unreadable	No answer, a server error after one retry, or an answer in an unexpected format
incomplete	A later page of a publication's archive could not be read; the posts read before it are returned
no-posts	The archive is empty, or has no post on or after postedAfter
no-new-posts	Monitoring: nothing new in any publication
invalid-input, duplicate-input, not-read, budget-reached	Input lines that were not read, and why

Notes

• Post bodies are not returned, for paid or free posts: a row carries what the public archive list shows for that post, including the short preview text.
• Like, comment and restack counts are the publication's public figures at the moment of the read; they keep moving on live posts.
• This Actor is not affiliated with or endorsed by Substack. You are responsible for how you use the data; respect Substack's Terms of Use and the rights of the writers. The data returned here is the public listing data only, not the posts themselves.