All notable changes to the Substack Posts Scraper are listed here. The version number is the
version field of .actor/actor.json.
First release.
- Scrapes posts from any number of Substack publications given as URLs, subdomains or custom
domains; follows publications that moved to a custom domain.
- Uses Substack's public JSON endpoints (
/api/v1/archive, /api/v1/posts/{slug}) plus the RSS
feed for the publication name; no login, no HTML parsing of post pages.
- Output per post: publication, IDs and URLs, title/subtitle/description, type, audience and a
derived
paywalled flag, publish date, authors (name + handle only), cover image, word count,
reaction/comment/restack counts, podcast audio URL and duration, scrape timestamp.
- Optional
includePostBody adds bodyHtml and bodyText (public preview only for paywalled
posts).
- Options:
maxPostsPerPublication (0 = all), sort (new/top), publishedAfter (absolute or
relative date), postTypes filter, proxy configuration (Apify datacenter proxies by default).
- Politeness and reliability: max 3 concurrent requests, 300-800 ms jitter per publication,
exponential backoff on 429/5xx with up to 5 retries, clear per-publication error messages,
partial failures do not abort the run.
- Pay-per-event pricing: dataset items charged automatically, optional
actor-start event; the
crawler stops on its own once the maximum cost per run is reached.
- Safety nets: an archive page whose posts could not be stored is retried without losing posts,
a page that merely repeats the previous one ends the paging, and two input entries that
redirect to the same publication are scraped only once.
