Substack Scraper – Posts, Engagement & Metadata

Pull every post from any Substack publication's archive as clean JSON: title, subtitle, publish date, word count, reaction/comment/restack counts and the public post URL, no login and no browser. Counts only — comment and reaction text is not included. Built for researchers and journalists tracking a beat across newsletters, RAG/LLM ingestion pipelines that need a structured feed of a publication's back catalog, market/media analysts watching engagement trends across writers, and dashboards and alerting that poll a publication for new posts on a schedule. Plain HTTPS calls against Substack's own archive API (/api/v1/archive) with retries and a silent-failure check, so a scheduled run keeps working even when one publication in the list turns out not to be on Substack at all.

What you get

• Every post as one row – id, title, subtitle, slug, canonical URL, publish date, type (newsletter/podcast/thread), audience, word count, reactions, comments, restacks, a short description/preview, and the raw Substack API fields for everything else (cover image, podcast metadata, section, SEO title/description, and more).
• Any publication, two ways to name it – a short name (astralcodexten), a custom domain, or a full URL; redirects are followed automatically, so astralcodexten.substack.com and its real domain www.astralcodexten.com each reach the same publication. Pick one form per publication: listing both in the same run fetches and bills it twice (see Pricing).
• Filters that keep a run's cost predictable – a per-publication post cap, a postedAfter date cutoff, and a whole-word searchTerm match on title/subtitle (so searching "NFL" never matches "inflation" the way a naive substring search would).

No personal data, ever

The Actor never outputs subscriber, contact or author-profile data. Substack's archive API returns a publishedBylines object per post (author name, id, profile photo, bio) — this Actor drops that field entirely, along with anything else in the raw response whose field name matches email, phone, subscriber, address, ip_address, password or token, before the row is ever written. What's left is publication-level content metadata: post titles, dates and engagement counts, the same thing you'd see browsing the publication's public archive page. That makes the output safe to store, join and hand to a model without a data-processing agreement.

Sample row

A real row from a default run against astralcodexten (https://www.astralcodexten.com):

{
  "publication": "astralcodexten",
  "host": "astralcodexten.substack.com",
  "id": 210677400,
  "title": "Your Book Review: The Tale Of Genji",
  "subtitle": "Finalist #8 in the Book Review Contest",
  "slug": "your-book-review-the-tale-of-genji",
  "canonical_url": "https://www.astralcodexten.com/p/your-book-review-the-tale-of-genji",
  "post_date": "2026-09-04T19:34:24.551Z",
  "type": "newsletter",
  "audience": "everyone",
  "wordcount": 9885,
  "reaction_count": 212,
  "comment_count": 106,
  "restacks": 8,
  "description": "Finalist #8 in the Book Review Contest",
  "truncated_body_text": null,
  "cover_image": "https://substackcdn.com/image/fetch/...jpeg",
  "podcast_url": null,
  "podcast_duration": null,
  "section_name": null,
  "scraped_at": "2026-09-10T18:24:12+00:00"
}

Every row also carries the rest of Substack's own archive fields (SEO title/description, section, cover-image flags, podcast metadata and so on) — 65 fields per post before any are dropped for privacy. description and truncated_body_text (a short plain-text preview) are always included; the full article isn't.

Full article text (opt-in)

Substack's archive listing can include body_html (rendered article HTML) and body_json (the same content as structured rich-text JSON) — the publication's actual copyrighted words, not just metadata about the post. Both are off by default. Turn on includeBodyHtml to add them to every row. Note that Substack's archive endpoint itself only populates full body content for some posts/publications; when it's empty there, this Actor can't invent it either.

Filters

Input	Default	What it does
publications	["astralcodexten", "slowboring", "garymarcus"]	Publications to scrape: short name, custom domain, or full URL.
maxPostsPerPublication	100	Stop after this many posts per publication (newest first).
postedAfter	""	Keep only posts published on or after this ISO date.
searchTerm	""	Keep only posts whose title or subtitle contains this whole word/phrase (case-insensitive, word-boundary match — not a naive substring).
includeBodyHtml	false	Add body_html and body_json (full article content) to each row.

Pricing

Pay per event, no subscription: $1 per 1,000 posts. A default run (3 publications, up to 100 posts each) costs a few cents to a few dimes depending on how many posts each publication actually has. A publication that isn't reachable or isn't on Substack produces a single unbilled error row, so one bad entry among several never wastes budget on the rest — the run finishes and you get the posts from the good ones. If every publication in your list fails, the run is marked failed rather than quietly returning nothing, and you are charged for nothing. Error rows are never billed either way.

Listing the same publication twice in two different forms (example and www.example.com) counts as two publications and is fetched, and billed, twice; the Actor can only tell they are the same after following the redirect. List each publication once.

Use it from your tools

• API and SDKs – call it via the Apify API or the official Python/JavaScript clients: one call to start the run, one to fetch the post list as JSON or CSV.
• Schedules – run it daily or weekly inside Apify and pipe newly-published posts into Google Sheets, a webhook or your own storage.
• n8n, Make and Zapier – trigger runs and feed extracted posts into a workflow through Apify's integration for each.
• AI agents and MCP – eligible for agentic use via Apify's MCP server with pay-per-event pricing, so an agent can call it mid-task and pay only for the posts it actually pulls.

Details

• Source: https://<publication-host>/api/v1/archive, Substack's own public archive endpoint, the same one the publication's website uses to render its archive page. No authentication, no proxies, no browser.
• Not a Substack publication – if a given domain returns something other than a JSON post list (an HTML page, a 404), the Actor never crashes; it logs an unbilled error row categorized network, rate_limit, blocked, not_found or other and continues with the rest of publications.
• Reliability: 429 and 5xx responses are retried with exponential backoff (5 tries); 404 is never retried; a 200 response that isn't actually a post list counts as a failure, not a silent empty success.
• Run stats: the STATS record in the run's key-value store holds post/error counts and per-category request-error counts.
• Politeness: 0.2 s between requests, 30 s timeout, redirects followed, a User-Agent that names the Actor.
• Not affiliated with Substack. Built against Substack's own public archive API; no login, no paywall bypass, no subscriber data.