Substack Scraper — Newsletters, Posts & Full Articles

Scrape Substack publications and posts in one clean, normalized, de-duplicated schema — with full article bodies.

Get titles, subtitles, authors, publish dates, audience (free/paid), wordcount, reactions, comments, sections, tags, cover images and the full article HTML. Discover publications by keyword, or scrape a list you provide.

Why not just use RSS? RSS gives you a headline and a link. This gives you structured metadata + full text + engagement signals across thousands of publications — ready for media monitoring, AI datasets, or sponsorship research.

What you get

• Publications & posts — metadata for every post, from any Substack publication.
• Full article bodies (body_html) — real content, not just teasers (for accessible posts).
• Engagement signals — reactions, comment counts, wordcount, audience type.
• Discovery — find publications by keyword/category, or paste your own list.
• Normalized + de-duplicated — one schema across every publication, deduped by post id.
• Provenance on every row — pub_domain, canonical_url, fetched_at, schema_version.

Output fields (26 per row)

pub_domain, pub_name, post_id, title, subtitle, slug, post_type, canonical_url, post_date, updated_at, audience, is_paid, wordcount, reactions_count, comments_count, section_name, section_slug, tags, description, body_html, body_text, podcast_url, cover_image, language, schema_version, fetched_at.

Sample row

{
  "pub_domain": "astralcodexten.com",
  "title": "AI Safety Is Not A Corporate Plot",
  "subtitle": "...",
  "slug": "ai-safety-is-not-a-corporate-plot",
  "post_type": "newsletter",
  "canonical_url": "https://www.astralcodexten.com/p/ai-safety-is-not-a-corporate-plot",
  "post_date": "2026-10-15T01:15:38+00:00",
  "audience": "everyone",
  "is_paid": false,
  "wordcount": 3148,
  "reactions_count": 414,
  "comments_count": 532,
  "body_html": "<p>...</p>",
  "schema_version": "1.0.0",
  "fetched_at": "2026-10-16T03:35:00+00:00"
}

Input

• Publications — Substack domains/URLs, e.g. ["slowboring.com", "lennysnewsletter.com"]. Leave empty to use the built-in seed list (30+ top publications).
• Discover by keyword — e.g. "artificial intelligence" to auto-find publications.
• Max posts per publication / Max publications / Max posts, Include full article bodies, Only free posts, Sort (new/top).

Use cases

• AI / LLM datasets — high-quality long-form text for training and RAG.
• Media monitoring & news aggregation — track writers, topics, and trends.
• Sponsorship & marketing research — find publications by niche and size.
• Competitive/creator research — engagement metrics across the creator economy.

Pricing

Pay per result — $1.00 per 1,000 posts (+ $0.005 per run start). Only pay for posts returned. Free plan covers testing.

Notes

• Public Substack data only; no login required.
• Paid/subscriber-only posts may expose limited body text (Substack paywalls the rest) — metadata is still captured.
• The bundled seed list is a curated starting point, not the full universe — add your own publications.

Sample output

{"pub_domain": "platformer.news", "title": "The AI bubble question", "post_type": "newsletter", "is_paid": false, "wordcount": 1832, "reactions_count": 214, "post_date": "2026-10-13T11:02:00Z"}

More actors by joey001

• ATS Jobs Scraper — Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Workday
• Multi-Board Jobs Scraper — RemoteOK, WeWorkRemotely, Arbeitnow, Adzuna, USAJobs in one schema
• Reed.co.uk Scraper — UK job listings, salaries, working options and apply links
• Threads Scraper — Posts, keyword search, profiles, engagement and media (no login)
• Telegram Channel Scraper — Public channel messages, views, media, forwards and replies
• Steam Scraper — Store search, full app details (price, genres, Metacritic) and reviews
• CoinGecko Scraper — Crypto prices, coin details, trending, search and history (no API key)
• Apple Podcasts Scraper — Shows and full episode feeds with audio URLs and durations
• SEC EDGAR Scraper — Filings, XBRL financials, Form 4 insider trades and full-text search
• Otodom Scraper — Poland real-estate listings (price, area, rooms, location)
• Otto.de Scraper — German marketplace products (price, brand, rating, availability)

Found this useful?

A quick review helps a lot and keeps the Actor maintained:

→ Leave a review (Reviews tab)