Substack Scraper (posts, newsletters, no login)

You get public Substack posts as clean JSON/CSV: title, date, free or paid status, word count, reactions, restacks, comment count, authors and the full text of free posts. No Substack account, no API key. Price: $4.50 per 1,000 posts (pay per event: post = $0.0045). Same price with or without text. Error rows are free.

Why use it

• Competitor and niche monitoring: track what rival newsletters publish and which posts get the most reactions and comments.
• LLM and research datasets: collect full text of free posts for summaries, topic models and RAG (respect author copyright).
• Sponsorship and creator research: compare posting frequency, engagement and authors across newsletters found by keyword.

Input

Field	Description
publications	Subdomain, host, custom domain or URL. A post URL (.../p/some-post) scrapes just that post
searchQuery	Optional keyword. The top matching newsletters are added to the run
maxSearchPublications	Newsletters taken from the search (default 5, max 25)
maxPosts	Max posts per newsletter, newest first (default 50, max 10000)
since	Optional ISO date such as 2026-01-01. Older posts are skipped
includeContent	true adds contentHtml and text for free posts (one extra request per post)

{
  "publications": ["astralcodexten", "https://www.lennysnewsletter.com"],
  "searchQuery": "machine learning",
  "maxSearchPublications": 3,
  "maxPosts": 100,
  "since": "2026-06-01",
  "includeContent": true
}

Error row example: { "publication": "nope-xyz.substack.com", "error": "Publication not found or not reachable (HTTP 404)." }

Example output

publication	title	date	audience	wordCount	reactionCount	restacks	commentCount	author	url
astralcodexten.substack.com	Open Thread 456	2026-10-19T17:51:46.436Z	free	422	50	1	259	Scott Alexander	https://www.astralcodexten.com/p/open-thread-456

Every row also has subtitle, slug, type, reactions (per emoji), authors (name, handle, url), coverImage, tags, scrapedAt, and with includeContent on free posts contentHtml and text.

{
  "publication": "astralcodexten.substack.com", "title": "Open Thread 456", "slug": "open-thread-456",
  "url": "https://www.astralcodexten.com/p/open-thread-456", "date": "2026-10-19T17:51:46.436Z",
  "type": "newsletter", "audience": "free", "wordCount": 422,
  "reactionCount": 50, "reactions": { "❤": 50 }, "restacks": 1, "commentCount": 259,
  "authors": [{ "name": "Scott Alexander", "handle": "astralcodexten", "url": "https://substack.com/@astralcodexten" }],
  "author": "Scott Alexander", "tags": [], "scrapedAt": "2026-10-20T05:56:37.786Z",
  "text": "This is the weekly visible open thread..."
}

For a paid post, audience is paid and text / contentHtml are not included.

Use with AI agents / MCP

Call this actor from any AI agent through the Apify MCP server (https://mcp.apify.com) or the Apify API, using the actor id TuqVqmgph62ZaatCR. An agent can ask "which posts about machine learning got the most reactions this month?" using searchQuery and since. Output is flat JSON with stable keys (publication, slug), so an agent can sort by reactionCount, filter by audience or summarise text directly.

Sample tool input an agent can send:

{
  "publications": ["astralcodexten"],
  "searchQuery": "machine learning",
  "maxSearchPublications": 3,
  "maxPosts": 20,
  "since": "2026-09-01",
  "includeContent": false
}

Limits and notes

• Public data only. Paid posts return metadata only: the paywall is never bypassed and the actor never logs in.
• No comments, subscriber counts or emails (not public). You get the comment count only.
• Podcasts, videos and threads can lack a word count.
• About one request per second, no proxies needed for normal volumes. Set a maximum cost per run in the run options; the actor stops cleanly when it is reached.

FAQ

Can it read paid posts? No, metadata only.

How do I pick a newsletter? Use its subdomain, its name.substack.com host, its custom domain, or paste any post URL.

How far back can it go? To the first post in the public archive. Use maxPosts and since to limit the run.

Disclaimer

This is an unofficial tool. It is not affiliated with, endorsed by or sponsored by Substack. It collects only publicly available data from Substack's public web endpoints. You are responsible for using the data in line with Substack's terms, the authors' copyright and applicable law (for example GDPR). Full post text belongs to its authors; use it for research and analysis, not republishing. Substack can change its endpoints at any time, which may temporarily break the actor.

AI disclosure: this actor was built with AI assistance (Claude by Anthropic). Maintained by tinygrid. Issues and feature requests are welcome in the Issues tab.