Substack Posts Scraper — Any Newsletter, No API Key

Scrape any Substack newsletter — by handle or custom domain — and get one row per post: title, author, publish date, subtitle, cover image and word count. Optional full text.

No API key, no account. $0.004 per post — a publication with no feed is recorded and not charged.

Input

{ "publication": "bigtechnology.com", "maxPosts": 20 }

Bulk, mixing address shapes:

{
  "publications": ["platformer", "bigtechnology.com", "https://stratechery.com"],
  "maxPosts": 10,
  "includeFullText": true
}

Field	Type	Notes
publication	string	A handle (platformer), a custom domain (bigtechnology.com) or a full URL.
publications	string[]	Up to 20 newsletters per run.
maxPosts	integer	1–50. A Substack feed carries roughly the 20 most recent. Charged per post.
includeFullText	boolean	Adds the article body as plain text. Default false.

Output

{
  "ok": true,
  "publication": "bigtechnology.com",
  "publicationTitle": "Big Technology",
  "feedUrl": "https://www.bigtechnology.com/feed",
  "title": "When Artificial Intelligence Is Too Valuable To Sell",
  "author": "Alex Kantrowitz",
  "link": "https://www.bigtechnology.com/p/when-artificial-intelligence-is-too",
  "publishedAt": "2026-07-31T19:30:17.000Z",
  "subtitle": "Just because the AI labs have sold models on a meter until now…",
  "imageUrl": "https://substackcdn.com/image/fetch/…",
  "wordCount": 1840
}

What it gets right

• ⭐ Custom domains actually work. Most established newsletters publish on their own domain, and the obvious URL fails on them. Measured: bigtechnology.com/feed returns 404, while www.bigtechnology.com/feed and bigtechnology.substack.com/feed both return 20 items. This resolves by trying each candidate and keeping the first that yields posts, and tells you which one it used in feedUrl. A handle-only implementation silently returns nothing here.
• A 200 is not success. A custom domain can answer with a marketing page; a response with zero items is not accepted as the feed.
• Substack needs a browser user-agent. A bare client gets an empty body — measured, and the reason the request sets one.
• Dates are ISO-8601, not the raw RFC-822 string.
• wordCount comes from the real body (content:encoded), so you can sort long reads from short notes without pulling every article.
• Text fields are clean — tags stripped, entities decoded in the correct order.

Pricing

$0.004 per post returned. No start fee. A publication with no reachable feed produces an ok: false record naming the addresses tried and is not charged.

Limits

• A Substack RSS feed carries roughly the 20 most recent posts. There is no public archive endpoint, so older posts are not reachable this way — maxPosts above ~20 simply returns what exists.
• Paywalled posts appear in the feed with their public preview only; includeFullText returns that preview, not the paid body.
• 20 publications per run.

FAQ

Does it work with custom domains? Yes — that is the case this was built for. See above.

Can I get the whole archive? No. Substack's feed only exposes recent posts.

Do I need a Substack account? No.

Changelog

• 0.1 (2026-09-10) — first release. One row per post, custom-domain resolution with fallback, word counts, optional full text, bulk publications.