Substack Posts Scraper - Newsletter Content and Publication Data

Turn public Substack newsletter posts into structured data without copying articles by hand. Provide one or many publication URLs and receive recent post titles, authors, publication dates, summaries, categories, images, canonical URLs, and full public content when the feed exposes it. The Actor is designed for newsletter monitoring, content research, media intelligence, creator discovery, and scheduled archives.

The scraper reads each publication's public RSS feed, so it does not require a Substack login, browser automation, cookies, or an API key. This low-overhead approach is fast enough for recurring workflows and keeps the default run inexpensive. Download the resulting dataset as JSON, CSV, Excel, XML, or RSS, or connect it to the Apify API, Make, Zapier, n8n, Google Sheets, and your own database.

What the Actor does

• Accepts Substack home URLs, custom publication domains, or direct feed URLs
• Scrapes multiple publications in one run
• Returns the newest posts from each publication
• Extracts full public HTML and clean plain text when available
• Captures authors, dates, categories, images, summaries, and URLs
• Adds publication-level title, description, and URL to every row
• Deduplicates repeated posts across supplied feeds
• Fails clearly when no public posts are available

Data extracted

Each result represents one newsletter post and can include:

• post_id - public feed identifier or canonical URL
• title - post headline
• url - direct public post URL
• published_at - normalized publication timestamp
• author - source-provided author name
• subtitle - readable summary or preview
• content_text - clean full-text content when enabled and publicly provided
• content_html - source HTML when enabled
• categories - feed categories or tags
• image - featured image or first public article image
• publication_title and publication_description
• publication_url, requested URL, and resolved feed URL
• source and scrape timestamp

The output reflects what the publication makes available in its public feed. Some publishers provide a complete article; others provide only an excerpt. The Actor does not bypass paywalls or fabricate omitted content.

Input

Field	Description	Default
publicationUrls	Publication home URLs, custom domains, or direct public feeds	astralcodexten.substack.com
maxPostsPerPublication	Maximum recent posts saved for each publication	10
includeContent	Include public HTML and plain-text content	true

Example input

{
  "publicationUrls": [
    "https://astralcodexten.substack.com",
    "https://example.substack.com"
  ],
  "maxPostsPerPublication": 25,
  "includeContent": true
}

Example output

{
  "post_id": "https://astralcodexten.substack.com/p/example-post",
  "title": "Example newsletter post",
  "url": "https://astralcodexten.substack.com/p/example-post",
  "published_at": "2026-07-09T14:00:00+00:00",
  "author": "Scott Alexander",
  "subtitle": "A short public preview of the post...",
  "content_text": "The public article text...",
  "categories": ["Open Thread"],
  "image": "https://substackcdn.com/image/fetch/...",
  "publication_title": "Astral Codex Ten",
  "publication_url": "https://astralcodexten.substack.com",
  "source": "Substack RSS"
}

The example demonstrates the field shape. Actual values depend on the public feed at the time of the run.

Use cases

Newsletter and media monitoring

Run a stable publication list every hour or day, store post_id as the deduplication key, and notify a researcher when a new URL appears. This works well for competitor newsletters, policy publications, industry analysts, and independent media.

Creator and topic research

Combine title, subtitle, content, author, and category fields to study what a newsletter covers over time. Export the data to a notebook or warehouse for topic classification, entity extraction, keyword trends, or citation analysis.

Content operations

Feed new public posts into an internal reading queue, editorial dashboard, or knowledge base. Keep the canonical URL attached so readers always return to the original publication.

Public archive and change tracking

Create dated snapshots of currently public feed entries. Compare consecutive datasets to identify newly published posts. Respect copyright and retention requirements; structured access does not grant republication rights.

Pricing

The Actor uses transparent pay-per-event pricing: a very small one-time start event and $0.003 per result saved to the default dataset. One result is one newsletter post. The default is limited to ten posts per publication, and maxPostsPerPublication provides a predictable spending control for larger runs.

Reliability and data quality

Public RSS is substantially lighter and more stable than automating a logged-in website. Requests are retried for temporary failures, duplicate post IDs are suppressed, dates are normalized when possible, and a zero-item run fails visibly instead of looking successful. If a publication has disabled or changed its feed, the run log identifies the affected URL while continuing with other publications.

For recurring pipelines, preserve post_id, url, and published_at. Treat missing fields as source omissions. The Actor never guesses author names, categories, or publication dates.

Responsible use

This Actor collects only content exposed through public publication feeds. It does not bypass subscriptions or paywalls. You are responsible for complying with copyright, privacy, platform terms, and applicable law. Use article text for permitted analysis and internal workflows, and link back to the original publisher rather than republishing content without authorization.

FAQ

Do I need a Substack account?
No. The Actor reads public publication feeds.

Does it work with custom Substack domains?
Yes, when the custom domain exposes the standard public /feed route. You can also paste a direct feed URL.

Can it scrape paid posts?
It returns only the title, preview, or content that the public feed exposes. It does not unlock subscriber-only text.

Can I scrape several newsletters at once?
Yes. Add each publication to publicationUrls; the result limit applies to each publication.

Why is full content missing for some rows?
Publication owners choose whether feeds include the complete article or an excerpt.

How should I deduplicate scheduled runs?
Use post_id, falling back to url, as the stable unique key.

Can I use the output in Make, Zapier, or n8n?
Yes. Use the default dataset API or the built-in Apify integrations.

You might also like

• RSS Feed Scraper for general RSS and Atom feeds
• Google News Scraper for news monitoring
• Dev.to Articles Scraper for developer publishing data
• Website Content Crawler for full-site crawling

---

Keywords: Substack scraper, Substack posts scraper, newsletter scraper, Substack RSS, scrape newsletters, newsletter monitoring, Substack API alternative, creator content data, public newsletter archive, Substack export, newsletter analytics.