Substack Scraper — Newsletters, Authors & Public Posts

Substack Scraper finds newsletters by keyword or publication URL and collects author profiles, public subscriber labels, subscription plans, sections, and optional public posts. Export newsletter research as JSON, CSV, or Excel for creator discovery, partnership prospecting, and content monitoring.

🤖 What does Substack Scraper do?

• Find publications by keyword or specific public Substack URL.
• Return rich publication profiles including descriptions, author names and biographies, images, language, creation dates, sections, and available subscription information.
• Optionally include posts with titles, dates, authors, descriptions, public content or previews, categories, and media links.
• Export results as JSON, CSV, or Excel, or integrate through the Apify API and MCP.

Great for: researching newsletter niches, discovering potential partners, comparing publication profiles, and monitoring recent public writing.

🧠 Choose your collection mode

Discover newsletters by keyword

Enter one or more words or phrases in keywords, such as ["software", "artificial intelligence"]. Each keyword is searched in order, and _keyword identifies which search produced each result.

Look up known publications

Enter publication urls, such as https://pragmaticengineer.substack.com. You can combine URLs with keywords: supplied URLs are processed first, then keyword searches, sharing one maxItems limit. If publication details are unavailable, the URL is still included in your results.

In either mode, enable fetchPosts to include available recent public posts. This is not a complete historical archive.

🚀 How to use

1. Enter keywords, such as ["software", "technology"], supply publication urls, or use both.
2. Choose maxItems, the maximum publications to collect, or set it to 0 for unlimited results.
3. Enable fetchPosts if you also need available recent public posts.
4. Run the actor and export the Dataset.

📝 Input parameters

Parameter	Type	Required	Description
maxItems	Integer	Yes	Maximum publications across all keywords or supplied URLs; default 100. Set to 0 for unlimited results.
keywords	Array of strings	No*	Search each keyword or phrase in order. Blank entries are skipped. Each result includes its _keyword.
urls	Array of strings	No	Publication links such as https://pragmaticengineer.substack.com. Use the publication's Substack address, not a custom domain. Works alongside keywords; URLs are processed first within the shared maxItems limit. URLs without available details are included with _found: false.
fetchPosts	Boolean	No	Include available recent public posts under _posts. Default false. Adds a charge per returned publication, not per post, including publications with no available posts. Does not cover individual sections or the full archive.

* Provide at least one non-empty keyword or publication URL. Without either, no results are collected. See the actor's pricing information for current charges before enabling fetchPosts.

1️⃣ Find software and technology publications

{"keywords":["software","technology"],"maxItems":100}

2️⃣ Fetch a known publication and its public posts

{"urls":["https://pragmaticengineer.substack.com"],"maxItems":1,"fetchPosts":true}

3️⃣ Combine known publications with keyword discovery

{"urls":["https://pragmaticengineer.substack.com"],"keywords":["software"],"maxItems":25}

📊 Output and results

Each row contains one publication result, with available author details, audience information, subscription options, and optional posts:

• _keyword: the keyword or phrase that produced the result; absent when looking up supplied URLs.
• id, name, subdomain, base_url, custom_domain, hero_text: publication identity and description.
• author_name, author_handle, author_bio, author_photo_url, contributors: public author information.
• language, copyright, created_at, first_post_date: publication metadata.
• logo_url, cover_photo_url, sections, plans, subscription benefits, podcast and payment flags: available public profile details.
• freeSubscriberCount, rankingDetailFreeSubscriberCount, and related labels: public audience indicators where provided. These may be rounded labels, not exact counts.
• _url: the publication's Substack address.
• _found: true when publication details are available. Otherwise, the result includes the supplied publication's address and _found: false. This does not mean the publication does not exist.
• _posts: included only with fetchPosts. Posts can include titles, descriptions, links, authors, publication dates, public content or previews, categories, and media links. The number of entries is the returned post count.

A shortened example from a URL lookup with fetchPosts enabled:

{
  "id": 458709,
  "name": "The Pragmatic Engineer",
  "subdomain": "pragmaticengineer",
  "author_name": "Gergely Orosz",
  "language": "en",
  "_url": "https://pragmaticengineer.substack.com",
  "_found": true,
  "_posts": [
    {
      "title": "What’s the ROI of software engineers’ spending on AI?",
      "url": "https://newsletter.pragmaticengineer.com/p/whats-the-roi-of-software-engineers",
      "publishedAt": "2026-10-20T17:34:18Z"
    }
  ]
}

Some profile details may be unavailable. The Overview view shows selected fields; choose All fields when exporting to include complete profiles and posts. Private subscriber information is not collected.

URLs without available publication details still count toward maxItems and can include posts when fetchPosts is enabled. Unsupported or invalid URLs are skipped.

🧭 Tips and limitations

• Keep fetchPosts disabled for faster publication research.
• Start with a small maxItems, such as 10, and inspect the results before increasing your collection size.
• Use urls for known publications and keywords for discovery. Each keyword is searched separately, in input order, until the overall maxItems limit is reached.
• Post collection covers available recent posts, not a publication's complete history or individual sections.
• A publication stays in your results even when no posts are available. In that case, _posts is an empty list.
• Search rankings and fields can change. This actor returns the results currently made public by Substack.

⚖️ Legal and ethical considerations

• Use only publicly available publication information and posts.
• Respect Substack and publishers' terms and applicable privacy law.
• Keep collection volumes and run frequency reasonable, and preserve source links and author attribution when using the results.
• No sign-in, access to private subscriptions, subscriber-only podcasts, or paywall bypass is offered.
• Post content is limited to publicly available text and previews. Media links are included without downloading the files.
• This Actor is independent and is not affiliated with, endorsed by, or sponsored by Substack, Inc.

❓ FAQ

Why did I receive fewer publications than maxItems? The limit is a maximum, not a guaranteed count. The actor stops when the available search results are exhausted or the run budget is reached. Try additional relevant keywords to explore a wider niche.

What does _found: false mean? Publication details could not be found for the supplied URL. The URL is still saved, and the publication may still exist.

Does fetchPosts return every historical or paid post? No. It returns available recent posts with publicly accessible content or previews, not the full archive or subscriber-only content.

Can I provide a custom domain? Use the publication's substack.com address. Results can still include its custom website address when available.

🛟 Support

Need another field or workflow? Open an issue from the actor's Issues tab, or contact the developer.