Substack Discovery Scraper

The Substack Discovery Scraper is a powerful tool designed to efficiently extract and analyze data from Substack publications. Whether you're conducting market research, gathering competitive intelligence, or aggregating content, this Actor provides reliable and comprehensive data to support your needs.

Features

• Efficient Data Extraction: Quickly scrape data from Substack publications using customizable search queries.
• High Data Quality: Ensures accurate and up-to-date information with detailed metadata.
• Scalable Performance: Capable of handling large volumes of data with configurable limits.
• Proxy Support: Utilizes Apify's residential proxies to maintain anonymity and avoid IP bans.
• Versatile Output: Provides data in multiple formats, including JSON, CSV, and Excel.
• User-Friendly Interface: Easy setup and configuration through the Apify platform.
• Comprehensive Metadata: Extracts detailed information about publications, authors, and content.

Input Parameters

Parameter	Type	Required	Description	Example
searchQuery	String	Yes	The search query to use for finding relevant Substack content.	"project"
maxItems	Integer	No	Maximum number of items to scrape.	100
proxyConfiguration	Object	No	Configuration for proxy servers to hide the scraper's origin.	See below

Proxy Configuration Example

{
  "useApifyProxy": true,
  "apifyProxyGroups": ["RESIDENTIAL"]
}

Example Usage

Input

{
  "searchQuery": "project",
  "maxItems": 100,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Output

[
  {
    "id": 193517257,
    "title": "The ICE \"Box\" Tactic: Unhinged, Reckless, and Designed for Disaster",
    "subtitle": " April 7, 2026 | PATTERSON, CA ",
    "slug": "the-ice-box-tactic-unhinged-reckless",
    "type": "podcast",
    "audience": "everyone",
    "canonical_url": "https://eyesonice.substack.com/p/the-ice-box-tactic-unhinged-reckless",
    "post_date": "2026-04-07T23:05:28.866Z",
    "wordcount": 645,
    "truncated_body_text": "The Mechanics of the \u201cBox\u201d",
    "description": "April 7, 2026 | PATTERSON, CA",
    "cover_image": "https://substack-video.s3.amazonaws.com/video_upload/post/193517257/86d9024c-d8d2-4124-99d5-b617c32a7e27/transcoded-00001.png",
    "reaction_count": 140,
    "comment_count": 19,
    "restacks": 86,
    "podcast_duration": 34.403267,
    "podcast_url": "https://api.substack.com/api/v1/audio/upload/f732039a-8785-4653-8c02-4149e2eb7048/src",
    "podcast_episode_image_url": "https://substack-post-media.s3.amazonaws.com/public/images/78468f63-e8d8-4d7f-b9f8-91c873d2059d_1400x1400.png",
    "author_name": "Eyes On Ice",
    "author_handle": "eyesonice",
    "author_bio": "Humanist, and Seeker of Truth. Sharing updates on immigration policies, migrant rights, and fostering discussions. Stand in solidarity with immigrants. https://t.co/nrmNF5KQFv",
    "author_photo_url": "https://substack-post-media.s3.amazonaws.com/public/images/2b0044b1-5417-4e76-9229-1c32631348aa_1024x1024.png",
    "author_twitter": "CantStopPoppin",
    "publication_id": 4363505,
    "publication_name": "Eyes On ICE",
    "publication_subdomain": "eyesonice",
    "publication_logo": "https://substack-post-media.s3.amazonaws.com/public/images/5e3b87f3-826c-4cb1-8edd-40b71b346f98_1024x1024.png",
    "publication_hero_text": "Welcome to a space where compassion meets accountability. This substack is dedicated to monitoring immigration policies, advocating for migrant rights, and fostering informed, respectful discussions about one of the most pressing issues of our time.",
    "scraped_at": "2026-04-17T07:16:42.897Z"
  }
]

Use Cases

• Market Research and Analysis: Gather insights on trending topics and audience engagement.
• Competitive Intelligence: Monitor competitor publications and strategies.
• Content Aggregation: Compile articles and podcasts for newsletters or reports.
• Academic Research: Analyze publication trends and author networks.
• Business Automation: Automate data collection for business intelligence tools.

Installation and Usage

1. Search for "Substack Discovery Scraper" in the Apify Store.
2. Click "Try for free" or "Run".
3. Configure input parameters.
4. Click "Start" to begin extraction.
5. Monitor progress in the log.
6. Export results in your preferred format (JSON, CSV, Excel).

Output Format

The output is a JSON array containing objects with detailed information about each Substack post, including metadata such as title, author, publication date, and more. Key fields include id, title, author_name, post_date, and canonical_url.

Support

For custom/simplified outputs or bug reports, please contact:

• Email: support@getdataforme.com
• Subject line: "custom support"
• Contact form: https://getdataforme.com/contact/

We're here to help you get the most out of this Actor!