# Substack People Scraper (`automation-lab/substack-people-scraper`) Actor Find Substack creators by keyword and export public profile, publication, bio, website, and social-link data for outreach. - **URL**: https://apify.com/automation-lab/substack-people-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Substack People Scraper Find creators, newsletter operators, and publication leads from Substack's public People search. ### What does Substack People Scraper do? Substack People Scraper searches public Substack people results by keyword and exports structured creator, profile, and publication data. Use it to turn broad topics such as `AI newsletter`, `climate policy`, `sales ops`, or `web scraping` into a spreadsheet of public Substack profiles. The actor is designed for lead discovery, partnership research, sponsorship prospecting, market mapping, and creator outreach. ### Who is it for? - ๐Ÿ“ฃ Sponsorship teams building lists of newsletter creators in a niche. - ๐Ÿค Partnership managers finding Substack writers for co-marketing. - ๐Ÿงฒ Lead generation agencies researching creator-owned audiences. - ๐Ÿ“ฐ Media researchers mapping publications by topic. - ๐Ÿง‘โ€๐Ÿ’ผ B2B marketers looking for public creator profiles and websites. - ๐Ÿงช Analysts comparing newsletter categories and public creator metadata. ### Why use this actor? Substack's built-in search is useful for browsing, but it is not built for exporting repeatable research. This actor gives you a clean dataset with result rank, profile identity, bio, publication metadata, profile links, and the original search URL. You can run the same keyword set every week, enrich the output in your CRM, or combine it with other Apify actors. ### What data can I extract? | Field | Description | | --- | --- | | `query` | Keyword used for the search | | `rank` | Result position within that keyword | | `profileId` | Public Substack profile ID when available | | `profileName` | Creator or profile display name | | `handle` | Substack handle | | `profileUrl` | Public Substack profile URL | | `bio` | Profile biography | | `photoUrl` | Profile image URL | | `publicationName` | Attached primary publication name | | `publicationUrl` | Publication URL or custom domain | | `publicationDescription` | Publication hero text/description | | `publicationLanguage` | Language returned by Substack | | `paymentsState` | Public payment state when present | | `communityEnabled` | Whether community features are enabled | | `userLinks` | Public social/website links from the profile | | `sourceSearchUrl` | Browser search URL for the query | | `scrapedAt` | Timestamp for the row | ### How much does it cost to scrape Substack people search? The actor uses pay-per-event pricing. You pay a small start fee for the run and a per-result fee for saved people rows. For a typical lead discovery job, you can start with 50 results and scale up after reviewing output quality. Exact live pricing is shown on the Apify actor page before you run the actor. ### How to use Substack People Scraper 1. Open the actor on Apify. 2. Add one or more keywords in the **Keywords** field. 3. Keep `maxResults` small for the first test run. 4. Optionally add Substack people search URLs. 5. Click **Start**. 6. Download the dataset as JSON, CSV, Excel, or via API. ### Input options #### Keywords Use creator topics, audiences, niches, or business categories. Examples: - `AI newsletter` - `climate policy` - `venture capital` - `web scraping` - `local news` #### Substack people search URLs You can paste URLs such as: `https://substack.com/search/AI%20newsletter?searching=people` The actor extracts the query from the URL and searches public people results. #### Maximum people Controls the total number of rows saved across all keywords. Use a low value for a quick first run. #### Maximum pages per keyword A safety cap for pagination. The default is conservative and works for most discovery jobs. #### Language filter Substack supports language filters on some search surfaces. Leave this blank unless you specifically need a language-constrained result set. #### Include profile links When enabled, the actor saves public social and website links returned by Substack. ### Output example ```json { "query": "AI newsletter", "rank": 1, "profileName": "AI Fyndings Newsletter", "handle": "aifyndings", "profileUrl": "https://substack.com/@aifyndings", "bio": "Every Friday, one smart read on how AI is transforming Business...", "publicationName": "AI Fyndings Newsletter", "publicationUrl": "https://newsletter.pixelbin.io", "publicationDescription": "Every Friday, one smart read on how AI is transforming Business...", "sourceSearchUrl": "https://substack.com/search/AI%20newsletter?searching=people" } ```` ### Tips for better results - ๐ŸŽฏ Use specific niches instead of generic words. - ๐Ÿงฉ Combine topic and audience terms, such as `AI for lawyers`. - ๐Ÿงช Run several small keyword tests before scaling. - ๐Ÿ“Š Deduplicate by `profileId`, `handle`, or `publicationUrl` in your downstream workflow. - ๐Ÿ”— Use `userLinks` for enrichment in Clay, Airtable, or your CRM. ### Common use cases #### Sponsorship prospecting Find newsletters and creator profiles relevant to a product category. #### Creator partnership research Build a list of public creators for collaborations, guest posts, or swaps. #### Market mapping Track which Substack publications appear for important topics. #### PR outreach Find independent writers covering a theme or industry. #### Competitive intelligence Monitor public creator profiles around a category your company serves. ### Integrations Export results to CSV for spreadsheet review. Send the dataset to Google Sheets using Apify integrations. Connect the actor with Make or Zapier for recurring lead-list updates. Use the Apify API to push results into Airtable, Clay, HubSpot, or a warehouse. ### API usage #### Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/substack-people-scraper').call({ keywords: ['AI newsletter'], maxResults: 25 }); console.log(run.defaultDatasetId); ``` #### Python ```python from apify_client import ApifyClient client = ApifyClient('YOUR_APIFY_TOKEN') run = client.actor('automation-lab/substack-people-scraper').call(run_input={ 'keywords': ['AI newsletter'], 'maxResults': 25, }) print(run['defaultDatasetId']) ``` #### cURL ```bash curl -X POST "https://api.apify.com/v2/acts/automation-lab~substack-people-scraper/runs?token=$APIFY_TOKEN" \ -H 'Content-Type: application/json' \ -d '{"keywords":["AI newsletter"],"maxResults":25}' ``` ### MCP usage Use the Apify MCP server to run this actor from AI tools. MCP URL: `https://mcp.apify.com/?tools=automation-lab/substack-people-scraper` Claude Code setup: ```bash claude mcp add apify-substack-people "https://mcp.apify.com/?tools=automation-lab/substack-people-scraper" ``` Claude Desktop JSON config: ```json { "mcpServers": { "apify-substack-people": { "url": "https://mcp.apify.com/?tools=automation-lab/substack-people-scraper" } } } ``` Example prompts showing MCP usage: - "Use the Apify MCP tool `automation-lab/substack-people-scraper` for `AI newsletter` and summarize the top creator profiles with publication URLs." - "Run the Substack People Scraper MCP tool to find 50 Substack creators writing about B2B SaaS. Return profile URLs, publication URLs, and website links." - "Use the Substack people MCP tool to find public creators around climate policy, then rank the results by publication relevance and available website links." ### Scheduling You can schedule recurring runs in Apify. A weekly schedule works well for monitoring new creators in a niche. Use the same keyword list each time and compare datasets by `profileId` or `handle`. ### Data quality notes The actor returns public data supplied by Substack's people search. Some profiles have no attached publication. Some publications use custom domains. Some profiles have no social links. Result order follows Substack's public search ranking at run time. ### Limitations This actor does not log in. It does not scrape private subscriber lists. It does not extract email addresses that are not publicly returned by Substack. It does not scrape post archives; use related post/archive actors for that workflow. ### Legality and responsible use This actor extracts publicly available Substack search/profile metadata. You are responsible for using the data in line with applicable laws, privacy rules, platform terms, and outreach regulations. Avoid spam. Respect opt-outs. Use the data for legitimate research and outreach workflows. ### Troubleshooting #### Why did I get fewer results than requested? Substack may return fewer public people results for a narrow keyword. Try broader related keywords or increase the number of keywords. #### Why are some publication fields empty? Not every public profile has a primary publication attached in search results. The actor keeps profile rows even when publication metadata is missing. #### Why are links missing? Creators choose which public links to add to their profiles. If Substack does not return links, the `userLinks` array may be empty. ### Related scrapers - https://apify.com/automation-lab/substack-scraper - https://apify.com/automation-lab/google-search-scraper - https://apify.com/automation-lab/google-news-scraper - https://apify.com/automation-lab/website-health-report ### Changelog Initial version: public Substack people search by keyword with profile, publication, and user-link extraction. ### Support If a run fails or returns unexpected results, open an issue on the Apify actor page and include your input and run ID. ### FAQ #### Can I search multiple topics at once? Yes. Add multiple keywords. The actor searches each keyword and combines rows in one dataset. #### Can I use this for newsletter sponsorship lists? Yes. The output is designed for sponsorship and partnership research. #### Does it require proxies? No proxy is configured for the MVP because the public endpoint is reachable with normal HTTP requests. #### Does it scrape posts? No. This actor focuses on people/profile discovery. #### Can I export to Excel? Yes. Apify datasets can be downloaded as Excel, CSV, JSON, XML, RSS, or HTML. # Actor input Schema ## `keywords` (type: `array`): Search terms to run in Substack's public People search. Use creator niches, industries, audience topics, or sponsorship categories. ## `startUrls` (type: `array`): Optional Substack search URLs such as https://substack.com/search/AI%20newsletter?searching=people. The actor extracts the query from each URL. ## `maxResults` (type: `integer`): Maximum number of creator/profile rows to save across all keywords. ## `maxPagesPerKeyword` (type: `integer`): Safety cap for Substack result pagination per keyword. Keep the default unless you need deeper discovery. ## `language` (type: `string`): Optional language code accepted by Substack search, for example en or es. Leave empty for all languages. ## `includeUserLinks` (type: `boolean`): Save social and website links from each public profile when Substack returns them. ## Actor input object example ```json { "keywords": [ "AI newsletter", "web scraping" ], "startUrls": [], "maxResults": 20, "maxPagesPerKeyword": 5, "includeUserLinks": true } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "keywords": [ "AI newsletter", "web scraping" ], "startUrls": [] }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/substack-people-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "keywords": [ "AI newsletter", "web scraping", ], "startUrls": [], } # Run the Actor and wait for it to finish run = client.actor("automation-lab/substack-people-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "keywords": [ "AI newsletter", "web scraping" ], "startUrls": [] }' | apify call automation-lab/substack-people-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/substack-people-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Substack People Scraper", "description": "Find Substack creators by keyword and export public profile, publication, bio, website, and social-link data for outreach.", "version": "0.1", "x-build-id": "lirkBENrc4uTU9SrY" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~substack-people-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-substack-people-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~substack-people-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-substack-people-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~substack-people-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-substack-people-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "keywords" ], "properties": { "keywords": { "title": "Keywords", "type": "array", "description": "Search terms to run in Substack's public People search. Use creator niches, industries, audience topics, or sponsorship categories.", "items": { "type": "string" } }, "startUrls": { "title": "Substack people search URLs", "type": "array", "description": "Optional Substack search URLs such as https://substack.com/search/AI%20newsletter?searching=people. The actor extracts the query from each URL.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxResults": { "title": "Maximum people", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of creator/profile rows to save across all keywords.", "default": 20 }, "maxPagesPerKeyword": { "title": "Maximum pages per keyword", "minimum": 1, "maximum": 50, "type": "integer", "description": "Safety cap for Substack result pagination per keyword. Keep the default unless you need deeper discovery.", "default": 5 }, "language": { "title": "Language filter", "type": "string", "description": "Optional language code accepted by Substack search, for example en or es. Leave empty for all languages." }, "includeUserLinks": { "title": "Include profile links", "type": "boolean", "description": "Save social and website links from each public profile when Substack returns them.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```