# Substack & Beehiiv Newsletter Sponsor Intelligence (`dataflow-tools/newsletter-sponsor-intelligence`) Actor Discovers Substack and Beehiiv newsletters, enriches them with direct contact info, and audits recent issues for sponsor disclosures and commercial links to surface high-intent sponsorship leads. - **URL**: https://apify.com/dataflow-tools/newsletter-sponsor-intelligence.md - **Developed by:** [DataFlow Tools](https://apify.com/dataflow-tools) (community) - **Categories:** - **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.10 / 1,000 enriched publications This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## 📬 Substack & Beehiiv Newsletter Lead Scraper & Sponsor Intelligence Find newsletters that already run paid sponsors, extract verified author contacts, and build targeted B2B sponsorship outreach lists automatically. This Actor discovers or directly crawls Substack and Beehiiv publications, enriches each record with direct contact emails, social links (Twitter/X, LinkedIn), and subscriber tiers, and audits recent issues for verified sponsor disclosures and commercial tracking links. *** ### ⚡ Why Use This Actor? - **100% Browserless & Fast:** Built on `CheerioCrawler` and `got-scraping` (zero Playwright/Puppeteer overhead). Runs complete in sub-seconds with near-zero compute cost. - **Automated Category Discovery:** Discover publications automatically by category via Substack leaderboards and recommendation graph traversal, or crawl an exact list of custom URLs. - **Direct Author Contacts:** Crawls `/about` pages *and* recent post footers to extract direct author emails (filtering out platform noise, image-filename false positives like `logo@2x.png`, and generic mailboxes) alongside LinkedIn, Twitter/X, and personal website links. When no raw email is published, falls back to any sponsorship-intake link found (Typeform, Calendly, Passionfroot, Google Forms, or an on-site `/advertise` page) via `sponsorBookingUrl`. - **High-Precision Sponsor Detection:** Flags sponsors only when backed by explicit disclosure markers (*"sponsored by"*, *"presented by"*, *"brought to you by"*, *"in partnership with"*) or ad-network UTM parameters (`utm_medium=sponsor`, `utm_source=*newsletter*`, `promo=` codes). Deliberately ignores Substack's internal `ref=` tags to eliminate false positives. - **Stateful Lead Feeds (`onlyNewSinceLastRun`):** Tracks historical publications and sponsor domains in a Key-Value store. Scheduled runs only emit newly discovered publications or newsletters with newly detected sponsors. *** ### 🎯 Common Use Cases 1. **Sponsorship & Ad Sales Outreach:** Generate verified lead lists of newsletter operators who actively monetize and accept sponsor placements. 2. **Competitor Ad Intelligence:** Track which brands and competitors are actively buying newsletter sponsorships in your niche. 3. **Affiliate & Co-Marketing Scouting:** Discover publications with engaged, commercial-friendly audiences for partnerships. 4. **Creator PR & Podcast Pitching:** Pull direct author contacts and publication bios for PR and media outreach. *** ### 🛠️ Input Configuration | Field | Type | Required | Description | |---|---|---|---| | `searchMode` | string | – | `DISCOVERY` (default) finds publications by category. `DIRECT_URLS` crawls your specified URLs. | | `categories` | array | – | Substack categories to scan (e.g. `technology`, `business`, `finance`, `crypto`, `ai`, `culture`). | | `startUrls` | array | – | Substack or Beehiiv URLs to crawl in `DIRECT_URLS` mode (supports custom domains). | | `maxPublications` | integer | – | Maximum publications to discover/process per run (Default `50`, max `1000`). | | `recentPostsPerPublication` | integer | – | Number of recent issues to audit for sponsors per publication (Default `10`, max `30`). | | `enrichContacts` | boolean | – | Crawl `/about` pages for direct emails, social links, bio, and subscriber tiers (Default `true`). | | `onlyNewSinceLastRun` | boolean | – | Only emit publications that are brand new or running a newly detected sponsor (Default `false`). | | `proxyConfiguration` | object | – | Optional proxy settings (runs browserless without proxies by default). | #### Sample Input — Discovery Mode ```json { "searchMode": "DISCOVERY", "categories": ["technology", "business", "ai"], "maxPublications": 50, "recentPostsPerPublication": 10, "enrichContacts": true, "onlyNewSinceLastRun": true } ``` #### 📊 Output Dataset Format Each record represents one publication with complete contact details and an inline audit of detected sponsors: ```json { "publicationName": "Platformer", "platform": "substack", "url": "https://www.platformer.news", "canonicalDomain": "platformer.news", "status": "OK", "authorName": "Casey Newton", "authorBio": "Platformer is an independent publication about AI, social platforms, and tech policy...", "contactEmail": "casey@platformer.news", "socialProfiles": { "twitter": "https://x.com/platformer", "linkedin": "https://www.linkedin.com/in/caseynewton1", "website": null }, "subscriberTier": "50,000 subscribers", "sponsorBookingUrl": null, "postCadence": "Daily", "acceptsSponsors": true, "totalSponsorsDetected": 1, "recentSponsors": [ { "sponsorDomain": "mercury.com", "trackingUrl": "https://mercury.com/spend-management?utm_source=platformer&utm_medium=sponsored_newsletter&utm_campaign=26q3_brand_campaign", "disclosureText": "Today's issue is brought to you by Mercury...", "issueTitle": "An LLM wiki changed how I work", "issueDate": "2026-08-19T00:28:38.000Z", "issueUrl": "https://www.platformer.news/karpathy-llm-wiki-journalism-productivity/" } ], "isNew": true, "scrapedAt": "2026-08-30T15:06:14.940Z" } ``` A failed publication (e.g. an unreachable domain or a 403/404) is still pushed as its own record, with the real failure reason always populated in `error` — never a blank or generic placeholder: ```json { "url": "https://example-newsletter.com", "canonicalDomain": "example-newsletter.com", "status": "FAILED", "error": "HTTP 404: Not Found", "scrapedAt": "2026-08-30T15:06:15.201Z" } ``` #### 💳 Pricing (Pay-Per-Event) You only pay for enriched publication records delivered: ``` Event Price Details ------------------------------ ------------------------------ ------------------------------------------------------------ Actor Start / Compute Free / $0.00 Zero compute overhead Enriched Publication Record $0.003 ($3.00 / 1k records) Charged per structured publication record output to the dataset ``` #### 🔄 Automated Workflows & Integrations Connect this Actor to your existing sales stack: Slack / Discord Alerts: Trigger an Apify Webhook on run completion to send a daily digest of newly sponsored newsletters to your sales team. Make.com / n8n / Zapier: Route new leads directly into HubSpot, Notion, Airtable, or your cold email outreach sequences. Weekly Schedule: Set up a weekly Apify Schedule (0 9 \* \* 1) with onlyNewSinceLastRun: true to receive a recurring pipeline of fresh leads. #### ⚙️ How It Works 1. Discovery Engine: Resolves category slugs to Substack internal category IDs (/api/v1/category/public/{id}/all), falling back to search queries when leaderboards are unavailable. Traverses publication recommendation graphs (/api/v1/recommendations) to surface high-relevance adjacent newsletters. 2. Contact Enricher: Crawls /about pages to extract direct author names, bios, verified mailto: addresses, and Twitter/LinkedIn links. 3. Sponsor Intelligence Auditor: Analyzes the last N issues, preferring each publication's public RSS feed (`/feed`) for Substack — which always carries the full rendered post body — and only falling back to the JSON posts API (re-fetching each post's own detail endpoint for its complete content) if no feed is reachable. Beehiiv issues are pulled from archive pages. Identifies commercial links using intent keywords and ad-network UTM parameters. 4. Stateful Diffing: Persists publication IDs and historic sponsor domains in a named Key-Value store to prevent duplicate alerts. #### ⚠️ Known Limitations 1. Discovery Scope: Substack discovery relies on public leaderboard and category endpoints. Direct URLs mode is recommended when targeting specific custom lists. 2. Beehiiv Discovery: Beehiiv does not expose a public global category directory; Beehiiv publications are crawled via DIRECT\_URLS mode. 3. Custom Landing Pages: Publications with custom splash pages that completely hide their post archive will return contact info without post-level sponsor audits. #### ❓ FAQ Why doesn't ref= alone trigger a sponsor detection? Substack automatically adds ref= to every single outbound link for internal analytics. Relying on ref= alone creates massive false-positive noise. This Actor requires explicit disclosure phrasing or ad-network campaign parameters. Does this support newsletters on custom domains? Yes. The scraper detects platform signatures from metadata and underlying CDN headers, parsing custom domains (e.g., platformer.news, lennysnewsletter.com) seamlessly. Can I run this as an MCP tool in Claude or Cursor? Yes. You can invoke this Actor directly via Apify's hosted MCP server (https://mcp.apify.com). Why is `contactEmail` (or `linkedin`/`website`) null for some publications? Substack's `/about` page renders most of its content client-side; this Actor recovers what it can directly from the page's own embedded data (author name/bio, Twitter handle, subscriber count) without needing a browser, and also pools any `mailto:`/footer email found in the publication's own recent posts — not just `/about`. A direct email, LinkedIn, or personal website is only ever returned when the author actually published it somewhere scrapable; many simply don't, in which case `null` is the accurate answer, not a scraping failure. When no raw email is found at all, check `sponsorBookingUrl` — most newsletters that skip a public email instead link to a Typeform/Calendly/Passionfroot form or an on-site "/advertise" page for sponsorship inquiries, and that link is captured there. # Actor input Schema ## `searchMode` (type: `string`): DISCOVERY finds new publications by category via Substack's category/leaderboard/recommendation graph. DIRECT\_URLS crawls the exact publication URLs you provide (Substack or Beehiiv). ## `categories` (type: `array`): Substack categories to scan when Search mode is Discovery (e.g. technology, business, finance, crypto, ai, culture). ## `startUrls` (type: `array`): Substack or Beehiiv publication URLs to crawl when Search mode is Direct URLs (e.g. https://example.substack.com, https://newsletter.example.com). ## `maxPublications` (type: `integer`): Upper bound on how many publications to discover/process in this run. ## `recentPostsPerPublication` (type: `integer`): Number of the most recent issues to fetch and scan for sponsor disclosures, per publication. ## `enrichContacts` (type: `boolean`): When enabled, crawls each publication's /about page for a direct contact email, social profiles, author bio, and subscriber tier. ## `onlyNewSinceLastRun` (type: `boolean`): When enabled, a publication is only included in this run's output if it wasn't seen in any previous run OR it is now running a sponsor that wasn't detected before. Disable to always emit the full current snapshot. ## `proxyConfiguration` (type: `object`): Optional Apify Proxy or custom proxy URLs to use for all requests. ## Actor input object example ```json { "searchMode": "DISCOVERY", "categories": [ "technology", "business", "finance" ], "startUrls": [ "https://www.lennysnewsletter.com" ], "maxPublications": 50, "recentPostsPerPublication": 10, "enrichContacts": true, "onlyNewSinceLastRun": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `dataset` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "categories": [ "technology", "business", "finance" ], "startUrls": [ "https://www.lennysnewsletter.com" ], "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("dataflow-tools/newsletter-sponsor-intelligence").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "categories": [ "technology", "business", "finance", ], "startUrls": ["https://www.lennysnewsletter.com"], "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("dataflow-tools/newsletter-sponsor-intelligence").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "categories": [ "technology", "business", "finance" ], "startUrls": [ "https://www.lennysnewsletter.com" ], "proxyConfiguration": { "useApifyProxy": false } }' | apify call dataflow-tools/newsletter-sponsor-intelligence --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,dataflow-tools/newsletter-sponsor-intelligence" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/Z3NUBieXFBCmQzThB/builds/VF4OhnhT9S0dnCTIS/openapi.json