📬 Substack & Beehiiv Newsletter Lead Scraper & Sponsor Intelligence

Find newsletters that already run paid sponsors, extract verified author contacts, and build targeted B2B sponsorship outreach lists automatically.

This Actor discovers or directly crawls Substack and Beehiiv publications, enriches each record with direct contact emails, social links (Twitter/X, LinkedIn), and subscriber tiers, and audits recent issues for verified sponsor disclosures and commercial tracking links.

---

⚡ Why Use This Actor?

• 100% Browserless & Fast: Built on CheerioCrawler and got-scraping (zero Playwright/Puppeteer overhead). Runs complete in sub-seconds with near-zero compute cost.
• Automated Category Discovery: Discover publications automatically by category via Substack leaderboards and recommendation graph traversal, or crawl an exact list of custom URLs.
• Direct Author Contacts: Crawls /about pages and recent post footers to extract direct author emails (filtering out platform noise, image-filename false positives like logo@2x.png, and generic mailboxes) alongside LinkedIn, Twitter/X, and personal website links. When no raw email is published, falls back to any sponsorship-intake link found (Typeform, Calendly, Passionfroot, Google Forms, or an on-site /advertise page) via sponsorBookingUrl.
• High-Precision Sponsor Detection: Flags sponsors only when backed by explicit disclosure markers ("sponsored by", "presented by", "brought to you by", "in partnership with") or ad-network UTM parameters (utm_medium=sponsor, utm_source=*newsletter*, promo= codes). Deliberately ignores Substack's internal ref= tags to eliminate false positives.
• Stateful Lead Feeds (onlyNewSinceLastRun): Tracks historical publications and sponsor domains in a Key-Value store. Scheduled runs only emit newly discovered publications or newsletters with newly detected sponsors.

---

🎯 Common Use Cases

1. Sponsorship & Ad Sales Outreach: Generate verified lead lists of newsletter operators who actively monetize and accept sponsor placements.
2. Competitor Ad Intelligence: Track which brands and competitors are actively buying newsletter sponsorships in your niche.
3. Affiliate & Co-Marketing Scouting: Discover publications with engaged, commercial-friendly audiences for partnerships.
4. Creator PR & Podcast Pitching: Pull direct author contacts and publication bios for PR and media outreach.

---

🛠️ Input Configuration

Field	Type	Required	Description
searchMode	string	–	DISCOVERY (default) finds publications by category. DIRECT_URLS crawls your specified URLs.
categories	array	–	Substack categories to scan (e.g. technology, business, finance, crypto, ai, culture).
startUrls	array	–	Substack or Beehiiv URLs to crawl in DIRECT_URLS mode (supports custom domains).
maxPublications	integer	–	Maximum publications to discover/process per run (Default 50, max 1000).
recentPostsPerPublication	integer	–	Number of recent issues to audit for sponsors per publication (Default 10, max 30).
enrichContacts	boolean	–	Crawl /about pages for direct emails, social links, bio, and subscriber tiers (Default true).
onlyNewSinceLastRun	boolean	–	Only emit publications that are brand new or running a newly detected sponsor (Default false).
proxyConfiguration	object	–	Optional proxy settings (runs browserless without proxies by default).

Sample Input — Discovery Mode

{
  "searchMode": "DISCOVERY",
  "categories": ["technology", "business", "ai"],
  "maxPublications": 50,
  "recentPostsPerPublication": 10,
  "enrichContacts": true,
  "onlyNewSinceLastRun": true
}

📊 Output Dataset Format

Each record represents one publication with complete contact details and an inline audit of detected sponsors:

{
  "publicationName": "Platformer",
  "platform": "substack",
  "url": "https://www.platformer.news",
  "canonicalDomain": "platformer.news",
  "status": "OK",
  "authorName": "Casey Newton",
  "authorBio": "Platformer is an independent publication about AI, social platforms, and tech policy...",
  "contactEmail": "casey@platformer.news",
  "socialProfiles": {
    "twitter": "https://x.com/platformer",
    "linkedin": "https://www.linkedin.com/in/caseynewton1",
    "website": null
  },
  "subscriberTier": "50,000 subscribers",
  "sponsorBookingUrl": null,
  "postCadence": "Daily",
  "acceptsSponsors": true,
  "totalSponsorsDetected": 1,
  "recentSponsors": [
    {
      "sponsorDomain": "mercury.com",
      "trackingUrl": "https://mercury.com/spend-management?utm_source=platformer&utm_medium=sponsored_newsletter&utm_campaign=26q3_brand_campaign",
      "disclosureText": "Today's issue is brought to you by Mercury...",
      "issueTitle": "An LLM wiki changed how I work",
      "issueDate": "2026-08-19T00:28:38.000Z",
      "issueUrl": "https://www.platformer.news/karpathy-llm-wiki-journalism-productivity/"
    }
  ],
  "isNew": true,
  "scrapedAt": "2026-08-30T15:06:14.940Z"
}

A failed publication (e.g. an unreachable domain or a 403/404) is still pushed as its own record, with the real failure reason always populated in error — never a blank or generic placeholder:

{
  "url": "https://example-newsletter.com",
  "canonicalDomain": "example-newsletter.com",
  "status": "FAILED",
  "error": "HTTP 404: Not Found",
  "scrapedAt": "2026-08-30T15:06:15.201Z"
}

💳 Pricing (Pay-Per-Event)

You only pay for enriched publication records delivered:

Event                          Price                          Details
------------------------------ ------------------------------ ------------------------------------------------------------
Actor Start / Compute          Free / $0.00                   Zero compute overhead
Enriched Publication Record    $0.003 ($3.00 / 1k records)    Charged per structured publication record output to the dataset

🔄 Automated Workflows & Integrations

Connect this Actor to your existing sales stack:

Slack / Discord Alerts: Trigger an Apify Webhook on run completion to send a daily digest of newly sponsored newsletters to your sales team.

Make.com / n8n / Zapier: Route new leads directly into HubSpot, Notion, Airtable, or your cold email outreach sequences.

Weekly Schedule: Set up a weekly Apify Schedule (0 9 * * 1) with onlyNewSinceLastRun: true to receive a recurring pipeline of fresh leads.

⚙️ How It Works

1. Discovery Engine: Resolves category slugs to Substack internal category IDs (/api/v1/category/public/{id}/all), falling back to search queries when leaderboards are unavailable. Traverses publication recommendation graphs (/api/v1/recommendations) to surface high-relevance adjacent newsletters.

2. Contact Enricher: Crawls /about pages to extract direct author names, bios, verified mailto: addresses, and Twitter/LinkedIn links.

3. Sponsor Intelligence Auditor: Analyzes the last N issues, preferring each publication's public RSS feed (/feed) for Substack — which always carries the full rendered post body — and only falling back to the JSON posts API (re-fetching each post's own detail endpoint for its complete content) if no feed is reachable. Beehiiv issues are pulled from archive pages. Identifies commercial links using intent keywords and ad-network UTM parameters.

4. Stateful Diffing: Persists publication IDs and historic sponsor domains in a named Key-Value store to prevent duplicate alerts.

⚠️ Known Limitations

1. Discovery Scope: Substack discovery relies on public leaderboard and category endpoints. Direct URLs mode is recommended when targeting specific custom lists.

2. Beehiiv Discovery: Beehiiv does not expose a public global category directory; Beehiiv publications are crawled via DIRECT_URLS mode.

3. Custom Landing Pages: Publications with custom splash pages that completely hide their post archive will return contact info without post-level sponsor audits.

❓ FAQ

Why doesn't ref= alone trigger a sponsor detection? Substack automatically adds ref=

Does this support newsletters on custom domains? Yes. The scraper detects platform signatures from metadata and underlying CDN headers, parsing custom domains (e.g., platformer.news, lennysnewsletter.com) seamlessly.

Can I run this as an MCP tool in Claude or Cursor? Yes. You can invoke this Actor directly via Apify's hosted MCP server (https://mcp.apify.com).

Why is contactEmail (or linkedin/website) null for some publications? Substack's /about page renders most of its content client-side; this Actor recovers what it can directly from the page's own embedded data (author name/bio, Twitter handle, subscriber count) without needing a browser, and also pools any mailto:/footer email found in the publication's own recent posts — not just /about. A direct email, LinkedIn, or personal website is only ever returned when the author actually published it somewhere scrapable; many simply don't, in which case null is the accurate answer, not a scraping failure. When no raw email is found at all, check sponsorBookingUrl — most newsletters that skip a public email instead link to a Typeform/Calendly/Passionfroot form or an on-site "/advertise" page for sponsorship inquiries, and that link is captured there.