Beehiiv Newsletter Scraper 🐝

Extract complete data from Beehiiv newsletters including posts, authors, engagement metrics, and full article text (in both HTML and clean text formats, ready for LLMs).

What does this Actor do? 🎯

This actor scrapes Beehiiv newsletters and extracts 15 data points per post:

Content Data

• Post headline & subheading
• Full article text (cleanly parsed for AI models)
• Raw article HTML
• Post URL & unique Slug
• Internal UUID

Author Information

• Author name

Publishing Data

• Publication date
• Audience status (free vs premium/paywalled)
• Estimated reading time
• Post status (e.g., published)

Engagement Metrics

• Number of likes ❤️

Perfect for 💡

• AI Developers - Collect massive amounts of clean training data directly formatted for LLMs & RAG pipelines
• Content Researchers - Analyze trends across fast-growing Beehiiv newsletters
• Competitive Analysis - Track what competitors publish
• Data Scientists - Build training datasets
• Writers - Research popular topics

Input

{
  "beehiivUrls": [
    "https://www.therundown.ai/"
  ],
  "maxPostsPerPublication": 0,
  "batchSize": 5
}

Field	Type	Required	Description
beehiivUrls	Array	Yes	List of Beehiiv publication URLs
maxPostsPerPublication	Number	No	Limit posts (0 = unlimited)
batchSize	Number	No	Concurrency limit for headless browsers

Output

{
  "beehiiv_url": "https://www.therundown.ai/",
  "post_url": "https://www.therundown.ai/p/nvidia-big-ai-day-at-gtc",
  "id": "bfe910b3-6e6f-4899-968b-9079f10aebf3",
  "slug": "nvidia-big-ai-day-at-gtc",
  "headline": "Nvidia's big AI day at GTC",
  "subheading": "PLUS: How to use Grok for free automated research",
  "publish_date": "2024-03-20T12:00:00.000Z",
  "status": "published",
  "audience": "free",
  "estimated_reading_time": 13,
  "author_name": "Zach Mink",
  "likes": 156,
  "article_text": "Read Online | Sign Up | Advertise\nGood morning ...",
  "article_html": "<div class=\"post-content-node\">...",
  "content_type": "full"
}

Features ✨

• Native API Discovery - Uses Beehiiv's native JSON endpoints for lightning-fast, perfectly paginated post discovery without flaky sitemap parsing
• PerimeterX Bypass - Bypasses robust bot-protections with advanced Playwright stealth rendering
• LLM Ready - Outputs clean, plain-text content natively formatted for AI models
• Smart extraction - Gracefully handles paywalled content
• Structured output - Clean JSON/CSV format

Check out our other Scrapers 🔗

• Substack Newsletter Scraper - The perfect companion for scraping Substack publications with the same high-quality output schema.

Pricing 💰

Based on Pay-Per-Event (PPE) or Compute Units (CUs):

Scale	Approximate Cost	Time
10 Beehiivs	$0.05-0.10	2-5 min
100 Beehiivs	$0.50-1.00	30-60 min
1,000 Beehiivs	$5-10	5-10 hours

Start with Apify's free tier - includes $5 monthly credit!

Tips 💪

1. Test first - Try 2-3 Beehiivs initially
2. Use Residential Proxies - Highly recommended for bypassing PerimeterX at scale
3. Set limits - Use maxPostsPerPublication for large newsletters
4. Schedule runs - Set up weekly/monthly automation

FAQ ❓

Q: Can I scrape paywalled content? A: No, you'll get previews (partial content) for paid posts, along with full metadata. The content_type field explicitly marks paywall statuses.

Q: How does this bypass PerimeterX? A: We combine stealth Playwright browsers with intelligent concurrency (batchSize). To minimize captchas effectively during bulk runs, integrating Residential Proxies is perfectly supported. If a block occurs, the actor gracefully saves the metadata.

Support 📧

• 📖 Apify Documentation
• 💬 Contact Support

About 👨‍💻

Built by MUSTAPHA LIAICHI - Automation & Web Scraping Specialist

---

Happy Scraping! 🚀