Beehiiv Scraper - Newsletters, Posts & Content

Extract public Beehiiv newsletter posts from publication homepages, custom domains, sitemaps, and direct post URLs. Get authors, publication details, dates, images, full public text, HTML, Markdown, and cited links without a Beehiiv API key.

Use this Beehiiv scraper for newsletter monitoring, competitive intelligence, content research, lead generation, archives, and AI or RAG datasets.

Premium content is not unlocked. The Actor returns public metadata or preview content when Beehiiv marks a post as paywalled.

What does Beehiiv Newsletter Scraper do?

The Actor turns one or more Beehiiv-powered sites into structured post records. It supports standard *.beehiiv.com publications and Beehiiv sites hosted on custom domains.

• Discover posts through each publication's public sitemap
• Scrape a direct /p/<slug> URL without crawling the full archive
• Extract titles, descriptions, authors, publication dates, images, and tags
• Return public article content as plain text, Markdown, and HTML
• Collect unique outbound links cited in each post
• Identify free and premium posts without bypassing access controls
• Filter by keyword, publication date, access level, and reading time
• Process multiple publications concurrently
• Store run totals and failures in a STATS record

The Actor uses lightweight HTTP extraction rather than launching a browser for every post. This keeps normal runs fast and reduces memory requirements.

Use cases

Newsletter competitive intelligence

• Monitor competitor publishing cadence and editorial themes.
• Compare authors, headlines, reading time, and cited sources.
• Schedule recurring runs across a publication watchlist.

Content and market research

• Search public posts for products, companies, or topics.
• Build a reviewable archive of recent newsletter coverage.
• Export structured data to Sheets, a CRM, or a warehouse.

AI and RAG datasets

• Create a public newsletter corpus in clean Markdown or text.
• Preserve canonical URLs and timestamps for provenance.
• Use word counts to estimate embedding or model-processing costs.

Sponsorship and lead research

• Identify publications and authors active in a target niche.
• Review public content before sponsorship or partnership outreach.
• Analyze outbound brands and sources mentioned in newsletters.

Data you can extract

Category	Fields
Post	post_id, slug, post_url, headline, subheading, description
Publication	beehiiv_url, publication ID, name, domain, logo, social URLs
Author	author_name, authors, author_urls, author_image_urls
Dates and access	published_at, updated_at, audience, is_paywalled, content_type
Content	article_text, article_markdown, article_html, word_count
Media and taxonomy	cover_image_url, tags
Link intelligence	outbound_links
Provenance	scraped_at

Optional values are omitted or left empty when the public page does not expose them. The Actor does not invent engagement metrics, subscriber counts, dates, or authors.

How to scrape Beehiiv newsletters

1. Open Beehiiv Newsletter Scraper in Apify Console.
2. Add a publication homepage, custom domain, sitemap, or direct post URL.
3. Keep the prefilled three-post limit for the first run.
4. Optionally set keywords, dates, access, or reading-time filters.
5. Disable HTML when you only need text or Markdown.
6. Click Start.
7. Review results in the Dataset tab.
8. Export the dataset as JSON, CSV, Excel, XML, or RSS.
9. Create an Apify schedule for recurring monitoring.

Minimal input:

{
  "beehiivUrls": ["https://superhuman.beehiiv.com"],
  "maxPostsPerPublication": 10,
  "maxItems": 10
}

Direct post extraction:

{
  "beehiivUrls": [
    "https://www.superhuman.ai/p/robotics-special-unitree-lands-a-blockbuster-ipo"
  ],
  "maxItems": 1,
  "includeMarkdown": true,
  "includeHtml": false
}

Topic monitoring:

{
  "beehiivUrls": [
    "https://superhuman.beehiiv.com",
    "https://tldrnewsletter.beehiiv.com"
  ],
  "maxPostsPerPublication": 50,
  "maxItems": 50,
  "publishedAfter": "2026-08-01",
  "keywords": ["OpenAI", "Claude", "agents"],
  "onlyFree": true,
  "includeOutboundLinks": true
}

Input parameters

Parameter	Type	Default	Description
beehiivUrls	array	required	Publication, custom-domain, sitemap, or direct post URLs.
maxPostsPerPublication	integer	50	Maximum newest sitemap posts per publication; 0 means full sitemap.
maxItems	integer	50	Total matching posts saved; 0 allows up to 10,000.
keywords	string array	[]	Keep posts matching any keyword.
publishedAfter	date	empty	Inclusive lower publication-date bound.
publishedBefore	date	empty	Inclusive upper publication-date bound.
onlyFree	boolean	false	Exclude premium or paywalled posts.
minReadingTime	integer	0	Minimum estimated reading time in minutes.
maxReadingTime	integer	0	Maximum reading time; 0 means no limit.
includeContent	boolean	true	Return public article text and selected formats.
includeMarkdown	boolean	true	Include clean Markdown.
includeHtml	boolean	true	Include public article HTML.
includeOutboundLinks	boolean	true	Include external links cited in the article.
postConcurrency	integer	10	Parallel post requests, from 1 to 25.
proxyConfiguration	object	disabled	Optional proxy retry for protected custom domains.

The legacy batchSize, startDate, and endDate fields remain accepted for existing integrations.

Output example

{
  "record_type": "post",
  "beehiiv_url": "https://www.superhuman.ai/",
  "post_url": "https://www.superhuman.ai/p/robotics-special-unitree-lands-a-blockbuster-ipo",
  "post_id": "robotics-special-unitree-lands-a-blockbuster-ipo",
  "slug": "robotics-special-unitree-lands-a-blockbuster-ipo",
  "headline": "Robotics Special: Unitree lands a blockbuster IPO",
  "published_at": "2026-08-22T14:00:00.000Z",
  "audience": "free",
  "is_paywalled": false,
  "estimated_reading_time": 5,
  "word_count": 1110,
  "author_name": "Zain Kahn",
  "authors": ["Zain Kahn"],
  "cover_image_url": "https://media.beehiiv.com/...",
  "publication": {
    "id": "faa6a747-8c1c-43c1-8155-91aa43268f01",
    "name": "Superhuman AI",
    "url": "https://www.superhuman.ai/",
    "domain": "www.superhuman.ai"
  },
  "article_text": "Public article text...",
  "article_markdown": "Public article Markdown...",
  "outbound_links": ["https://example.com/source"],
  "content_type": "full",
  "scraped_at": "2026-08-23T12:00:00+00:00"
}

content_type can be full, preview_only, metadata_only, or unavailable.

Pricing

This Actor uses pay-per-event pricing:

Event	Price
Actor start	$0.00005
One dataset result	$0.005

At the current rate, 100 results cost about $0.50 and 1,000 results cost about $5.00, plus the small start event. Check the live Pricing tab for current rates before a large run.

API examples

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("scraper_guru/beehiiv-scraper").call(run_input={
    "beehiivUrls": ["https://superhuman.beehiiv.com"],
    "maxPostsPerPublication": 10,
    "maxItems": 10,
    "includeHtml": False,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["headline"], item["post_url"])

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('scraper_guru/beehiiv-scraper').call({
  beehiivUrls: ['https://superhuman.beehiiv.com'],
  maxPostsPerPublication: 10,
  maxItems: 10,
  includeHtml: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/scraper_guru~beehiiv-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "beehiivUrls": ["https://superhuman.beehiiv.com"],
    "maxPostsPerPublication": 10,
    "maxItems": 10
  }'

Keep API tokens in environment variables or a secret manager.

Limitations and responsible use

• The Actor extracts publicly accessible Beehiiv pages only.
• It does not log in, subscribe, or bypass paywalls.
• Protected custom domains may require Apify Proxy.
• Site owners can change templates or block automated requests.
• Public metadata varies by publication and post design.
• Respect copyright, privacy, applicable law, and each site's terms.

FAQ

Do I need a Beehiiv API key?

No. The Actor reads public sitemaps and server-rendered post pages.

Does it support custom Beehiiv domains?

Yes. Add the publication's public custom domain. If direct requests are rejected, enable the proxy option.

Can it scrape a single post?

Yes. Add any public /p/<slug> URL. Sitemap discovery is skipped.

Can it scrape premium posts?

It records public metadata or preview content when available. It never unlocks protected content.

Why does a result have no article body?

The post may be premium, unavailable, removed, or protected by the publication's hosting configuration. Check content_type and the run log.

How can I monitor new posts?

Create an Apify Task from a working input, schedule it daily or weekly, and use a date window or dataset deduplication downstream.

Which export formats are available?

Apify datasets support JSON, CSV, Excel, XML, and RSS exports.

Is scraping Beehiiv legal?

Public web scraping rules depend on jurisdiction and use. Respect access controls, copyright, personal data, and applicable terms.

---

Built by Mustapha Liaichi, Automation and Web Scraping Specialist.