# Beehiiv Newsletter Scraper (`khadinakbar/beehiiv-newsletter-scraper`) Actor Scrape Beehiiv newsletters for posts, authors, newsletter metadata, and content discovery. MCP/API-ready. - **URL**: https://apify.com/khadinakbar/beehiiv-newsletter-scraper.md - **Developed by:** [Khadin Akbar](https://apify.com/khadinakbar) (community) - **Categories:** Lead generation, MCP servers, AI - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 newsletter record scrapeds This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## 🐝 Beehiiv Newsletter Scraper — Extract Posts, Metadata & Full Content ### What Does Beehiiv Newsletter Scraper Do? **Beehiiv Newsletter Scraper** extracts structured data from any public Beehiiv newsletter publication — no login required. Give it one or more Beehiiv newsletter URLs and it returns the publication's name, description, author info, social links, and a complete list of posts with titles, dates, slugs, and previews. With **full content mode**, it also scrapes the complete article text of every post — perfect for LLM training, competitive research, and content archiving. ### Why Use Beehiiv Newsletter Scraper? - **No login or cookies needed** — scrapes all public newsletter content without authentication - **Three extraction modes** — pay only for the data depth you actually need - **MCP-ready output** — semantic field names, consistent record shapes, instant AI agent compatibility - **Multi-publication batching** — scrape dozens of newsletters in a single run - **Flexible input** — publication homepage URLs, direct post URLs, or both combined ### What Data Can Beehiiv Newsletter Scraper Extract? | Field | Type | Description | |-------|------|-------------| | `record_type` | string | `newsletter` (publication metadata) or `post` (individual post) | | `publication_name` | string | Name of the newsletter | | `publication_url` | string | Homepage URL of the publication | | `description` | string | Publication tagline or post excerpt | | `author_name` | string | Creator/author name | | `logo_url` | string | Publication logo image URL | | `cover_image_url` | string | OG cover image URL | | `website_url` | string | Canonical publication URL | | `social_links` | object | Twitter, LinkedIn, Instagram, Facebook, YouTube, TikTok links | | `post_count` | integer | Number of posts found on the homepage | | `post_url` | string | Full URL of individual post | | `post_title` | string | Post headline | | `post_subtitle` | string | Post secondary heading | | `post_date` | string | ISO 8601 publication timestamp | | `post_preview` | string | Short excerpt (from OG description) | | `post_content` | string | Full post text (full mode only) | | `post_slug` | string | URL slug identifier | | `is_premium` | boolean | Whether post is behind a paywall | | `read_time_minutes` | integer | Estimated reading time | | `tags` | array | Post tags and topic categories | | `scraped_at` | string | ISO 8601 scrape timestamp | | `source_url` | string | URL that produced this record | --- ### How to Use Beehiiv Newsletter Scraper #### Step 1 — Provide Input URLs **Publication URLs** (scrape a whole newsletter): ```` https://aibreakfast.beehiiv.com https://bensbites.beehiiv.com https://tldrnewsletter.com ``` **Post URLs** (scrape specific posts): ``` https://aibreakfast.beehiiv.com/p/some-post-slug ```` You can mix and match both input types in the same run. #### Step 2 — Choose a Scrape Mode | Mode | Records Produced | Best For | |------|-----------------|----------| | `metadata` | 1 newsletter record per publication | Quick research, directory building | | `metadata_posts` | 1 newsletter + N post records | Lead gen, competitive intel, content analysis | | `full` | 1 newsletter + N post records with full content | LLM training, archiving, RAG pipelines | #### Step 3 — Set Limits - **Max Posts Per Publication** — controls how many posts to scrape per newsletter (default: 50) - **Max Total Results** — hard cap on total records returned across all publications (default: 100) --- ### Output Examples #### Newsletter Metadata Record ```json { "record_type": "newsletter", "publication_url": "https://aibreakfast.beehiiv.com", "publication_name": "AI Breakfast", "description": "The best AI newsletter — delivered daily", "author_name": "Devansh Mehta", "author_bio": null, "logo_url": "https://media.beehiiv.com/cdn-cgi/image/logo.png", "cover_image_url": "https://media.beehiiv.com/cdn-cgi/image/cover.png", "website_url": "https://aibreakfast.beehiiv.com", "social_links": { "twitter": "https://twitter.com/aibreakfast" }, "post_count": 142, "post_url": null, "post_title": null, "post_date": null, "is_premium": null, "tags": [], "scraped_at": "2026-04-08T10:00:00.000Z", "source_url": "https://aibreakfast.beehiiv.com" } ```` #### Post Record ```json { "record_type": "post", "publication_url": "https://aibreakfast.beehiiv.com", "publication_name": "AI Breakfast", "post_url": "https://aibreakfast.beehiiv.com/p/gpt-5-is-here", "post_title": "GPT-5 Just Dropped — Here's What You Need to Know", "post_subtitle": "Plus: Claude 4 leaks and Google's Gemini 2.0 update", "post_date": "2024-05-15T08:00:00.000Z", "post_preview": "Today we dive deep into GPT-5...", "post_slug": "gpt-5-is-here", "is_premium": false, "read_time_minutes": 5, "tags": ["AI", "GPT"], "scraped_at": "2026-04-08T10:01:00.000Z", "source_url": "https://aibreakfast.beehiiv.com/p/gpt-5-is-here" } ``` *** ### Pricing This actor uses **pay-per-event pricing**. | Event | Price | |-------|-------| | Actor start | $0.00005 | | Each record (newsletter or post) | $0.002 | #### Cost Examples | Task | Records | Estimated Cost | |------|---------|----------------| | Scrape 10 newsletter profiles (metadata only) | 10 | ~$0.02 | | Scrape 5 newsletters + 50 posts each | 255 | ~$0.51 | | Full archive: 10 newsletters × 100 posts | 1,010 | ~$2.02 | *** ### Run via API ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_API_TOKEN' }); const run = await client.actor('khadinakbar/beehiiv-newsletter-scraper').call({ publicationUrls: [ { url: 'https://aibreakfast.beehiiv.com' }, { url: 'https://bensbites.beehiiv.com' } ], scrapeMode: 'metadata_posts', maxPostsPerPublication: 50, maxResults: 200 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` ```python from apify_client import ApifyClient client = ApifyClient("YOUR_API_TOKEN") run = client.actor("khadinakbar/beehiiv-newsletter-scraper").call(run_input={ "publicationUrls": [ { "url": "https://aibreakfast.beehiiv.com" } ], "scrapeMode": "full", "maxPostsPerPublication": 20, "maxResults": 50 }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item["post_title"], item["post_date"]) ``` *** ### Use Cases **Newsletter Research & Competitive Intelligence** — Scrape competing newsletters to analyze their content strategy, posting frequency, and topics covered. **LLM Training Data** — Use full content mode to build high-quality newsletter corpora for fine-tuning or RAG pipelines. Beehiiv newsletters are generally high-signal, expert-written content. **Lead Generation & Outreach** — Collect author names, social profiles, and website URLs from newsletters in your niche for B2B outreach campaigns. **Content Archiving** — Backup newsletter archives before they go behind paywalls or disappear. **Newsletter Discovery** — Build directories of Beehiiv newsletters in specific niches by scraping known publication URLs. *** ### FAQ **Does this actor require a login or Beehiiv account?** No. It scrapes publicly available newsletter content only. Premium/paid subscriber posts are detected (the `is_premium` field will be `true`) but their content cannot be extracted without authentication. **What Beehiiv URL formats are supported?** Standard Beehiiv subdomains (`https://name.beehiiv.com`), custom domains that run on Beehiiv, and direct post URLs (`https://name.beehiiv.com/p/post-slug`). **Why is `post_content` null even in metadata\_posts mode?** Full article content is only extracted when `scrapeMode` is set to `full`. In `metadata` and `metadata_posts` modes, `post_content` is always `null` to save cost and time. **Why is `subscriber_count` always null?** Beehiiv does not publicly display subscriber counts on publication pages — this information is only available inside the creator's dashboard. The field is included in the schema for future compatibility. **What happens with paywalled posts?** The actor detects paywalled posts via multiple signals (subscribe-to-read prompts, paywall CSS classes) and sets `is_premium: true`. The post metadata (title, date, preview) is still extracted. Full content behind a paywall cannot be accessed without authentication. **The actor returned fewer posts than I expected.** The actor collects post links from the publication homepage. If the newsletter has a large archive, only the posts visible on the homepage (typically 20-30) will be discovered per page. Pagination is handled automatically. Increase `maxPostsPerPublication` to get more posts. *** ### Legal Disclaimer *This actor is intended for lawful data collection from publicly available sources. Users are responsible for compliance with applicable laws, terms of service, and data protection regulations (GDPR, CCPA, etc.). The actor only accesses public-facing web pages and does not bypass any authentication or access controls. Do not use this actor to collect personal data in violation of applicable privacy laws.* *** ### Works Great With - **[Shopify All-in-One Scraper](https://apify.com/khadinakbar/shopify-all-in-one-scraper)** — Research what products newsletter sponsors are selling - **[Google Jobs Scraper](https://apify.com/khadinakbar/google-jobs-scraper)** — Find jobs at newsletter companies - **[Instagram Reels Scraper](https://apify.com/khadinakbar/instagram-reels-scraper)** — Complement newsletter research with social content data Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools. # Actor input Schema ## `publicationUrls` (type: `array`): Use this when you have Beehiiv newsletter homepage URLs (e.g. https://aibreakfast.beehiiv.com). The actor will scrape publication metadata and, based on scrapeMode, also enumerate and scrape posts. Use postUrls instead when you have direct post links. ## `postUrls` (type: `array`): Use this when you have direct Beehiiv post URLs (e.g. https://aibreakfast.beehiiv.com/p/some-post-slug). Scrapes individual posts directly. Use publicationUrls instead to discover all posts from a newsletter homepage. ## `scrapeMode` (type: `string`): Controls how much data to extract. 'metadata' = newsletter info only (fastest, cheapest). 'metadata\_posts' = newsletter info + full post list with dates and previews (recommended for most use cases). 'full' = everything including full post content (best for LLM training, research, archiving). ## `maxPostsPerPublication` (type: `integer`): Maximum number of posts to scrape per newsletter publication. Lower values reduce cost and run time. Set to 500 for full archive scraping. ## `maxResults` (type: `integer`): Maximum total number of records (newsletter + post records combined) to return across all input publications. The actor stops as soon as this limit is reached. ## Actor input object example ```json { "publicationUrls": [ { "url": "https://aibreakfast.beehiiv.com" } ], "scrapeMode": "metadata_posts", "maxPostsPerPublication": 10, "maxResults": 20 } ``` # Actor output Schema ## `results` (type: `string`): Scraped newsletter and post records. Filter by record\_type: 'newsletter' or 'post'. Fields include publication\_name, publication\_url, post\_title, post\_url, post\_preview, author, post\_date, scraped\_at, and more. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "publicationUrls": [ { "url": "https://aibreakfast.beehiiv.com" } ], "scrapeMode": "metadata_posts", "maxPostsPerPublication": 10, "maxResults": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("khadinakbar/beehiiv-newsletter-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "publicationUrls": [{ "url": "https://aibreakfast.beehiiv.com" }], "scrapeMode": "metadata_posts", "maxPostsPerPublication": 10, "maxResults": 20, } # Run the Actor and wait for it to finish run = client.actor("khadinakbar/beehiiv-newsletter-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "publicationUrls": [ { "url": "https://aibreakfast.beehiiv.com" } ], "scrapeMode": "metadata_posts", "maxPostsPerPublication": 10, "maxResults": 20 }' | apify call khadinakbar/beehiiv-newsletter-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=khadinakbar/beehiiv-newsletter-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Beehiiv Newsletter Scraper", "description": "Scrape Beehiiv newsletters for posts, authors, newsletter metadata, and content discovery. MCP/API-ready.", "version": "1.0", "x-build-id": "5eMv5TojFqsdRgDep" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/khadinakbar~beehiiv-newsletter-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-khadinakbar-beehiiv-newsletter-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/khadinakbar~beehiiv-newsletter-scraper/runs": { "post": { "operationId": "runs-sync-khadinakbar-beehiiv-newsletter-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/khadinakbar~beehiiv-newsletter-scraper/run-sync": { "post": { "operationId": "run-sync-khadinakbar-beehiiv-newsletter-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "publicationUrls": { "title": "Publication URLs", "type": "array", "description": "Use this when you have Beehiiv newsletter homepage URLs (e.g. https://aibreakfast.beehiiv.com). The actor will scrape publication metadata and, based on scrapeMode, also enumerate and scrape posts. Use postUrls instead when you have direct post links.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "postUrls": { "title": "Post URLs", "type": "array", "description": "Use this when you have direct Beehiiv post URLs (e.g. https://aibreakfast.beehiiv.com/p/some-post-slug). Scrapes individual posts directly. Use publicationUrls instead to discover all posts from a newsletter homepage.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "scrapeMode": { "title": "Scrape Mode", "enum": [ "metadata", "metadata_posts", "full" ], "type": "string", "description": "Controls how much data to extract. 'metadata' = newsletter info only (fastest, cheapest). 'metadata_posts' = newsletter info + full post list with dates and previews (recommended for most use cases). 'full' = everything including full post content (best for LLM training, research, archiving).", "default": "metadata_posts" }, "maxPostsPerPublication": { "title": "Max Posts Per Publication", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of posts to scrape per newsletter publication. Lower values reduce cost and run time. Set to 500 for full archive scraping.", "default": 50 }, "maxResults": { "title": "Max Total Results", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Maximum total number of records (newsletter + post records combined) to return across all input publications. The actor stops as soon as this limit is reached.", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```