# Beehiiv Newsletter & Post Scraper (`daftvader/beehiiv-newsletter-scraper`) Actor

Extract public Beehiiv newsletter profiles and complete post archives without login or API keys.

- **URL**: https://apify.com/daftvader/beehiiv-newsletter-scraper.md
- **Developed by:** [Sam Towers](https://apify.com/daftvader) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Beehiiv Newsletter & Post Scraper

Turn public Beehiiv newsletters into structured data for research, analysis, monitoring, and downstream workflows. Supply one or more publication or post URLs and receive clean newsletter profiles and public post records in the default Apify dataset.

Use the Actor to build newsletter archives, compare publishing activity, research topics and authors, or collect public article text and metadata for further analysis.

### What you can collect

Depending on the selected mode, each run can return:

- **Newsletter profiles** — publication name, description, canonical URL, and image.
- **Public posts** — title, description, author, publication date, image, canonical URL, publication details, and public article content.
- **Full public content when needed** — extracted article text and HTML can be included or omitted to keep results smaller.

You can start from publication homepages, Beehiiv custom domains, or individual public post URLs.

Private, subscriber-only, email-only, and paywalled content is not bypassed.

### Common use cases

- Build a structured archive of a public Beehiiv newsletter.
- Research newsletter topics, titles, authors, and publishing dates.
- Compare public content across multiple Beehiiv publications.
- Collect post text for text analysis or other downstream processing.
- Extract publication profiles without collecting individual posts.
- Filter an archive to posts published after a date or matching selected keywords.

### How to use the Actor

Add one or more public Beehiiv publication or post URLs in `startUrls`, then choose what to collect:

- `posts` — collect public post records. This is the default.
- `profiles` — collect newsletter profile records only.
- `both` — collect profiles and posts.

Example input:

```json
{
  "startUrls": [{"url": "https://product.beehiiv.com"}],
  "mode": "both",
  "maxItems": 25,
  "includeContent": true,
  "publishedAfter": "2025-01-01",
  "keywords": ["growth", "newsletter"]
}
```

#### Main inputs

| Input | What it does |
| --- | --- |
| `startUrls` | Public Beehiiv publication homepages, custom-domain publication URLs, or individual post URLs. Required. |
| `mode` | Selects `posts`, `profiles`, or `both`. |
| `maxItems` | Sets the maximum number of post records to collect, from 1 to 10,000. Profile records are additional in `both` mode. |
| `includeContent` | Includes or omits public article text and HTML. |
| `publishedAfter` | Keeps posts published on or after the supplied ISO 8601 date or date-time. |
| `keywords` | Keeps posts matching at least one keyword in the title, description, or extracted content. |

Concurrency, retry, and timeout controls are also available for larger or more specialised runs.

### Results

Results are written to the default Apify dataset. Each item is either a `profile` or `post` record and includes `recordType` so mixed runs are easy to separate.

Example post result:

```json
{
  "recordType": "post",
  "url": "https://product.beehiiv.com/p/example",
  "publicationUrl": "https://product.beehiiv.com/",
  "publicationName": "beehiiv product updates",
  "title": "Example update",
  "description": "A concise preview",
  "author": "Example Author",
  "publishedAt": "2025-01-15T12:00:00Z",
  "imageUrl": "https://example.com/image.png",
  "contentText": "Public article text…",
  "contentHtml": "<article>…</article>"
}
```

Run metadata is available separately in the key-value store:

- `SUMMARY` — record, request, and failure counts.
- `FAILURES` — input-level errors when some supplied inputs could not be processed.

### Control your results

Use `maxItems` to cap the number of posts returned. Set `includeContent` to `false` when you only need titles, metadata, and URLs rather than full article bodies.

Use `publishedAfter` to focus on newer posts and `keywords` to keep records matching topics relevant to your research.

### Pricing

This Actor uses pay-per-result pricing at **USD $2.50 per 1,000 dataset results** (**$0.0025 per result**).

Each profile or post written to the default dataset counts as one billable result. `SUMMARY` and `FAILURES` are stored separately and are not billed as dataset results.

### Limitations

- Only publicly available content from supplied publications or post URLs is returned.
- Publishers can remove pages, change metadata, restrict publications, or alter their public sites.
- Custom domains must expose a Beehiiv-compatible `/sitemap.xml` for archive collection.
- The Actor does not discover unknown newsletters or browse Beehiiv's directory for you.

### Support

For a reproducible support report, include the sanitized Actor input, run ID, affected public URL, and expected versus actual behaviour. Do not include cookies, tokens, subscriber data, or private content.

# Actor input Schema

## `startUrls` (type: `array`):

Public Beehiiv publication homepages, custom-domain publication URLs, or individual post URLs.

## `mode` (type: `string`):

Choose whether to collect newsletter profiles, posts, or both.

## `maxItems` (type: `integer`):

Maximum number of post pages fetched per run. Profile records are additional when mode is both.

## `includeContent` (type: `boolean`):

Fetches each post page and includes its public article text and HTML.

## `publishedAfter` (type: `string`):

Optional ISO 8601 date or date-time. Older posts are omitted.

## `keywords` (type: `array`):

Keep posts matching at least one keyword in title, description, or extracted content.

## `maxConcurrency` (type: `integer`):

Maximum simultaneous HTTP requests.

## `maxRetries` (type: `integer`):

Retries after transient HTTP failures.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://product.beehiiv.com"
    }
  ],
  "mode": "posts",
  "maxItems": 100,
  "includeContent": true,
  "maxConcurrency": 8,
  "maxRetries": 2,
  "requestTimeoutSecs": 30
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `failures` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://product.beehiiv.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("daftvader/beehiiv-newsletter-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://product.beehiiv.com" }] }

# Run the Actor and wait for it to finish
run = client.actor("daftvader/beehiiv-newsletter-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://product.beehiiv.com"
    }
  ]
}' |
apify call daftvader/beehiiv-newsletter-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,daftvader/beehiiv-newsletter-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/0EGqEOVzZEW8Ani4e/builds/NzJE2TOBYOqNF192X/openapi.json