# Public Webpage HTML Downloader (`automation-lab/public-webpage-html-downloader`) Actor

Download raw or browser-rendered HTML files from batches of public webpage URLs with final URL, HTTP status, byte size, retrieval mode, and bounded errors.

- **URL**: https://apify.com/automation-lab/public-webpage-html-downloader.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Developer tools, Automation, Integrations
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.48 / 1,000 webpage downloadeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Public Webpage HTML Downloader

Download raw response HTML or browser-rendered DOM files from batches of anonymous public webpage URLs.
Each successful page is saved as an individual `.HTML` file, while the default dataset provides its final URL, HTTP status, content type, byte size, retrieval mode, title, timestamp, and download link.

This Actor is designed for repeatable webpage HTML export: archival jobs, downstream parsers, migration audits, evidence capture, and scheduled data pipelines.
It handles a webpage HTML only—rather than images, stylesheets, videos, or a full offline website bundle—so results stay focused and integration-friendly.

### What does Public Webpage HTML Downloader do?

For every supplied URL, the Actor:

1. validates that the target is a public HTTP or HTTPS address;
2. downloads the server response in `raw` mode or opens it in Chromium in `rendered` mode;
3. follows only redirects that remain on public addresses;
4. enforces a configurable timeout and HTML size ceiling;
5. saves successful HTML to the run's key-value store;
6. writes retrieval metadata or a bounded error to the default dataset.

Unlike a generic scraper template, the output is ready to archive or pass directly to another parser.
Duplicate input URLs are processed once, private-network targets are rejected, and an error for one URL can be isolated without losing the rest of a batch.

### Who is it for?

- **Data engineers** staging HTML before parsing or enrichment.
- **Archivists and researchers** preserving public page snapshots with retrieval metadata.
- **SEO and migration teams** collecting source documents before link, metadata, or template audits.
- **Automation builders** passing stored HTML links into Make, Zapier, n8n, Python, or Apify workflows.
- **Developers** reproducing public-page responses and comparing raw HTML with the browser DOM.

Use a source-specific scraper when you need normalized products, reviews, profiles, or other domain entities instead of HTML files.

### Raw HTML or browser-rendered HTML?

| Mode | Best for | What is stored | Resource profile |
|---|---|---|---|
| `raw` | Server-rendered pages, APIs returning HTML, efficient bulk archives | Exact decoded HTML response after redirects | Fastest and cheapest |
| `rendered` | JavaScript-driven pages whose useful DOM appears after load | Chromium DOM from `page.content()` | More compute-intensive |

Rendered mode blocks images, media, and fonts to reduce transfer and runtime.
It does not automatically switch to a proxy, residential IP, login, CAPTCHA solver, or another paid route.
A protected page returns an explicit error rather than creating an unexpected cost.

### Input parameters

| Field | Type | Default | Description |
|---|---|---:|---|
| `startUrls` | array | required | Public HTTP(S) webpage URLs. Request-list objects and plain strings are accepted. |
| `retrievalMode` | `raw` or `rendered` | `raw` | Select server response HTML or a Chromium-rendered DOM. |
| `maxItems` | integer | `20` | Maximum unique URLs processed, from 1 to 1,000. |
| `requestTimeoutSecs` | integer | `30` | Per-URL timeout, from 5 to 120 seconds. |
| `renderWaitSecs` | number | `0` | Rendered mode only: extra wait after DOMContentLoaded, from 0 to 30 seconds. |
| `maxHtmlBytes` | integer | `5000000` | Per-page safety ceiling, from 10 KB to 20 MB. |
| `continueOnError` | boolean | `true` | Save an uncharged error row and continue, or fail on the first URL error. |

Localhost, private IP ranges, URL credentials, and non-HTTP protocols are rejected.
The Actor accepts supplied URLs only; it does not crawl links discovered within a page.

### How to download webpage HTML

1. Open the Actor in Apify Console.
2. Add one or more anonymously reachable public webpages under **Public webpage URLs**.
3. Keep **Raw HTTP response** for normal HTML pages, or choose **Browser-rendered DOM** for JavaScript content.
4. Keep the default 5 MB limit unless you know larger documents are required.
5. Click **Start**.
6. Open **Downloaded webpage metadata** for status and file links.
7. Open **Stored HTML files** when you need all files in the run's key-value store.

A useful first run is:

```json
{
  "startUrls": [
    { "url": "https://en.wikipedia.org/wiki/Web_scraping" },
    { "url": "https://news.ycombinator.com/" }
  ],
  "retrievalMode": "raw",
  "maxItems": 2
}
```

### Output data

The default dataset contains one row per processed unique URL.
A representative successful record is:

```json
{
  "sourceUrl": "https://news.ycombinator.com/",
  "finalUrl": "https://news.ycombinator.com/",
  "statusCode": 200,
  "contentType": "text/html; charset=utf-8",
  "byteSize": 34502,
  "retrievalMode": "raw",
  "retrievedAt": "2026-08-21T14:10:00.000Z",
  "title": "Hacker News",
  "htmlKey": "PAGE_0002_0f63a2a5a562.HTML",
  "htmlUrl": "https://api.apify.com/v2/key-value-stores/STORE_ID/records/PAGE_0002_0f63a2a5a562.HTML",
  "status": "success",
  "errorType": null,
  "errorMessage": null
}
```

`htmlKey` and `htmlUrl` are null for errors.
`statusCode`, `finalUrl`, and content metadata may also be null when no valid HTTP response was received.
Error messages are bounded to 500 characters and are intended for diagnosis, not as stored response bodies.

### How are HTML files named and stored?

Successful files use keys such as `PAGE_0001_9e0c729bb0e8.HTML`.
The sequence preserves input order and the short hash keeps names stable enough to identify the source without embedding a long URL.
Files live in each run's default key-value store, so separate runs do not mix archives.

Dataset exports contain metadata, not the full HTML body.
This keeps CSV, Excel, JSON, and integration payloads manageable while preserving direct file access through `htmlUrl`.
Storage retention follows your Apify account and storage settings.

### How much does it cost to download public webpage HTML?

Pay-per-event pricing includes a **$0.001 run-start charge** and one `Webpage downloaded` event for every successfully stored HTML file.
Error rows are not charged as webpage downloads.
The per-page rate decreases by Apify subscription tier:

| Tier | Price per successful page |
|---|---:|
| FREE | $0.00092 |
| BRONZE | $0.00080 |
| SILVER | $0.000624 |
| GOLD | $0.00048 |
| PLATINUM | $0.00032 |
| DIAMOND | $0.000224 |

Examples at the BRONZE rate:

- 1 successful page: about **$0.0018** including run start.
- 10 successful pages: about **$0.009** including run start.
- 100 successful pages: about **$0.081** including run start.

Compute-heavy rendered runs still use the same event curve; the price is based on measured safe operation.
Your exact total depends on successful files and your active Apify tier.

### Batch archival and downstream workflows

Common patterns include:

1. **Scheduled archive** — run daily with a stable URL list, then copy `htmlUrl`, `retrievedAt`, and `statusCode` into an archive index.
2. **Parser staging** — download pages once and let several extraction jobs consume the same immutable run files.
3. **Raw-versus-rendered audit** — run the same URLs in each mode and compare `byteSize`, title, and stored DOM.
4. **Website migration evidence** — preserve selected old pages before a deployment and selected new pages afterward.
5. **Failure queue** — filter `status = error`, inspect `errorType`, and retry only the affected public URLs with an adjusted timeout or mode.

The Actor does not calculate changes between runs.
For versioned diffs, use [Website HTML & Text Change Monitor](https://apify.com/automation-lab/website-html-text-change-monitor).

### Run with the Apify API using cURL

Replace `APIFY_TOKEN` with your token:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~public-webpage-html-downloader/runs?token=APIFY_TOKEN&waitForFinish=120" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{"url":"https://news.ycombinator.com/"}],
    "retrievalMode": "raw",
    "maxItems": 1
  }'
```

Use the returned `defaultDatasetId` for metadata and `defaultKeyValueStoreId` for HTML files.
For asynchronous production jobs, omit `waitForFinish` and poll the run endpoint.

### Run with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/public-webpage-html-downloader').call({
  startUrls: [
    { url: 'https://en.wikipedia.org/wiki/Web_scraping' },
    { url: 'https://news.ycombinator.com/' },
  ],
  retrievalMode: 'raw',
  maxItems: 2,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map(({ sourceUrl, htmlUrl, status }) => ({ sourceUrl, htmlUrl, status })));
```

Download a successful file from its `htmlUrl`, or use `client.keyValueStore(run.defaultKeyValueStoreId).getRecord(htmlKey)`.

### Run with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/public-webpage-html-downloader').call(run_input={
    'startUrls': [{'url': 'https://docs.npmjs.com/about-npm'}],
    'retrievalMode': 'raw',
    'maxItems': 1,
})

for row in client.dataset(run['defaultDatasetId']).iterate_items():
    print(row['status'], row.get('htmlUrl'))
```

Keep tokens in environment variables or a secret manager, never in source control.

### Use with Apify MCP

Add the Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/public-webpage-html-downloader"
```

#### Claude Desktop, Cursor, and VS Code setup

Claude Desktop, Cursor, and VS Code can use this MCP configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/public-webpage-html-downloader"
    }
  }
}
```

Example prompts:

- "Download raw HTML for these three public documentation URLs and return the stored file links."
- "Capture the browser-rendered DOM of this public JavaScript page and report its final URL and byte size."
- "Run the HTML archive and list only URLs with bounded retrieval errors."

### Reliability, limits, and error behavior

The Actor intentionally uses bounded behavior:

- up to 1,000 unique URLs per run;
- up to 20 MB of HTML per page;
- up to 120 seconds per URL;
- up to 30 seconds of extra render wait;
- no recursive link crawling;
- no automatic proxy or login fallback;
- sequential processing to favor stability and bounded browser memory.

`continueOnError: true` saves errors alongside successes.
Set it to `false` when a pipeline should fail fast.
HTTP error pages that contain HTML can still be stored successfully with their actual status code, which is useful for auditing 404 or 503 documents.
Non-HTML responses are rejected.

### Troubleshooting

**The stored page is an empty JavaScript shell.**
Use `retrievalMode: "rendered"` and, if necessary, add a small `renderWaitSecs` value.
The Actor waits for DOMContentLoaded before the optional delay.

**The page times out.**
Confirm it is anonymously reachable, then raise `requestTimeoutSecs` within the 120-second limit.
Do not repeatedly retry a login wall or CAPTCHA.

**I received `PRIVATE_ADDRESS`.**
The hostname resolved to localhost, a private network, link-local space, or another non-public address.
This safety boundary cannot be disabled.

**I received `HTML_TOO_LARGE`.**
Raise `maxHtmlBytes` only if you trust the public source and need the entire document.
The maximum accepted value is 20 MB.

**The dataset has an error row but no HTML file.**
Files are created only after a valid HTML result passes all checks.
Use `errorType` and the run log to diagnose that URL.

### Responsible and legal use

Download only public pages you are authorized to access and process.
Respect website terms, robots guidance where applicable, copyright, database rights, rate limits, and privacy laws.
Do not use the Actor to bypass authentication, paywalls, access controls, or technical protections.

HTML can contain personal data, copyrighted material, scripts, and unsafe markup.
Treat stored files as untrusted input: sanitize before displaying them, and avoid executing downloaded scripts in privileged environments.
You are responsible for retention and deletion policies for your run storage.

### Related Automation Lab Actors

- [Website HTML & Text Change Monitor](https://apify.com/automation-lab/website-html-text-change-monitor) — create versioned snapshots and machine-readable page changes.
- [Webpage Text Extractor](https://apify.com/automation-lab/webpage-text-extractor) — return readable text instead of raw HTML files.
- [HTML Table to Excel Exporter](https://apify.com/automation-lab/html-table-to-excel-exporter) — turn native webpage tables into XLSX worksheets.

These products provide normalized or comparative outputs.
Choose Public Webpage HTML Downloader when the reusable HTML document itself is the required artifact.

### FAQ

#### Does it download a complete website with CSS, images, and JavaScript files?

No. It stores one HTML document per supplied URL.
It does not create an offline mirror or rewrite asset links.

#### Can it download pages behind a login?

No. The supported scope is anonymously reachable public webpages.
There are no credential or cookie inputs.

#### Does rendered mode capture the visual page?

It captures the post-JavaScript DOM as HTML, not a screenshot, PDF, video, or network archive.

#### Are failed URLs charged as downloaded webpages?

No. A bounded error row is saved for diagnosis, but the per-page event is charged only after HTML has been stored successfully.
The one-time run-start event still applies.

#### Can I schedule recurring archives?

Yes. Save an Actor Task with a stable URL list and attach an Apify schedule.
Each run gets its own dataset and key-value store identifiers for downstream indexing.

#### Why is raw mode the default?

Raw mode is faster, cheaper, and closer to the response a downstream parser receives.
Use rendered mode only when JavaScript-generated DOM content is required.

# Actor input Schema

## `startUrls` (type: `array`):

HTTP or HTTPS pages to download. Duplicate URLs are processed once; local and private-network addresses are rejected.

## `retrievalMode` (type: `string`):

Raw downloads the server response efficiently. Rendered opens each page in Chromium and saves the DOM after JavaScript execution.

## `maxItems` (type: `integer`):

Maximum number of unique input URLs to process in this run.

## `requestTimeoutSecs` (type: `integer`):

Maximum retrieval time for each page.

## `renderWaitSecs` (type: `number`):

Rendered mode only: additional bounded wait after DOMContentLoaded for delayed JavaScript content.

## `maxHtmlBytes` (type: `integer`):

Reject a response or rendered DOM larger than this safety limit.

## `continueOnError` (type: `boolean`):

Save a bounded error row and continue with the remaining URLs. Disable to fail the run on the first retrieval error.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    },
    {
      "url": "https://news.ycombinator.com/"
    }
  ],
  "retrievalMode": "raw",
  "maxItems": 20,
  "requestTimeoutSecs": 30,
  "renderWaitSecs": 0,
  "maxHtmlBytes": 5000000,
  "continueOnError": true
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing one success or bounded error row per processed URL.

## `htmlFiles` (type: `string`):

Key-value store containing HTML files for successful downloads.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://en.wikipedia.org/wiki/Web_scraping"
        },
        {
            "url": "https://news.ycombinator.com/"
        }
    ],
    "retrievalMode": "raw"
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/public-webpage-html-downloader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [
        { "url": "https://en.wikipedia.org/wiki/Web_scraping" },
        { "url": "https://news.ycombinator.com/" },
    ],
    "retrievalMode": "raw",
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/public-webpage-html-downloader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    },
    {
      "url": "https://news.ycombinator.com/"
    }
  ],
  "retrievalMode": "raw"
}' |
apify call automation-lab/public-webpage-html-downloader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/public-webpage-html-downloader"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/SC6HzcmVAbFlmQKja/builds/HFmbY7oJmigdgUVEE/openapi.json