# Headless Browser HTML Scraper (`patel_dev_automation/browserless-html-scraper`) Actor
Render any URL in a real headless browser and return the fully-rendered HTML, the page text, or a selected area by CSS selector. Scroll for lazy content, wait for elements, and capture screenshots. A browserless-style HTML API on Apify.
- **URL**: https://apify.com/patel\_dev\_automation/browserless-html-scraper.md
- **Developed by:** [Dev Patel](https://apify.com/patel_dev_automation) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet
## Pricing
from $1.00 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Headless Browser HTML Scraper
A generic, **browserless-style HTML API**. Give it any URL and it opens a real headless Chromium browser, fully renders the JavaScript, optionally scrolls and waits, then returns the **full rendered HTML** โ or just a **selected area** by CSS selector.
Think of it as a self-hosted [browserless.io](https://www.browserless.io/) `/content` + `/scrape` on Apify.
### What it does
- ๐ Renders **any URL** with a real browser (JavaScript executed)
- ๐งฉ **Selected area** โ pass a CSS selector and get every matching element's HTML, text, attributes, and position
- ๐ **Scroll to bottom** โ trigger infinite-scroll / lazy-loaded content with real wheel events
- โณ **Wait** for a selector, a load event, or a fixed delay
- ๐ผ๏ธ Optional **full-page screenshot**
- ๐ซ Block images/media/fonts/CSS to speed up and cut bandwidth
- ๐ Use it synchronously as an API (`run-sync-get-dataset-items`)
### Input
| Field | Type | Description |
|-------|------|-------------|
| `urls` | array | **Required.** URLs to render and scrape. |
| `selector` | string | Optional CSS selector for the "selected area". Returns each match's HTML/text/attributes/position. Empty = full page only. |
| `scrollToBottom` | boolean | Scroll down to load lazy content. Default `false`. |
| `maxScrolls` | integer | Max scroll rounds when scrolling. Default `15`. |
| `waitForSelector` | string | Wait until this selector appears (โค30s). |
| `waitUntil` | enum | `domcontentloaded` (default) ยท `load` ยท `networkidle`. |
| `waitMs` | integer | Extra fixed wait after load (ms). |
| `blockResources` | array | Resource types to block. Default `["media","font"]`. |
| `returnFullHtml` | boolean | Include full rendered HTML. Default `true`. |
| `returnText` | boolean | Include page visible text. Default `true`. |
| `includeScreenshot` | boolean | Capture a full-page screenshot and return its URL. Default `false`. |
| `proxyConfiguration` | object | Apify Proxy (datacenter) by default; use Residential for bot-protected sites. |
#### Example: full HTML of a JS-rendered page
```json
{ "urls": [{ "url": "https://www.example.com" }], "waitUntil": "networkidle" }
````
#### Example: extract a selected area, after scrolling
```json
{
"urls": [{ "url": "https://news.ycombinator.com" }],
"selector": "span.titleline a",
"scrollToBottom": true
}
```
### Output
One record per URL:
```json
{
"url": "https://www.example.com",
"loadedUrl": "https://www.example.com/",
"statusCode": 200,
"title": "Example Domain",
"html": "...",
"text": "Example Domain\nThis domain is for use in...",
"selectedCount": 30,
"selectedElements": [
{
"text": "Some headline",
"html": "Some headline",
"attributes": [{ "name": "href", "value": "https://..." }],
"width": 320, "height": 18, "top": 140, "left": 24
}
],
"screenshotUrl": "https://api.apify.com/v2/key-value-stores/.../records/screenshot-1",
"scrapedAt": "2026-06-13T08:00:00.000Z"
}
```
### Use as an API
```bash
curl -X POST "https://api.apify.com/v2/acts/USERNAME~browserless-html-scraper/run-sync-get-dataset-items?token=TOKEN" \
-H "Content-Type: application/json" \
-d '{"urls":[{"url":"https://www.example.com"}],"selector":"h1"}'
```
### Notes
- For bot-protected sites, switch `proxyConfiguration` to **Residential**.
- Blocking `image`/`stylesheet` speeds things up but can break layout-dependent lazy scrolling on some sites โ keep them enabled (don't block) when using `scrollToBottom` on such pages.
# Actor input Schema
## `urls` (type: `array`):
One or more URLs to open in a real headless browser and scrape. JavaScript is fully rendered.
## `selector` (type: `string`):
Optional. A CSS selector for the area you want to extract (like browserless /scrape). Every matching element is returned with its HTML, text, attributes, and position. Leave empty to just get the full page HTML.
## `scrollToBottom` (type: `boolean`):
Scroll the page down with real wheel events to trigger infinite-scroll / lazy-loaded content before scraping.
## `maxScrolls` (type: `integer`):
Maximum scroll iterations when 'Scroll to bottom' is on. Stops early when the page stops growing.
## `waitForSelector` (type: `string`):
Optional. Wait until this CSS selector appears before scraping (up to 30s).
## `waitUntil` (type: `string`):
When to consider navigation finished.
## `waitMs` (type: `integer`):
Optional fixed wait after the page loads, in milliseconds (max 60000).
## `blockResources` (type: `array`):
Resource types to block to speed up loading and cut bandwidth. Blocking 'image'/'stylesheet' can break layout-dependent lazy scrolling on some sites.
## `returnFullHtml` (type: `boolean`):
Include the full rendered page HTML in the output (like browserless /content).
## `returnText` (type: `boolean`):
Include the page's visible text (innerText) in the output.
## `includeScreenshot` (type: `boolean`):
Take a full-page JPEG screenshot, store it, and return its URL in the output.
## `proxyConfiguration` (type: `object`):
Proxy to route requests through. Apify Proxy (datacenter) by default; switch to Residential for bot-protected sites.
## Actor input object example
```json
{
"urls": [
{
"url": "https://www.example.com"
}
],
"scrollToBottom": false,
"maxScrolls": 15,
"waitUntil": "domcontentloaded",
"waitMs": 0,
"blockResources": [
"media",
"font"
],
"returnFullHtml": true,
"returnText": true,
"includeScreenshot": false,
"proxyConfiguration": {
"useApifyProxy": true
}
}
```
# Actor output Schema
## `pages` (type: `string`):
One record per URL: rendered HTML, page text, selected elements, and optional screenshot URL.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"urls": [
{
"url": "https://www.example.com"
}
],
"proxyConfiguration": {
"useApifyProxy": true
}
};
// Run the Actor and wait for it to finish
const run = await client.actor("patel_dev_automation/browserless-html-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`๐พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// ๐ Want to learn more ๐? Go to โ https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"urls": [{ "url": "https://www.example.com" }],
"proxyConfiguration": { "useApifyProxy": True },
}
# Run the Actor and wait for it to finish
run = client.actor("patel_dev_automation/browserless-html-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("๐พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# ๐ Want to learn more ๐? Go to โ https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"urls": [
{
"url": "https://www.example.com"
}
],
"proxyConfiguration": {
"useApifyProxy": true
}
}' |
apify call patel_dev_automation/browserless-html-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=patel_dev_automation/browserless-html-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Headless Browser HTML Scraper",
"description": "Render any URL in a real headless browser and return the fully-rendered HTML, the page text, or a selected area by CSS selector. Scroll for lazy content, wait for elements, and capture screenshots. A browserless-style HTML API on Apify.",
"version": "0.1",
"x-build-id": "JB3gbz3xb8oq4vGJe"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/patel_dev_automation~browserless-html-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-patel_dev_automation-browserless-html-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/patel_dev_automation~browserless-html-scraper/runs": {
"post": {
"operationId": "runs-sync-patel_dev_automation-browserless-html-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/patel_dev_automation~browserless-html-scraper/run-sync": {
"post": {
"operationId": "run-sync-patel_dev_automation-browserless-html-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"urls"
],
"properties": {
"urls": {
"title": "URLs",
"type": "array",
"description": "One or more URLs to open in a real headless browser and scrape. JavaScript is fully rendered.",
"items": {
"type": "object",
"required": [
"url"
],
"properties": {
"url": {
"type": "string",
"title": "URL of a web page",
"format": "uri"
}
}
}
},
"selector": {
"title": "Selected area (CSS selector)",
"type": "string",
"description": "Optional. A CSS selector for the area you want to extract (like browserless /scrape). Every matching element is returned with its HTML, text, attributes, and position. Leave empty to just get the full page HTML."
},
"scrollToBottom": {
"title": "Scroll to bottom (load lazy content)",
"type": "boolean",
"description": "Scroll the page down with real wheel events to trigger infinite-scroll / lazy-loaded content before scraping.",
"default": false
},
"maxScrolls": {
"title": "Max scroll rounds",
"minimum": 1,
"type": "integer",
"description": "Maximum scroll iterations when 'Scroll to bottom' is on. Stops early when the page stops growing.",
"default": 15
},
"waitForSelector": {
"title": "Wait for selector",
"type": "string",
"description": "Optional. Wait until this CSS selector appears before scraping (up to 30s)."
},
"waitUntil": {
"title": "Wait until (page load event)",
"enum": [
"domcontentloaded",
"load",
"networkidle"
],
"type": "string",
"description": "When to consider navigation finished.",
"default": "domcontentloaded"
},
"waitMs": {
"title": "Extra wait (ms)",
"minimum": 0,
"type": "integer",
"description": "Optional fixed wait after the page loads, in milliseconds (max 60000).",
"default": 0
},
"blockResources": {
"title": "Block resource types",
"type": "array",
"description": "Resource types to block to speed up loading and cut bandwidth. Blocking 'image'/'stylesheet' can break layout-dependent lazy scrolling on some sites.",
"items": {
"type": "string",
"enum": [
"image",
"media",
"font",
"stylesheet"
],
"enumTitles": [
"Images",
"Media (video/audio)",
"Fonts",
"Stylesheets (CSS)"
]
},
"default": [
"media",
"font"
]
},
"returnFullHtml": {
"title": "Return full page HTML",
"type": "boolean",
"description": "Include the full rendered page HTML in the output (like browserless /content).",
"default": true
},
"returnText": {
"title": "Return page text",
"type": "boolean",
"description": "Include the page's visible text (innerText) in the output.",
"default": true
},
"includeScreenshot": {
"title": "Capture full-page screenshot",
"type": "boolean",
"description": "Take a full-page JPEG screenshot, store it, and return its URL in the output.",
"default": false
},
"proxyConfiguration": {
"title": "Proxy configuration",
"type": "object",
"description": "Proxy to route requests through. Apify Proxy (datacenter) by default; switch to Residential for bot-protected sites.",
"default": {
"useApifyProxy": true
}
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```