# Web Page Fetch & Extract (`cyberwarex/web-page-fetch-extract`) Actor

Fetch any live web page as clean LLM-ready markdown/text/HTML, extract elements by CSS selector, or capture a screenshot — JavaScript fully rendered.

- **URL**: https://apify.com/cyberwarex/web-page-fetch-extract.md
- **Developed by:** [Beatrice Chalatian](https://apify.com/cyberwarex) (community)
- **Categories:** Developer tools, AI
- **Stats:** 1 total users, 1 monthly users, 93.8% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Web Page Fetch & Extract

Turn any live web page into **clean, LLM-ready data** — JavaScript fully rendered in a real browser, nav/ads/boilerplate stripped. Paste one or more URLs and get back page content as **Markdown, plain text, or HTML**, **elements matched by a CSS selector**, or a **full-page screenshot**. No scraping stack, no headless-browser plumbing, no proxy rotation to manage.

### What it does

For each URL you pass, pick one operation:

- **Fetch** — the fully-rendered page as clean **Markdown** (great for RAG and LLM prompts), **plain text**, or **cleaned HTML**. Handles SPAs and JS-heavy sites that a plain HTTP GET returns empty or garbled.
- **Extract** — pass a **CSS selector** (`h1`, `.price`, `table tr`) and get back just the matching elements' text and attributes — perfect for prices, links, table rows, and headings.
- **Screenshot** — a pixel-accurate **PNG** (viewport or full-page, base64) rendered in a real browser, for visual verification or as input to a vision model.

Results land in the dataset, one row per URL — export as JSON, CSV, or Excel, or pull via the API.

### Input

| Field | Type | Description |
|---|---|---|
| `urls` | array | One or more absolute `http(s)` URLs. **Required.** |
| `operation` | string | `fetch` (default), `extract`, or `screenshot`. |
| `format` | string | `markdown` (default), `text`, or `html`. Applies to `fetch`. |
| `selector` | string | CSS selector. **Required when** `operation = extract`. |
| `fullpage` | boolean | Full-page vs viewport. Applies to `screenshot`. Default `false`. |
| `width` | integer | Viewport width in px (320–4096). Applies to `screenshot`. Default `1280`. |

#### Example input

```json
{
  "urls": ["https://example.com"],
  "operation": "fetch",
  "format": "markdown"
}
```

### Example output (one dataset row)

```json
{
  "input_url": "https://example.com",
  "operation": "fetch",
  "http_status": 200,
  "format": "markdown",
  "content": "# Example Domain\n\nThis domain is for use in documentation examples..."
}
```

### Use cases

- Feed clean page content into **RAG pipelines** and LLM summarizers
- **Price / listing / table monitoring** with a targeted CSS selector
- **Visual verification** and layout capture via screenshots
- Convert JS-rendered pages to Markdown for **content archives** and drafts

### Pricing

Pay-per-result: you're charged once per URL successfully processed. No monthly minimum, no idle cost.

### Notes

- URLs are validated and internal/private/file targets are blocked — the backend only fetches public web pages.
- A URL that fails returns a row with `http_status` and an `error` field instead of content, so one bad URL never fails the run.

### Who runs this

Built and operated by **[CyberWareX](https://cyberwarex.com)** — a suite of pay-per-call data and AI
APIs for autonomous agents. Every result row carries `provider` / `provider_url` fields so the data's
origin stays traceable wherever it travels. Full service directory and machine-readable docs:
[cyberwarex.com](https://cyberwarex.com) · daily correctness proofs: [cyberwarex.com/trust](https://cyberwarex.com/trust).

# Actor input Schema

## `urls` (type: `array`):

One or more absolute http(s) URLs to process. Each URL becomes one dataset row.

## `operation` (type: `string`):

fetch = clean page content; extract = elements matching a CSS selector; screenshot = full PNG (base64).

## `format` (type: `string`):

Output format when operation = fetch. markdown (LLM-ready), text (plain), or html (cleaned).

## `selector` (type: `string`):

Required when operation = extract. A CSS selector, e.g. h1, .price, table tr.

## `fullpage` (type: `boolean`):

Capture the entire scrollable page instead of just the viewport. Applies to operation = screenshot.

## `width` (type: `integer`):

Browser viewport width for screenshots (320–4096).

## Actor input object example

```json
{
  "urls": [
    "https://example.com"
  ],
  "operation": "fetch",
  "format": "markdown",
  "fullpage": false,
  "width": 1280
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset items with page content per input URL.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("cyberwarex/web-page-fetch-extract").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://example.com"] }

# Run the Actor and wait for it to finish
run = client.actor("cyberwarex/web-page-fetch-extract").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com"
  ]
}' |
apify call cyberwarex/web-page-fetch-extract --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cyberwarex/web-page-fetch-extract"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/hGpbOPr71pHnqPLLY/builds/UAqmWRAr4BTIaGoNi/openapi.json