# Webpage Summarizer Scraper (`automation-lab/url-webpage-summarizer`) Actor

Batch-summarize public webpage URLs into deterministic summaries, key points, and cleaned-text provenance for research and content review.

- **URL**: https://apify.com/automation-lab/url-webpage-summarizer.md
- **Developed by:** [Automation Lab](https://apify.com/automation-lab) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.40 / 1,000 page summarizeds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Webpage Summarizer Scraper

Turn batches of anonymously reachable public links into structured, source-grounded summaries.
This **webpage summarizer** fetches each URL, isolates readable text, and returns a concise deterministic extractive summary, key points, cleaned-text provenance, the final source URL, retrieval status, and timestamp.

It is designed for recurring research triage, editorial review, knowledge ingestion, and compact content-monitoring feeds.
Unlike a generative webpage summarizer AI service, it uses source sentences rather than inventing new prose.
No model API key is required.

### What does Webpage Summarizer Scraper do?

For every unique public HTTP or HTTPS URL, the Actor:

1. validates that the destination is not a local or private-network address;
2. fetches server-delivered HTML with bounded redirects and transient retries;
3. isolates readable main content;
4. normalizes whitespace;
5. ranks source sentences deterministically;
6. builds a concise summary and key-point list;
7. records text length, word count, extraction method, and SHA-256 hash;
8. saves one typed success or error record.

The default dataset is ready for Apify API clients, webhooks, Make, Zapier, n8n, Google Sheets, and downstream databases.

### Who is it for?

#### Researchers and analysts

Skim reading lists before deciding which pages deserve deep review.
Keep the original URL and cleaned-text hash beside every digest.

#### Content and editorial teams

Create a repeatable first-pass review feed from public articles, documentation, and landing pages.
Use retrieval status to separate unavailable sources from reviewed content.

#### Knowledge and RAG engineers

Send normalized source text, summaries, and provenance into an indexing pipeline.
Disable full cleaned text when only a compact triage record is needed.

#### Automation builders

Schedule the same Task with a stable URL list.
Compare `contentSha256` values downstream to detect source-text changes before invoking expensive processing.

### Why use deterministic extractive summaries?

Extractive output is auditable.
Every summary sentence and key point comes from the extracted source text.
Running the same source content with the same settings produces the same summary selection.

This approach offers:

- no external AI account or model key;
- no token bill;
- no prompt variability;
- source-grounded text suitable for review queues;
- predictable batch behavior;
- compact provenance for repeat runs.

It does not rewrite, translate, infer sentiment, or provide a human-quality abstractive interpretation.
Use it when traceability and repeatability matter more than generated prose.

### What data is returned?

| Field | Type | Meaning |
| --- | --- | --- |
| `requestedUrl` | string | Normalized URL supplied to the Actor |
| `finalUrl` | string or null | Final URL after bounded redirects |
| `retrievalStatus` | string | `success` or `error` |
| `httpStatus` | number or null | Successful HTTP response status |
| `title` | string or null | Readability or page-metadata title |
| `summary` | string or null | Ranked source sentences in document order |
| `keyPoints` | string\[] | Highest-scoring source-grounded sentences |
| `cleanedText` | string or null | Normalized readable text when enabled |
| `extractionMethod` | string or null | `readability` or `main-content-fallback` |
| `sourceContentType` | string or null | Source response Content-Type |
| `sourceBytes` | number or null | Downloaded HTML byte size |
| `cleanedCharacterCount` | number or null | Complete normalized-text character count |
| `wordCount` | number or null | Complete normalized-text word count |
| `contentSha256` | string or null | Hash of complete normalized text |
| `retrievedAt` | string | ISO 8601 retrieval-attempt timestamp |
| `error` | string or null | Bounded page-level failure reason |

Failed pages are represented explicitly rather than disappearing from the batch.
Only successfully summarized pages incur the per-page event charge.

### How to summarize webpage URLs

1. Open the Actor in Apify Console.
2. Add one or more URLs under **Public webpage URLs**.
3. Choose the number of summary sentences and key points.
4. Leave **Include cleaned text** enabled for extraction or RAG workflows.
5. Disable it for a smaller monitoring or triage feed.
6. Click **Start**.
7. Open the **Webpage summaries** dataset view.
8. Export JSON, CSV, Excel, XML, or RSS, or consume the dataset API.

The prefilled Wikipedia and Node.js pages are real public examples suitable for a first run.

### Input parameters

| Parameter | Default | Limits | Description |
| --- | ---: | ---: | --- |
| `startUrls` | required | 1–500 | Public HTML webpage URLs |
| `maxItems` | 20 | 1–500 | Maximum unique URLs processed |
| `summarySentences` | 3 | 1–8 | Source sentences in the summary |
| `keyPointCount` | 5 | 1–10 | Key points returned |
| `includeCleanedText` | true | boolean | Include normalized source text |
| `maxTextCharacters` | 100000 | 1000–500000 | Text used and optionally returned |
| `maxConcurrency` | 5 | 1–10 | Parallel page requests |
| `requestTimeoutSecs` | 30 | 5–120 | Per-request timeout |
| `maxRequestRetries` | 2 | 0–5 | Retries for transient failures |
| `maxContentBytes` | 3000000 | 50000–10000000 | Maximum HTML response size |
| `continueOnError` | true | boolean | Continue and save error rows |

Duplicate normalized URLs are processed once.
URL fragments are removed because they do not change the HTTP resource.

### Example input

```json
{
  "startUrls": [
    { "url": "https://en.wikipedia.org/wiki/Web_scraping" },
    { "url": "https://nodejs.org/en/learn/getting-started/introduction-to-nodejs" }
  ],
  "summarySentences": 3,
  "keyPointCount": 5,
  "includeCleanedText": true,
  "maxConcurrency": 2
}
```

### Example output

The exact text and timestamp follow the live source at retrieval time.
A successful record has this shape:

```json
{
  "requestedUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "finalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "retrievalStatus": "success",
  "httpStatus": 200,
  "title": "Web scraping",
  "summary": "Web scraping is the process of extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser.",
  "keyPoints": [
    "Web scraping is the process of extracting data from websites.",
    "Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser."
  ],
  "cleanedText": "Web scraping is the process of extracting data from websites...",
  "extractionMethod": "readability",
  "sourceContentType": "text/html; charset=utf-8",
  "sourceBytes": 485321,
  "cleanedCharacterCount": 38640,
  "wordCount": 6201,
  "contentSha256": "10fdd8e688259fb9464b325d65b08649b4c1f845aff25d4695b9399db5a88a7a",
  "retrievedAt": "2026-09-10T12:00:00.000Z",
  "error": null
}
```

### How summaries and key points are selected

The Actor splits normalized readable content into sentences.
It scores candidate sentences with:

- capped term frequency across the page;
- overlap with meaningful title words;
- a modest early-document position signal;
- stable source order as the tie breaker.

The highest-scoring sentences become key points.
Summary sentences are reordered into document order for readability.
The algorithm does not call an LLM and does not create facts absent from the extracted text.

### Cleaned-text provenance and change detection

`contentSha256` hashes the complete normalized extracted text, even when `cleanedText` is disabled or limited by `maxTextCharacters`.
`cleanedCharacterCount` and `wordCount` also describe the complete extracted text.

A recurring workflow can:

1. run on a schedule;
2. upsert records by `requestedUrl`;
3. compare the latest `contentSha256` with the prior value;
4. route changed pages to human or AI review;
5. ignore unchanged pages.

The Actor emits snapshots, not alerts or historical comparisons itself.
Store history in your own dataset, database, or automation workflow.

### How much does it cost to summarize webpage URLs?

Pricing uses a one-time **$0.005 start event** plus a tiered charge for each successfully summarized page.
At the BRONZE tier, a successful page costs **$0.004**.
Error rows do not incur the page event.

Approximate BRONZE examples:

| Successful pages | Approximate total |
| ---: | ---: |
| 1 | $0.009 |
| 10 | $0.045 |
| 25 | $0.105 |
| 100 | $0.405 |

Apify plan tiers may use lower per-page event prices.
The Console displays the active price for your account before a run.

### Batch research-triage workflow

Use a URL list collected by a crawler, feed reader, spreadsheet, or analyst.
Run this Actor with three to five summary sentences.
Export the resulting rows to a review table.
Sort or filter by title, status, word count, and retrieval time.

Because output is deterministic, reviewers can focus on source changes rather than model variation.

### Compact recurring content-review workflow

Set `includeCleanedText` to `false`.
Keep `summary`, `keyPoints`, `contentSha256`, lengths, final URL, and timestamp.
This reduces dataset size while preserving a useful review record.

Schedule the Task in Apify Console and send results through a webhook.
Compare hashes downstream before triggering a more expensive enrichment step.

### RAG and knowledge-ingestion workflow

Keep `includeCleanedText` enabled.
Use `cleanedText` as the source document and retain:

- `requestedUrl` and `finalUrl` as citations;
- `retrievedAt` for freshness;
- `contentSha256` for deduplication;
- `summary` and `keyPoints` as triage metadata;
- `extractionMethod` for quality diagnostics.

Chunking, embeddings, vector storage, and answer generation remain downstream responsibilities.

### API access with cURL

Replace `APIFY_TOKEN` with your token:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~url-webpage-summarizer/run-sync-get-dataset-items?token=APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],
    "summarySentences": 3,
    "keyPointCount": 5
  }'
```

For larger batches, start an asynchronous run and retrieve the default dataset after completion.

### JavaScript API example

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/url-webpage-summarizer').call({
    startUrls: [
        { url: 'https://nodejs.org/en/learn/getting-started/introduction-to-nodejs' },
    ],
    summarySentences: 4,
    keyPointCount: 6,
    includeCleanedText: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### Python API example

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/url-webpage-summarizer').call(run_input={
    'startUrls': [
        {'url': 'https://docs.apify.com/platform/actors'},
        {'url': 'https://crawlee.dev/js/docs/introduction'},
    ],
    'includeCleanedText': False,
    'summarySentences': 5,
})

items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use with Apify MCP

Add the Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/url-webpage-summarizer"
```

#### Claude Desktop setup

Add this server to Claude Desktop's MCP configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/url-webpage-summarizer"
    }
  }
}
```

#### Cursor setup

Add the same `mcpServers.apify.url` value to your Cursor MCP configuration.

#### VS Code setup

Add the same HTTP endpoint to your VS Code MCP server configuration, then select `automation-lab/url-webpage-summarizer` as the tool.

Example prompts:

- “Summarize these three public research links and return the key points with source URLs.”
- “Create compact review records for these documentation pages without full cleaned text.”
- “Extract clean text and provenance from this public article for my knowledge base.”

### Reliability and retry behavior

The Actor retries network failures, HTTP 408/425/429, and temporary 5xx responses with bounded exponential backoff and jitter.
It does not blindly retry stable client errors, private addresses, unsupported content types, oversized pages, or recognized challenge shells.

Redirects are limited to five.
Every redirect destination is checked again against local and private networks.
A batch can continue after page failures, or fail on the first page error with `continueOnError: false`.

If every URL fails, the run fails after saving diagnostic error records.
This prevents an all-empty result from looking successful.

### Limits and unsupported pages

This Actor supports anonymously reachable server-delivered HTML.
It does not promise:

- JavaScript-only content requiring a browser;
- login-gated, paywalled, or private pages;
- CAPTCHA or anti-bot bypass;
- PDFs, office files, images, audio, or video;
- sentiment, translation, fact checking, or abstractive rewriting;
- semantic understanding equal to an LLM or human editor;
- alert delivery or built-in snapshot history.

Navigation-heavy pages can produce less focused fallback text than article-like pages.
Always inspect source links before making consequential decisions.

### Troubleshooting

#### Why did I receive an error row?

Read the `error` field.
Common causes are HTTP errors, non-HTML content, authentication challenges, response-size limits, private-network destinations, or too little readable text.
Confirm that the URL opens anonymously and returns meaningful HTML.

#### Why is cleanedText null on a successful row?

`includeCleanedText` was disabled.
The summary, key points, hash, text length, word count, extraction method, and source metadata remain available.

#### Why is the summary not rewritten in simpler language?

This is an extractive summarizer.
It deliberately selects source sentences for determinism and auditability.
Use a downstream generative model when paraphrasing is required.

#### Why did a JavaScript application return little text?

The direct HTML response may contain only an application shell.
This Actor does not launch a browser.
Use a browser-rendering extractor for that source, then summarize the resulting text downstream.

### Responsible and legal use

Process only content you are permitted to access and use.
Respect website terms, robots guidance, copyright, privacy, and applicable laws.
Do not use the Actor to access private systems, bypass authentication, or collect sensitive personal information.

A summary is not a substitute for the source.
Retain attribution, verify important claims, and follow the final source URL.

### FAQ

#### Does it use AI?

No external generative model is called.
The ranking algorithm is deterministic and extractive.

#### Can I summarize many links in one run?

Yes.
Provide up to 500 input URLs and set `maxItems` to your desired bound.

#### Are failed pages charged?

No per-page event is charged for an error record.
The one-time run start event still applies.

#### Can it monitor changes?

It supplies timestamps and a cleaned-text hash designed for downstream comparison.
Scheduling, history, and alerts are configured in Apify or your automation stack.

#### Can it summarize PDFs?

No.
The source response must be HTML or XHTML.

#### Does it follow redirects?

Yes, up to five public HTTP(S) destinations.
Each destination is validated against private-network access.

### Related automation-lab Actors

- [Public Webpage HTML Downloader](https://apify.com/automation-lab/public-webpage-html-downloader) saves raw or browser-rendered HTML when the original markup is the product.
- [Multi-Site Article Content Extractor](https://apify.com/automation-lab/multi-site-article-content-extractor) exports richer article metadata, clean text, HTML, and links without summary selection.
- [Website HTML & Text Change Monitor](https://apify.com/automation-lab/website-html-text-change-monitor) maintains versioned snapshots and machine-readable changes when built-in comparison is required.

Choose this Actor when the primary output is a compact, deterministic summary record with key points and cleaned-text provenance.

# Actor input Schema

## `startUrls` (type: `array`):

HTTP or HTTPS pages to summarize. Duplicate URLs are processed once; local, private-network, credential-bearing, and non-HTML URLs are rejected.

## `maxItems` (type: `integer`):

Maximum number of unique input URLs to process.

## `summarySentences` (type: `integer`):

Number of high-scoring source sentences in each deterministic extractive summary.

## `keyPointCount` (type: `integer`):

Maximum number of source-grounded key points returned per successful page.

## `includeCleanedText` (type: `boolean`):

Include normalized source text in each successful record. Disable for smaller datasets while retaining its length, word count, extraction method, and SHA-256 provenance.

## `maxTextCharacters` (type: `integer`):

Maximum source-text characters used for summarization and included in cleanedText. Provenance counts and hash still describe the complete extracted text.

## `maxConcurrency` (type: `integer`):

Maximum pages fetched in parallel.

## `requestTimeoutSecs` (type: `integer`):

Maximum retrieval time for each HTTP request.

## `maxRequestRetries` (type: `integer`):

Retries for network failures, rate limits, and temporary server errors. Deterministic client errors are not retried.

## `maxContentBytes` (type: `integer`):

Reject an HTML response larger than this per-page safety limit.

## `continueOnError` (type: `boolean`):

Save a typed error record and continue with other URLs. When disabled, the run fails after the first page error.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    },
    {
      "url": "https://nodejs.org/en/learn/getting-started/introduction-to-nodejs"
    }
  ],
  "maxItems": 20,
  "summarySentences": 3,
  "keyPointCount": 5,
  "includeCleanedText": true,
  "maxTextCharacters": 100000,
  "maxConcurrency": 5,
  "requestTimeoutSecs": 30,
  "maxRequestRetries": 2,
  "maxContentBytes": 3000000,
  "continueOnError": true
}
```

# Actor output Schema

## `dataset` (type: `string`):

Structured summaries, key points, cleaned-text provenance, source URLs, statuses, and timestamps.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://en.wikipedia.org/wiki/Web_scraping"
        },
        {
            "url": "https://nodejs.org/en/learn/getting-started/introduction-to-nodejs"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/url-webpage-summarizer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        { "url": "https://en.wikipedia.org/wiki/Web_scraping" },
        { "url": "https://nodejs.org/en/learn/getting-started/introduction-to-nodejs" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/url-webpage-summarizer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    },
    {
      "url": "https://nodejs.org/en/learn/getting-started/introduction-to-nodejs"
    }
  ]
}' |
apify call automation-lab/url-webpage-summarizer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/url-webpage-summarizer"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vV6hFBrcDkMbmBUbq/builds/92LrE9GYhh1SAoGgf/openapi.json