Webpage Summarizer Scraper

Turn batches of anonymously reachable public links into structured, source-grounded summaries. This webpage summarizer fetches each URL, isolates readable text, and returns a concise deterministic extractive summary, key points, cleaned-text provenance, the final source URL, retrieval status, and timestamp.

It is designed for recurring research triage, editorial review, knowledge ingestion, and compact content-monitoring feeds. Unlike a generative webpage summarizer AI service, it uses source sentences rather than inventing new prose. No model API key is required.

What does Webpage Summarizer Scraper do?

For every unique public HTTP or HTTPS URL, the Actor:

1. validates that the destination is not a local or private-network address;
2. fetches server-delivered HTML with bounded redirects and transient retries;
3. isolates readable main content;
4. normalizes whitespace;
5. ranks source sentences deterministically;
6. builds a concise summary and key-point list;
7. records text length, word count, extraction method, and SHA-256 hash;
8. saves one typed success or error record.

The default dataset is ready for Apify API clients, webhooks, Make, Zapier, n8n, Google Sheets, and downstream databases.

Who is it for?

Researchers and analysts

Skim reading lists before deciding which pages deserve deep review. Keep the original URL and cleaned-text hash beside every digest.

Content and editorial teams

Create a repeatable first-pass review feed from public articles, documentation, and landing pages. Use retrieval status to separate unavailable sources from reviewed content.

Knowledge and RAG engineers

Send normalized source text, summaries, and provenance into an indexing pipeline. Disable full cleaned text when only a compact triage record is needed.

Automation builders

Schedule the same Task with a stable URL list. Compare contentSha256 values downstream to detect source-text changes before invoking expensive processing.

Why use deterministic extractive summaries?

Extractive output is auditable. Every summary sentence and key point comes from the extracted source text. Running the same source content with the same settings produces the same summary selection.

This approach offers:

• no external AI account or model key;
• no token bill;
• no prompt variability;
• source-grounded text suitable for review queues;
• predictable batch behavior;
• compact provenance for repeat runs.

It does not rewrite, translate, infer sentiment, or provide a human-quality abstractive interpretation. Use it when traceability and repeatability matter more than generated prose.

What data is returned?

Field	Type	Meaning
requestedUrl	string	Normalized URL supplied to the Actor
finalUrl	string or null	Final URL after bounded redirects
retrievalStatus	string	success or error
httpStatus	number or null	Successful HTTP response status
title	string or null	Readability or page-metadata title
summary	string or null	Ranked source sentences in document order
keyPoints	string[]	Highest-scoring source-grounded sentences
cleanedText	string or null	Normalized readable text when enabled
extractionMethod	string or null	readability or main-content-fallback
sourceContentType	string or null	Source response Content-Type
sourceBytes	number or null	Downloaded HTML byte size
cleanedCharacterCount	number or null	Complete normalized-text character count
wordCount	number or null	Complete normalized-text word count
contentSha256	string or null	Hash of complete normalized text
retrievedAt	string	ISO 8601 retrieval-attempt timestamp
error	string or null	Bounded page-level failure reason

Failed pages are represented explicitly rather than disappearing from the batch. Only successfully summarized pages incur the per-page event charge.

How to summarize webpage URLs

1. Open the Actor in Apify Console.
2. Add one or more URLs under Public webpage URLs.
3. Choose the number of summary sentences and key points.
4. Leave Include cleaned text enabled for extraction or RAG workflows.
5. Disable it for a smaller monitoring or triage feed.
6. Click Start.
7. Open the Webpage summaries dataset view.
8. Export JSON, CSV, Excel, XML, or RSS, or consume the dataset API.

The prefilled Wikipedia and Node.js pages are real public examples suitable for a first run.

Input parameters

Parameter	Default	Limits	Description
startUrls	required	1–500	Public HTML webpage URLs
maxItems	20	1–500	Maximum unique URLs processed
summarySentences	3	1–8	Source sentences in the summary
keyPointCount	5	1–10	Key points returned
includeCleanedText	true	boolean	Include normalized source text
maxTextCharacters	100000	1000–500000	Text used and optionally returned
maxConcurrency	5	1–10	Parallel page requests
requestTimeoutSecs	30	5–120	Per-request timeout
maxRequestRetries	2	0–5	Retries for transient failures
maxContentBytes	3000000	50000–10000000	Maximum HTML response size
continueOnError	true	boolean	Continue and save error rows

Duplicate normalized URLs are processed once. URL fragments are removed because they do not change the HTTP resource.

Example input

{
  "startUrls": [
    { "url": "https://en.wikipedia.org/wiki/Web_scraping" },
    { "url": "https://nodejs.org/en/learn/getting-started/introduction-to-nodejs" }
  ],
  "summarySentences": 3,
  "keyPointCount": 5,
  "includeCleanedText": true,
  "maxConcurrency": 2
}

Example output

The exact text and timestamp follow the live source at retrieval time. A successful record has this shape:

{
  "requestedUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "finalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "retrievalStatus": "success",
  "httpStatus": 200,
  "title": "Web scraping",
  "summary": "Web scraping is the process of extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser.",
  "keyPoints": [
    "Web scraping is the process of extracting data from websites.",
    "Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser."
  ],
  "cleanedText": "Web scraping is the process of extracting data from websites...",
  "extractionMethod": "readability",
  "sourceContentType": "text/html; charset=utf-8",
  "sourceBytes": 485321,
  "cleanedCharacterCount": 38640,
  "wordCount": 6201,
  "contentSha256": "10fdd8e688259fb9464b325d65b08649b4c1f845aff25d4695b9399db5a88a7a",
  "retrievedAt": "2026-09-10T12:00:00.000Z",
  "error": null
}

How summaries and key points are selected

The Actor splits normalized readable content into sentences. It scores candidate sentences with:

• capped term frequency across the page;
• overlap with meaningful title words;
• a modest early-document position signal;
• stable source order as the tie breaker.

The highest-scoring sentences become key points. Summary sentences are reordered into document order for readability. The algorithm does not call an LLM and does not create facts absent from the extracted text.

Cleaned-text provenance and change detection

contentSha256 hashes the complete normalized extracted text, even when cleanedText is disabled or limited by maxTextCharacters. cleanedCharacterCount and wordCount also describe the complete extracted text.

A recurring workflow can:

1. run on a schedule;
2. upsert records by requestedUrl;
3. compare the latest contentSha256 with the prior value;
4. route changed pages to human or AI review;
5. ignore unchanged pages.

The Actor emits snapshots, not alerts or historical comparisons itself. Store history in your own dataset, database, or automation workflow.

How much does it cost to summarize webpage URLs?

Pricing uses a one-time $0.005 start event plus a tiered charge for each successfully summarized page. At the BRONZE tier, a successful page costs $0.004. Error rows do not incur the page event.

Approximate BRONZE examples:

Successful pages	Approximate total
1	$0.009
10	$0.045
25	$0.105
100	$0.405

Apify plan tiers may use lower per-page event prices. The Console displays the active price for your account before a run.

Batch research-triage workflow

Use a URL list collected by a crawler, feed reader, spreadsheet, or analyst. Run this Actor with three to five summary sentences. Export the resulting rows to a review table. Sort or filter by title, status, word count, and retrieval time.

Because output is deterministic, reviewers can focus on source changes rather than model variation.

Compact recurring content-review workflow

Set includeCleanedText to false. Keep summary, keyPoints, contentSha256, lengths, final URL, and timestamp. This reduces dataset size while preserving a useful review record.

Schedule the Task in Apify Console and send results through a webhook. Compare hashes downstream before triggering a more expensive enrichment step.

RAG and knowledge-ingestion workflow

Keep includeCleanedText enabled. Use cleanedText as the source document and retain:

• requestedUrl and finalUrl as citations;
• retrievedAt for freshness;
• contentSha256 for deduplication;
• summary and keyPoints as triage metadata;
• extractionMethod for quality diagnostics.

Chunking, embeddings, vector storage, and answer generation remain downstream responsibilities.

API access with cURL

Replace APIFY_TOKEN with your token:

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~url-webpage-summarizer/run-sync-get-dataset-items?token=APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],
    "summarySentences": 3,
    "keyPointCount": 5
  }'

For larger batches, start an asynchronous run and retrieve the default dataset after completion.

JavaScript API example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/url-webpage-summarizer').call({
    startUrls: [
        { url: 'https://nodejs.org/en/learn/getting-started/introduction-to-nodejs' },
    ],
    summarySentences: 4,
    keyPointCount: 6,
    includeCleanedText: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python API example

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/url-webpage-summarizer').call(run_input={
    'startUrls': [
        {'url': 'https://docs.apify.com/platform/actors'},
        {'url': 'https://crawlee.dev/js/docs/introduction'},
    ],
    'includeCleanedText': False,
    'summarySentences': 5,
})

items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

Use with Apify MCP

Add the Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/url-webpage-summarizer"

Claude Desktop setup

Add this server to Claude Desktop's MCP configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/url-webpage-summarizer"
    }
  }
}

Cursor setup

Add the same mcpServers.apify.url value to your Cursor MCP configuration.

VS Code setup

Add the same HTTP endpoint to your VS Code MCP server configuration, then select automation-lab/url-webpage-summarizer as the tool.

Example prompts:

• “Summarize these three public research links and return the key points with source URLs.”
• “Create compact review records for these documentation pages without full cleaned text.”
• “Extract clean text and provenance from this public article for my knowledge base.”

Reliability and retry behavior

The Actor retries network failures, HTTP 408/425/429, and temporary 5xx responses with bounded exponential backoff and jitter. It does not blindly retry stable client errors, private addresses, unsupported content types, oversized pages, or recognized challenge shells.

Redirects are limited to five. Every redirect destination is checked again against local and private networks. A batch can continue after page failures, or fail on the first page error with continueOnError: false.

If every URL fails, the run fails after saving diagnostic error records. This prevents an all-empty result from looking successful.

Limits and unsupported pages

This Actor supports anonymously reachable server-delivered HTML. It does not promise:

• JavaScript-only content requiring a browser;
• login-gated, paywalled, or private pages;
• CAPTCHA or anti-bot bypass;
• PDFs, office files, images, audio, or video;
• sentiment, translation, fact checking, or abstractive rewriting;
• semantic understanding equal to an LLM or human editor;
• alert delivery or built-in snapshot history.

Navigation-heavy pages can produce less focused fallback text than article-like pages. Always inspect source links before making consequential decisions.

Troubleshooting

Why did I receive an error row?

Read the error field. Common causes are HTTP errors, non-HTML content, authentication challenges, response-size limits, private-network destinations, or too little readable text. Confirm that the URL opens anonymously and returns meaningful HTML.

Why is cleanedText null on a successful row?

includeCleanedText was disabled. The summary, key points, hash, text length, word count, extraction method, and source metadata remain available.

Why is the summary not rewritten in simpler language?

This is an extractive summarizer. It deliberately selects source sentences for determinism and auditability. Use a downstream generative model when paraphrasing is required.

Why did a JavaScript application return little text?

The direct HTML response may contain only an application shell. This Actor does not launch a browser. Use a browser-rendering extractor for that source, then summarize the resulting text downstream.

Responsible and legal use

Process only content you are permitted to access and use. Respect website terms, robots guidance, copyright, privacy, and applicable laws. Do not use the Actor to access private systems, bypass authentication, or collect sensitive personal information.

A summary is not a substitute for the source. Retain attribution, verify important claims, and follow the final source URL.

FAQ

Does it use AI?

No external generative model is called. The ranking algorithm is deterministic and extractive.

Can I summarize many links in one run?

Yes. Provide up to 500 input URLs and set maxItems to your desired bound.

Are failed pages charged?

No per-page event is charged for an error record. The one-time run start event still applies.

Can it monitor changes?

It supplies timestamps and a cleaned-text hash designed for downstream comparison. Scheduling, history, and alerts are configured in Apify or your automation stack.

Can it summarize PDFs?

No. The source response must be HTML or XHTML.

Does it follow redirects?

Yes, up to five public HTTP(S) destinations. Each destination is validated against private-network access.

Related automation-lab Actors

• Public Webpage HTML Downloader saves raw or browser-rendered HTML when the original markup is the product.
• Multi-Site Article Content Extractor exports richer article metadata, clean text, HTML, and links without summary selection.
• Website HTML & Text Change Monitor maintains versioned snapshots and machine-readable changes when built-in comparison is required.

Choose this Actor when the primary output is a compact, deterministic summary record with key points and cleaned-text provenance.