Web Page Text Scraper – Clean Text & Links

Scrape any web page as clean, readable text. It returns what is actually on the page — the visible prose, with the scripts, navigation, headers, footers, booking widgets and cookie banners taken out — and, when you ask for it, every link to the same site with the words that were clickable. Built for feeding LLMs and RAG pipelines, content and SEO auditing, site research and text mining.

It runs the page's JavaScript when the page needs it, so a site that builds itself in the browser reads the same as one that does not.

Two things make it different from a general crawler: it reads the pages you name and nothing else, and a page that could not be read never comes back looking like a page with nothing on it — it arrives as an error row with no text field at all, rather than as a page whose text is empty. See Failures — that distinction is the point of this Actor.

Features

• Clean text extraction — scripts, styles, navigation, headers, footers, cookie banners, booking and newsletter forms, carousel controls and repeated lines all removed.
• Full text, never summarised — not shortened, not rewritten; a name in the middle of a paragraph survives.
• Sentence boundaries preserved — a full stop at every block boundary, so Chef: Ana. Owner: Bo. never becomes the one person "Ana Owner".
• Same-site link discovery with anchor text — every internal link and the words that were clickable, in page order. External links excluded on purpose.
• JavaScript rendered only when needed — auto opens a browser only when the served HTML turns out not to hold the page; always and never force it.
• Redirects followed and reported — loadedUrl is where the page was really read from, including a redirect the page's own JavaScript performed.
• Raw HTML on request — the rendered DOM where a browser was used, the served HTML otherwise, for running your own extractor.
• Unreadable is never confused with empty — an error row carries no text key at all, so "could not read" and "nothing on the page" stay distinguishable.
• 404 and 410 are answers, not failures — returned as ordinary page rows with their status code.
• Bare hosts accepted — example.com works as well as a full URL.
• Deduplicated — the same page is never read twice in one run, and www.example.com and example.com are the same page.
• Runs stay green. Page problems are rows; only a bug fails a run.

Input

{
  "urls": ["https://www.whiteoakny.com/"],  // plain list; accepts a bare host too
  "mode": "discover",        // "discover" = text + same-site links, "text" = text alone
  "renderJs": "auto",        // "auto" | "always" | "never"
  "maxTextLength": 50000,    // safety net, not a trim
  "includeHtml": false,      // also return the page's raw markup as `html`
  "proxy": { "useApifyProxy": true }
}

Two modes: discover, then read

Discover — the page, plus where else to go

Give it an entry page, usually a site's home page.

{
  "urls": ["https://www.whiteoakny.com/"],
  "mode": "discover"
}

[
  {
    "type": "page",
    "url": "https://www.whiteoakny.com/",
    "loadedUrl": "https://www.whiteoakny.com/",
    "statusCode": 200,
    "title": "White Oak Tavern: Rustic & Old-School American Tavern in NYC",
    "text": "Named after the tall white oak trees of Kentucky… Our dishes are a representation of forgotten cooking skills and time-honored techniques… Executive Chef Jorge Madriz has created a menu that fuses the traditional with the contemporary…",
    "textLength": 1145,
    "textTruncated": false,
    "rendered": false,
    "links": [
      { "url": "https://www.whiteoakny.com/menus/", "anchor": "Menus" },
      { "url": "https://www.whiteoakny.com/who-we-are/", "anchor": "Who We Are" },
      { "url": "https://www.whiteoakny.com/the-oak-ledger-blog/", "anchor": "Blog: The Oak Ledger" }
    ],
    "retrievedAt": "2026-08-24T09:12:00.000Z"
  }
]

The anchor text is the useful part. Addresses are not reliable — /about is a 404 on four of the five sites this was first built against, while their "about" pages live at /who-we-are/, /our-story and /team. The words next to the link are what tell you which page is worth reading.

Links to other sites are left out on purpose. A page links to its parent company, its booking platform and its delivery partner, and none of those pages are about the site you asked about.

Text — read the pages you picked

{
  "urls": [
    "https://www.whiteoakny.com/who-we-are/",
    "https://www.whiteoakny.com/team"
  ],
  "mode": "text"
}

[
  {
    "type": "page",
    "url": "https://www.whiteoakny.com/who-we-are/",
    "loadedUrl": "https://www.whiteoakny.com/who-we-are/",
    "statusCode": 200,
    "title": "Who We Are",
    "text": "Our Story. Chef de Cuisine…",
    "textLength": 2310,
    "textTruncated": false,
    "rendered": false,
    "retrievedAt": "2026-08-24T09:12:04.000Z"
  },
  {
    "type": "page",
    "url": "https://www.whiteoakny.com/team",
    "loadedUrl": "https://www.whiteoakny.com/team/",
    "statusCode": 404,
    "retrievedAt": "2026-08-24T09:12:04.000Z"
  }
]

The rows have the same shape in both modes. Only links is added by discover.

Output

What text is

The page's visible prose, in full. It is not summarised, not shortened and not rewritten — a name is as often in the middle of a paragraph as in a heading, so nothing is left out on the grounds of looking unimportant. A typical page runs 1,200 to 7,000 characters, and the 50,000-character limit is a safety net rather than a policy: a page that exceeds it is cut at a word and marked textTruncated: true, so a shortened answer can never pass for a complete one.

Taken out, because none of it is the page saying anything:

• <script> and <style>, so a JavaScript variable can never be read as a name
• navigation, headers and footers — including the ones built without the semantic tags, which most page builders use
• cookie and consent banners
• booking and newsletter forms, whose time and date pickers otherwise arrive as a wall of "11:00 PM 10:30 PM 10:00 PM…"
• carousel controls, accessibility-widget instructions and "Skip to main content"
• a line that has already been said once

A full stop is placed at every block boundary.

<h3>Chef: Ana</h3><p>Owner: Bo</p>

If you would rather run your own extractor, turn on Also return the raw HTML and every row carries the page's markup as html beside the text — the rendered DOM where a browser was used, the served HTML otherwise.

How to use

From Apify Console

1. Open the Actor and click Try for free / Start.
2. Paste the addresses into Pages to read (plain list) — a bare host such as example.com works.
3. Leave Mode on Discover the first time: you get the page's text plus every same-site link with its anchor text, which tells you which pages are worth reading next.
4. Feed those links back in with Mode set to Text to read the pages you picked.
5. Leave Run the page's JavaScript on Auto — a browser is opened only when the served HTML turns out not to hold the page.
6. Click Start, then open the Dataset tab and export as JSON, CSV or Excel.

Skip rows that have no text key — those are pages that could not be read, not pages that were empty.

From the API

curl -s "https://api.apify.com/v2/acts/simple.actors~page-text-reader/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"urls": ["https://www.whiteoakny.com/"], "mode": "discover"}'

The two-step pattern — discover, then read what looks useful — with the JavaScript client:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const actor = client.actor('simple.actors/page-text-reader');

// 1. Discover: what pages does this site have, and what are they called?
const found = await actor.call({ urls: ['https://www.whiteoakny.com/'], mode: 'discover' });
const { items: [home] } = await client.dataset(found.defaultDatasetId).listItems();

// 2. Read the ones whose anchor text looks relevant.
const wanted = home.links
    .filter((l) => /about|story|team|who we are/i.test(l.anchor))
    .map((l) => l.url);
const read = await actor.call({ urls: wanted, mode: 'text' });
const { items } = await client.dataset(read.defaultDatasetId).listItems();

for (const row of items) {
    if (!('text' in row)) continue;   // unread, not empty — key on presence
    console.log(row.title, row.textLength);
}

Use cases

• Feed an LLM or RAG pipeline — clean prose with the boilerplate already removed, so you are not paying tokens for cookie banners and nav menus.
• Content and SEO auditing — pull the real on-page copy plus the internal link graph with anchor text for a set of pages.
• Site research and data collection — discover a small site's pages by their anchor text, then read only the ones that matter.
• Text mining and NLP corpora — build a clean text dataset from a list of URLs, with sentence boundaries intact.
• Monitoring page copy for changes — read the same pages on a schedule and diff text, without markup churn creating false positives.
• Checking whether a site needs a browser at all — rendered on every row tells you which pages built themselves in the browser.

Usage notes

What it costs

$0.002 per page read — $2.00 per 1,000 pages. One price, nothing else: no per-run fee, no charge for links, no charge for size.

A page is charged when it is delivered. That includes a page that answered 404, because "this page is not there" is a real answer about the site and costs the same to find out.

A page that could not be read is never charged. Refusals, challenges, timeouts, unusable addresses — those rows carry an error and cost nothing, so a night where a site was down bills only for the sites that answered.

Failures

An empty text on a 200 means one thing only: the page loaded and genuinely had nothing readable on it. Some pages really are like that — a home page that is one photograph and a menu of links has no prose, and saying so is a real answer.

So a page that could not be read is never reported that way:

• it comes back as a row with type: "error", an error code and an errorDescription, carrying no text field at all.

The rows are the failure channel, not the run status. A run succeeds even when some — or all — of its pages could not be read, so do not read a green run as "every page was fine". Its status message counts them (

8 of 10 page(s) read; 2 could not be read

What this gives you is the distinction that matters: text: "" means the page loaded and had nothing on it, and a missing text means it was never read. A night of refused requests can never arrive looking like a night of empty pages. If your pipeline acts on emptiness — writing a site off, putting it on a cooldown — key that decision on whether the row has a text field, and skip the rows that do not.

A 404 or 410 is not a failure. The page is not there, which is an answer, and it arrives as an ordinary page row carrying its status code and no text.

The browser

Most pages are read over plain HTTP, which is fast and cheap. A browser is opened only when the served HTML turns out not to hold the page — measured across twenty-six real sites, that was two of them: one served 51KB of markup with no readable text, the other 145 bytes of window.location.href="/lander". Rendering the other twenty-four added accessibility notices and carousel labels, and not one sentence of the sites' own writing.

rendered on each row says which route answered. You can force the choice with Run the page's JavaScript: always renders everything (slower and dearer), never refuses the browser entirely — and a page that needed one then comes back as a needs_browser error row rather than as a page with no text on it.

Notes

• Every request goes through Apify Proxy, and each retry lands on a different address, which is what clears most one-off refusals.
• The same page is never read twice in one run, and a link to www.example.com and one to example.com are the same page.
• Pages are read several at a time, so a batch costs about what its slowest page costs rather than the sum of all of them.
• Unofficial. It reads public pages as a browser would; it does not log in, solve challenges or ignore a site's refusal.

FAQ

Is scraping web page text legal? This Actor reads public pages as a browser would — it does not log in, solve challenges, or ignore a site's refusal. What you may do with the text depends on the site: check the platform's ToS and the site's own terms and robots policy before collecting or republishing content.

Do I need a browser or Puppeteer setup? No. renderJs: "auto" is the default and reads most pages over plain HTTP, opening a browser only when the served HTML turns out not to hold the page. Across twenty-six real sites, that was two of them.

Does it crawl a whole site or follow links automatically? No — it reads the pages you name and nothing else. Discover mode hands you the same-site links with their anchor text so you choose what to read next; feed those URLs back in a second run with mode: "text". That is the intended pattern, and it keeps you from paying for pages you did not want.

How do I tell an empty page from a page that failed? By whether the row has a text key. text: "" means the page loaded and genuinely had nothing readable on it. A row with no text field at all was never read, and carries type: "error" with an error code instead. If your pipeline acts on emptiness, key on the presence of text, not its value.

Why did my run succeed when pages failed? Page problems are rows, not run failures — a run succeeds even if every page in it failed. The status message counts them (

8 of 10 page(s) read; 2 could not be read

Is a 404 an error? No. The page not being there is a real answer about the site, so it arrives as an ordinary page row with its status code and no text.

Can I get the raw HTML instead of cleaned text? Yes — turn on includeHtml and every row carries html beside the text: the rendered DOM where a browser was used, the served HTML otherwise.

Will long pages be cut off? Only past 50,000 characters, which is a safety net rather than a policy — a typical page runs 1,200 to 7,000. A page that does exceed it is cut at a word boundary and marked textTruncated: true, so a shortened answer can never pass for a complete one.