Text Scraper (Free)

📝 Text Scraper (Free) – Extract Clean Website Text, Page Titles & Plain Content from Any URL

The Text Scraper (Free) turns any list of web page URLs into clean, readable plain text. Give it a set of links and it returns one record per page containing the page title, the canonical URL and the full text content stripped of HTML markup, scripts, styles and tag soup — the actual words a human reads, in a form a machine can process.

This text scraper exists for one job and does it without ceremony. There are no selectors to write, no templates to maintain per site, and no output schema that shifts depending on the page you point it at. Every record has the same three keys. That makes it an easy first stage in a longer pipeline: feed the text into an embedding model, a summariser, a keyword extractor, a translation workflow, a content audit spreadsheet or a search index, and you skip the usual afternoon of writing HTML cleanup code.

Because the actor runs on direct HTTP requests and an HTML parser rather than a headless browser, runs are fast and light. For the static and server-rendered pages that make up most of the web — marketing sites, documentation, blogs, news, policy pages, knowledge bases — that is exactly the right tool for the job.

---

📊 What Data Can You Extract with This Text Scraper?

Every URL you submit produces a single flat record. The output schema is deliberately small, so nothing surprises your downstream loader. Here is what each field gives you, grouped by the role it plays.

Category	Field(s)	What it gives you
🔗 Page identity	url	The canonical URL of the scraped page, so every block of text stays traceable to its exact source.
🏷️ Page title	title	The document title of the page — the headline shown in browser tabs and search results, and usually the single best one-line label for the content.
📄 Body text	textContent	The complete visible text of the page as plain text, with HTML tags, scripts and styling removed. This is the substance: headings, paragraphs, list items, navigation labels and button text, in document order.
🧹 Formatting control	textContent (shaped by the removeEmpty input)	The removeEmpty input decides whether blank lines survive into textContent. Left on, you get a compact block with no runs of empty lines; switched off, the original line spacing is preserved.

The field that does the real work here is textContent. It is not a truncated meta description or a snippet — it is the page's readable text in full, newline-separated in the order the content appears in the document. That ordering matters more than people expect: because headings, sub-headings and body copy arrive in their natural sequence, you can reconstruct a page's rough structure from the text alone, chunk it sensibly for a vector database, or diff two captures of the same page and see precisely which paragraph changed.

---

🌟 Key Features of the Text Scraper

Feature	Description
🎯 URL-list driven	Paste an array of URLs and run. No CSS selectors, no XPath, no per-site configuration to build or maintain.
🧼 Clean text extraction	HTML tags, inline scripts and stylesheet blocks are stripped out, leaving the words a visitor actually sees rather than raw markup.
🧹 Empty-line control	The removeEmpty boolean strips blank lines from the output when enabled (the default), producing compact text that tokenises efficiently.
⚡ HTTP-first architecture	Built on httpx and BeautifulSoup instead of a browser, so static and server-rendered pages are fetched and parsed quickly with a small compute footprint.
🧱 Three-key flat schema	title, url and textContent — the same three keys on every record, so CSV exports, database inserts and JSON parsers never break on a missing column.
🌐 Multi-domain batches	A single run can mix any number of unrelated domains; each URL is handled independently and stamped with its own url.
🧯 Null-safe fields	Every field is typed string-or-null, so a page that yields a title but little text still produces a usable record rather than failing the run.
💸 Free to run	The actor carries no per-result charge — you pay only the platform compute your run consumes.
📦 Dataset-native output	Results land in an Apify dataset and export to JSON, JSONL, CSV, Excel, XML or HTML, or stream straight out through the API.

---

🚀 Why Choose This Text Scraper?

Zero configuration, any website. Most text extraction projects begin with an hour of inspecting the DOM and end with a fragile selector that breaks at the next redesign. This text scraper takes one input — a list of URLs — and applies the same generic extraction to all of them. Adding a new site to your pipeline costs one line in an array.

Output that is genuinely ready for text analysis. The textContent field is plain text, not a full-page HTML blob you still have to clean. You can pass it straight into a tokeniser, an embedding call, a language detector, a readability score or a regex search. The removeEmpty option removes the most common annoyance in scraped text — long runs of blank lines — before the data ever reaches you.

Fast and inexpensive by design. Skipping the browser is the single biggest performance lever in web scraping. By fetching HTML over plain HTTP and parsing it in-process, this text scraper handles large URL batches in a fraction of the time a browser-based crawler would need, and it consumes far less memory doing it.

Predictable records that automate cleanly. Three fields, always present, always the same types. That reliability is what makes the actor a good building block: schedule it, hang a webhook off it, pipe it into a warehouse table, and nothing downstream needs defensive code for a schema that might change shape between runs.

---

📥 Input

The Text Scraper (Free) takes a list of URLs and one formatting switch. Nothing else is required.

{
  "urls": [
    "https://apify.com"
  ],
  "removeEmpty": true
}

🔧 Text Scraper Input Fields

Field	Type	Required	Default	Description
urls	array of strings	✅ Yes	["https://apify.com"]	List of URLs to scrape text from. Each entry is fetched and parsed into one output record. Rendered in the Console as a string list, one URL per line.
removeEmpty	boolean	❌ No	true	If true, empty lines will be removed from the text content. Set to false to keep the original blank-line spacing in textContent.

💡 Input Examples

Single page, default settings

{
  "urls": ["https://example.com/about"]
}

Batch of pages across several domains, compact output

{
  "urls": [
    "https://example.com/pricing",
    "https://another-site.org/docs/getting-started",
    "https://third-site.net/blog/2026-outlook"
  ],
  "removeEmpty": true
}

Preserving original line spacing for layout-sensitive text

{
  "urls": [
    "https://example.com/legal/terms",
    "https://example.com/legal/privacy"
  ],
  "removeEmpty": false
}

---

📤 Output

Each URL produces one JSON object in the dataset. Below is a real record from an actual run, with the long textContent value truncated for readability.

{
  "title": "Apify: The largest marketplace of trusted tools for AI",
  "url": "https://apify.com",
  "textContent": "Apify: The largest marketplace of trusted tools for AI\nSkip to content\nGet started\nLog in\nProdu…"
}

Note how the text arrives newline-separated and in document order — the title first, then the skip link, then the navigation labels, then the body copy. That sequence is preserved for the whole page.

🧾 Text Scraper Output Fields

Field	Type	Description
title	string | null	Title of the item — the page's document title.
url	string | null	Canonical URL of the scraped item.
textContent	string | null	Text content of the item: the page's visible text with HTML markup removed, newline-separated in document order. Blank lines are stripped when removeEmpty is true.

You can download the dataset in JSON, JSONL, CSV, Excel, XML or HTML. JSON and JSONL preserve the newline characters inside textContent exactly as scraped, which matters if you plan to split the text on line breaks later. CSV and Excel are the better choice when the output is going to an analyst or a stakeholder who just wants to read it.

---

💻 How to Use the Text Scraper (Step by Step)

Step 1: Build Your URL List for Text Extraction

Start by gathering the pages whose text you want. Useful sources include a sitemap.xml export, an RSS feed, a crawl report from an SEO tool, a competitor watchlist, a documentation table of contents, or simply a manual shortlist of pages you care about. The text scraper accepts a plain array of strings, so any of those sources can be reshaped into valid input with a spreadsheet formula or a couple of lines of script. Prefer a focused list over an indiscriminate one — fifty relevant pages produce a far more useful dataset than five thousand pages of pagination and tag archives.

Step 2: Open the Actor and Enter Your URLs

In the Apify Console, open Text Scraper (Free) and go to the Input tab. The urls field renders as a string list editor, so you can add entries one at a time or paste a whole block of URLs at once. The field is pre-filled with a sample URL you can replace. If you prefer working in raw JSON, switch to the JSON editor and supply the object shown in the Input section above.

Step 3: Decide How to Handle Empty Lines

The removeEmpty toggle controls whether blank lines survive into textContent. Leave it on — the default — when the text is heading into an LLM, an embedding model or a keyword analysis, because stripping blank lines makes the text more compact and avoids wasting tokens on whitespace. Switch it off when the vertical spacing itself carries meaning: legal documents, poetry, formatted disclaimers, or any case where you intend to reconstruct the page's visual rhythm from the text.

Step 4: Run the Text Scraper

Click Start and watch the log. The actor works through the URL list, fetching each page over HTTP and parsing it into the three-field record shape. Because there is no browser to launch per page, the log stays readable and throughput is high — you will see steady per-URL progress rather than long pauses.

Step 5: Review the Extracted Text in the Dataset

When the run finishes, open the Storage → Dataset tab. The preview table shows one row per URL with title, url and textContent as columns. Check the title column first: if the titles look right, the scraper reached the pages you intended. Then open two or three rows and read the start of textContent to confirm the body copy you expected is present and that navigation chrome has not swamped the substance.

Step 6: Export or Pipe the Text Data Onward

Use the export button to download JSON, JSONL, CSV, Excel, XML or HTML, or pull items through the API using the examples in the next section. For text pipelines, JSONL is often the most convenient format: one JSON object per line maps directly onto how most document loaders and chunking libraries expect to read a corpus.

Step 7: Schedule Recurring Text Scraping Runs

For ongoing monitoring, attach an Apify schedule so the same URL list is re-scraped daily, weekly or hourly. Add a webhook that fires on run success and your own system can pull new text the moment it lands. Comparing textContent between two scheduled runs is the simplest possible change-detection mechanism — if the string differs, the page changed.

---

🔌 API Access & Integrations

Run the text scraper directly from the Apify API and get the dataset items back in one synchronous call.

curl -X POST "https://api.apify.com/v2/acts/scrapers-hub~text-scraper-free/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://example.com/pricing",
      "https://example.com/docs/getting-started"
    ],
    "removeEmpty": true
  }'

The same run from Python using the official client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")

run_input = {
    "urls": [
        "https://example.com/pricing",
        "https://example.com/docs/getting-started",
    ],
    "removeEmpty": True,
}

run = client.actor("scrapers-hub/text-scraper-free").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    text = item.get("textContent") or ""
    print(item["title"], "—", item["url"])
    print(f"{len(text.split())} words extracted")
    print(text[:300])

Beyond the API, the actor connects to Zapier, Make, Google Sheets, Slack and generic HTTP webhooks, so extracted page text can flow into your content database, documentation pipeline, notification channels or spreadsheets without custom glue code.

---

💡 Best Use Cases for Website Text Data

🤖 Feeding RAG Pipelines and AI Knowledge Bases

Retrieval-augmented generation needs clean plain text, and textContent delivers exactly that without a markup-stripping step in between. Chunk the text, embed it, and store each chunk alongside its url and title so your assistant can cite the source page in its answers. Running with removeEmpty enabled keeps chunks dense, which means fewer tokens spent on whitespace and more usable context per retrieval.

🔍 Content Audits and SEO Analysis

Pull title and textContent for every page on a site and you have the raw material for a full content audit in one dataset. Compare title values across pages to find duplicates and missing titles, count words in textContent to spot thin pages, and search the text for target keywords to check whether the terms you rank for actually appear in the copy. Grouping by url path segment turns that into a section-by-section quality report.

📊 Competitor Messaging and Positioning Research

Scrape the homepage, pricing page, features pages and about page of every competitor in your market, then read the textContent fields side by side. Because the extraction is uniform across sites, you can run the same keyword and phrase analysis over all of them and see which claims, product terms and value propositions cluster where — something that is tedious to do by hand across a dozen tabs.

🕵️ Page Change Monitoring and Version Tracking

Websites change quietly. Re-run the same urls list on a schedule and compare each page's textContent string against the previous capture: any difference is a change, and a line-level diff shows you precisely which sentence moved. This works well for tracking competitor pricing pages, supplier terms, policy documents and product specifications where you need to know the moment wording shifts.

⚖️ Legal, Policy and Compliance Text Collection

Terms of service, privacy policies, cookie notices and regulatory disclosures are text-heavy documents where exact wording matters. Scraping them with removeEmpty set to false preserves the original blank-line spacing, keeping clause separation intact so the captured text reads closer to the published document. Store each capture with its url and the run date to build a defensible archive of what a page said and when.

🌍 Translation and Localisation Workflows

Translation tools want text, not HTML. Extracting textContent for a set of pages gives translators and machine translation APIs clean source material with no risk of a stray tag corrupting the output. The title field travels with it, so page titles get localised alongside the body copy rather than being forgotten.

📚 Building Training Corpora and Research Datasets

Academic and applied research often starts with "collect the text of these N pages". This text scraper turns that into a single run. Because the schema is only three fields, the resulting dataset drops straight into pandas, a text-processing notebook or a document store, and url keeps every document attributable to its source — which reviewers and reproducibility standards will expect.

---

⚙️ Tips for Better Text Scraping Results

• Point at content pages, not navigation hubs. Category listings, tag archives and paginated indexes are mostly link labels, so their textContent is dominated by navigation text. Submit the individual article, product or documentation permalinks when you want substance.
• Keep removeEmpty on unless spacing matters. The default produces compact text that tokenises efficiently and is easier to scan. Switch it off only when blank lines carry structural meaning, such as in legal or formatted documents.
• Deduplicate and normalise URLs before submitting. The same page frequently appears with tracking parameters, trailing slashes, www variants and AMP versions. Stripping query strings and canonicalising the list first avoids scraping the same content several times.
• Test a small sample from each new domain. Run five representative URLs before committing a batch of five hundred. That immediately tells you whether the site serves its text in the initial HTML and whether the extraction gives you what you need.
• Split very large lists across scheduled runs. Smaller batches keep the log readable, make it obvious which batch produced an anomaly, and reduce the amount of work lost if a run needs restarting.
• Filter navigation noise downstream, not upstream. Because every page's header and footer text appears in textContent, the same boilerplate lines repeat across all records from one site. Detecting and removing lines that appear on every page is a reliable way to isolate the unique body copy.

---

🛠️ Troubleshooting

Why is textContent empty or unexpectedly short for some URLs? Some sites build their body copy with client-side JavaScript after the initial HTML arrives. This text scraper fetches the served HTML directly rather than executing a browser, so content injected purely at runtime will not be present. Check by viewing the page source in your browser: if the text is missing from the raw HTML, it is being rendered client-side. Server-rendered and static pages — the large majority of documentation, blogs, news and marketing sites — extract normally.

Why does textContent include menu items, cookie banners and footer links? The scraper extracts the page's visible text, and navigation labels, banners and footers are visible text. This is faithful extraction rather than a bug. Because that boilerplate is identical across every page of a site, the simplest fix is downstream: collect all records from one domain, find the lines that repeat on every page, and drop them to leave the unique content.

A URL produced no record at all — what happened? The most common causes are a URL that is not publicly reachable without a login, a page behind a cookie or consent wall, a regional redirect that returns an error for the request's origin, or a typo in the URL. Open the link in a private browser window with no session to confirm what an anonymous visitor actually receives.

Why did I get fewer records than URLs submitted? Duplicate entries in the urls array and links that fail to resolve are the usual reasons. Compare the url values in the dataset against your input list to identify which entries dropped out, then re-check those specific links directly.

The text arrives as one long line and I expected paragraphs. The textContent value is newline-separated, but some export formats and spreadsheet viewers collapse or escape newline characters when displaying a cell. Export as JSON or JSONL and inspect the raw value to see the line breaks as scraped; splitting the string on \n in your own code gives you the individual lines back.

---

❓ Frequently Asked Questions About Text Scraping

What does Text Scraper (Free) actually do? It fetches each URL you supply, removes the HTML markup, scripts and styling, and returns the page's readable text along with the page title and canonical URL — one record per page.

Do I need to write CSS selectors or XPath for each site? No. The text scraper applies the same generic extraction to every URL, so there is no per-site configuration and nothing to repair when a website is redesigned.

What input does the text scraper require? Only urls, an array of the pages you want to scrape. The optional removeEmpty boolean controls blank-line handling and defaults to true.

Can I scrape multiple websites in one run? Yes. The urls array can mix domains freely. Each URL is fetched and parsed independently, and each record carries its own url so results stay attributable.

What does the removeEmpty option change? When true — the default — empty lines are removed from textContent, producing compact text. When false, the original blank-line spacing is kept, which is useful for legal documents and other text where vertical spacing carries meaning.

Does this text scraper use a headless browser? No. It uses direct HTTP requests with an HTML parser, which makes runs considerably faster and lighter. The trade-off is that pages which build their entire body in the browser after load will not yield their text.

Does it extract full page text or just a summary? Full page text. textContent contains the complete visible text of the page in document order, not a truncated snippet or meta description.

Is Text Scraper (Free) really free? The actor carries no per-result charge. You pay only for the Apify platform compute your runs consume, which is covered by the free platform tier for modest workloads.

How do I get the extracted text out of Apify? Export the dataset as JSON, JSONL, CSV, Excel, XML or HTML from the Console, or pull items programmatically through the API and the apify_client library. Both patterns are shown in the API Access section above.

Can I schedule the text scraper to run automatically? Yes. Attach an Apify schedule to re-run the same URL list on any cadence, and add a webhook so downstream systems are notified as soon as fresh text lands.

Is there a limit on how many URLs I can submit? The input schema sets no fixed cap. Practical limits come from run duration and memory, so very large lists are best split across several runs.

Can I use this text scraper on pages behind a login or paywall? No. Only content a website serves to anonymous visitors can be retrieved. Pages requiring authentication return whatever a logged-out visitor would see, which is usually a teaser or a login prompt.

How do I detect when a page's text has changed? Scrape the same URL on a schedule and compare the textContent strings between runs. Any difference means the page changed, and a line-level diff pinpoints exactly where.

Why is title sometimes null or different from the visible heading? The title field reflects the page's document title, which is not always identical to the on-page heading — sites often append a brand name or use a shorter tab title. If a page declares no title at all, the field is null.

What formats can I export the text data in? Apify datasets export to JSON, JSONL, CSV, Excel, XML, HTML and RSS. JSON and JSONL are the best choices for text work because they preserve the newline characters inside textContent intact.

---

🆘 Support & Feedback

Hit a page that extracts poorly, found a bug, or run into an edge case? Open a ticket on the actor's Issues tab with the exact URL and a short note on what you expected to see. Reproducible reports get resolved fastest.

Need something customised — additional fields, a different extraction strategy, deeper crawling across a whole site, or a private actor built around your specific set of sources? Email scraperhubapi@gmail.com and describe what you are trying to build.

If Text Scraper (Free) saves you time, please leave a review on the Apify Store. Ratings and written feedback genuinely shape which improvements get built next.

---

⚖️ Disclaimer

Text Scraper (Free) is built to collect publicly available data — pages any visitor can reach without logging in, bypassing a paywall or circumventing an access control. It makes no attempt to defeat authentication or subscription barriers.

You are responsible for how you use the text you extract. Before running the text scraper at scale, review the target website's Terms of Service and robots.txt and respect the restrictions they set out. Reasonable request volumes and sensible scheduling are part of scraping responsibly, and hammering a site is neither necessary nor acceptable.

Web pages can contain personal data — names, contact details, biographies and user-generated comments may all appear inside textContent. If you process personal data relating to people in the EU, UK or other regulated jurisdictions, you must comply with the GDPR and equivalent privacy laws: establish a lawful basis, honour data subject rights, minimise what you retain, and delete what you no longer need. Copyright applies too — extracted text remains the property of its publisher, and scraping it grants no licence to republish it.

If you believe text collected by this actor relates to you and you would like it removed, contact scraperhubapi@gmail.com with the details and we will action the request.