# Article Text Extractor: Clean Text & Markdown from Any URL (`larkmoor/article-text-extractor`) Actor

Extract the clean main text of any article or blog post as plain text and Markdown, with title, author, publish date, site, language, lead image, tags and word count. Ads, menus and footers removed. Paste URLs or discover articles from a homepage, RSS feed or sitemap. $1 per 1,000 articles.

- **URL**: https://apify.com/larkmoor/article-text-extractor.md
- **Developed by:** [Larkmoor Data](https://apify.com/larkmoor) (community)
- **Categories:** AI, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.85 / 1,000 article extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Article Text Extractor: Clean Text & Markdown from Any URL

Turn any article, blog post, press release or docs page into clean, structured data: the main text as plain text **and** Markdown, plus the title, author, publish date, site name, language, lead image, tags, categories, word count and reading time. Menus, ads, cookie banners, "related stories", footers and comments are stripped out, so what you get is the article itself.

Paste a list of URLs, or point it at a homepage, a section page, an RSS/Atom feed or a sitemap and let it find the articles for you. It is plain HTTP — no browser — so it is fast and cheap, and it reads only pages the site's robots.txt allows.

### What you can use it for

- **LLM and RAG pipelines**: feed clean Markdown (headings, lists and tables kept) into embeddings, summaries or a chatbot knowledge base instead of raw HTML full of navigation
- **News monitoring**: pull the full text of every new story from the feeds you follow, on a schedule
- **Content research and SEO**: word counts, publish dates, authors and tags across a competitor's blog
- **Archiving and datasets**: build a corpus of articles with clean text and consistent metadata
- **Brand and PR tracking**: extract the coverage your PR team collected as URLs, ready for analysis

### Why this one

- **$1 per 1,000 articles**, and you only pay for pages where an article was actually extracted — failed, blocked or too-short pages are free
- **Text and Markdown in one run** (clean HTML too, if you want it)
- **Rich metadata**: author, publish date, site name, language, lead image, tags and categories, read from the page's own metadata
- **Discovery built in**: a homepage or section page, an RSS/Atom feed or an XML sitemap (index files too) all work as a starting point; the site's own feed is used first when the page advertises one
- **Token-budget friendly**: optional character cap per field, optional link stripping
- **Honest rows**: a page that shows a bot check, a 404 or no real article says so in `error` instead of returning junk text

### Input

| Field | What it does |
|---|---|
| Article URLs | Pages to extract, one per line |
| Discover articles from | Homepages, section pages, RSS/Atom feeds or sitemaps |
| Max articles per discovery source / in total | Caps — you never pay for more than this |
| Plain text / Markdown / Clean HTML | Which body formats to return |
| Keep links / Keep tables | Formatting options for Markdown and HTML |
| Extraction mode | Balanced (more text) or Precision (cleanest text) |
| Minimum words | Pages below this are reported as "not an article" (free) |
| Max characters per field | Cut long bodies (0 = no limit) |
| Only output extracted articles | Drop failed rows from the dataset |
| Pages at once | 1–10 in parallel |
| Respect robots.txt | On by default |

Example input:

```json
{
  "articleUrls": ["https://en.wikipedia.org/wiki/Web_scraping"],
  "discoverFrom": ["https://techcrunch.com/feed/", "https://www.npr.org/"],
  "maxArticlesPerSite": 20,
  "includeMarkdown": true,
  "maxCharacters": 20000
}
```

### Output

One row per page. A real example (text shortened):

```json
{
  "url": "https://techcrunch.com/2026/10/25/5-best-apps-track-reading-discover-new-books/",
  "finalUrl": "https://techcrunch.com/2026/10/25/5-best-apps-track-reading-discover-new-books/",
  "title": "The best apps for tracking your reading and discovering new books",
  "author": "Aisha Malik",
  "publishedAt": "2026-10-25",
  "siteName": "TechCrunch",
  "hostname": "techcrunch.com",
  "language": "en",
  "description": "There are many apps that are designed to help you track your reading and discover new books, so we compiled a list of some of the best ones.",
  "image": "https://techcrunch.com/wp-content/uploads/2022/12/Screenshot-2026-10-22-at-2.48.45-PM.png?resize=1200,831",
  "tags": [
    "apps",
    "goodreads"
  ],
  "categories": [
    "Apps"
  ],
  "wordCount": 713,
  "readingTimeMinutes": 3,
  "text": "From discovering new books and organizing your reading list to tracking your reading goal and sharing recommendations, book apps make it easier to maintain reading as a hobby. There are numerous apps on the market that p…",
  "markdown": "From discovering new books and organizing your reading list to tracking your reading goal and sharing recommendations, book apps make it easier to maintain reading as a hobby. There are numerous apps on the market that p…",
  "httpStatus": 200,
  "extractedAt": "2026-10-26T10:39:05Z",
  "error": null
}
```

Fields: `url`, `finalUrl`, `title`, `author`, `publishedAt`, `siteName`, `hostname`, `language`, `description`, `image`, `tags`, `categories`, `wordCount`, `readingTimeMinutes`, `text`, `markdown`, `html`, `httpStatus`, `extractedAt`, `error`.

Download as JSON, CSV, Excel or HTML, or read the dataset through the API.

### How much does it cost?

**$0.001 per extracted article** ($1 per 1,000) plus a tiny start fee. Pages with no article (errors, bot checks, pages under the minimum word count) cost nothing. Apify's free plan credit covers several thousand articles a month.

Worked examples:

1. **A daily news digest**: 5 feeds × 20 stories = 100 articles a day → $0.10 a day, about $3 a month
2. **A RAG knowledge base from a 3,000-page docs site sitemap**: 3,000 × $0.001 = $3.00, once
3. **PR coverage report**: 250 article links from your agency → $0.25

Higher Apify plans get an automatic discount (Bronze −5%, Silver −10%, Gold and up −15%).

### FAQ

**Is this legal?** The actor reads publicly available pages over plain HTTP, at a polite pace, and respects each site's robots.txt by default. It does not log in, bypass paywalls or solve bot checks. What you may do with the text depends on the site's terms and the copyright rules where you are — summaries, research and internal analysis are typical uses; republishing full articles usually needs permission.

**Does it work on paywalled sites?** Only on what the site shows to any visitor without logging in. Paywalled parts are not extracted.

**A page came back with "probably not an article".** Its main text was shorter than "Minimum words" (video pages, index pages, apps that need JavaScript). Lower the minimum if you want short pages too.

**Does it run JavaScript?** No — it reads the HTML the server sends, which is how most news sites and blogs deliver their articles. Single-page apps that render text only in the browser will come back short.

**What languages work?** Any language; the `language` field is filled from the page.

**Can I get every article on a site?** Use the site's sitemap as the discovery source and raise the caps. Feeds usually only list the latest 10–50 stories.

### Integrations

Schedule it and send the dataset to Google Sheets, Airtable, Slack, a webhook, a vector database or your own app with Apify integrations, Make, Zapier, n8n, LangChain or the Apify API.

### Changelog

- 0.1 (Oct 2026): first release — text + Markdown + clean HTML, metadata, feed/sitemap/page discovery, free failed rows, parallel fetching.

# Actor input Schema

## `articleUrls` (type: `array`):

Pages to extract, one per line: news articles, blog posts, docs pages, press releases. Each page gives one row.

## `discoverFrom` (type: `array`):

Homepages, section pages, RSS/Atom feeds or sitemaps. The actor finds article links there (the site's own feed first, then article-looking links on the page) and extracts them.

## `maxArticlesPerSite` (type: `integer`):

How many article links to take from each source in 'Discover articles from' (newest first where the source is a feed).

## `maxArticles` (type: `integer`):

Hard cap on the pages extracted in this run.

## `includeText` (type: `boolean`):

Return the main text as plain text (paragraphs separated by new lines).

## `includeMarkdown` (type: `boolean`):

Return the main text as Markdown with headings, lists, bold/italic and tables kept — ready for LLMs and RAG.

## `includeHtml` (type: `boolean`):

Also return the main content as clean, minimal HTML.

## `keepLinks` (type: `boolean`):

Keep the hyperlinks inside the article text.

## `includeTables` (type: `boolean`):

Keep tables that are part of the article.

## `extractionMode` (type: `string`):

Balanced keeps more of the article (good default). Precision drops anything doubtful, for the cleanest text.

## `minWords` (type: `integer`):

Pages with less main text than this are reported as 'not an article' and not charged. 0 keeps everything.

## `maxCharacters` (type: `integer`):

Cut the text/Markdown fields to this many characters (0 = no limit). Handy for LLM token budgets.

## `skipFailed` (type: `boolean`):

Leave out rows for pages that failed or had no article (they are never charged either way).

## `concurrency` (type: `integer`):

How many pages to fetch in parallel (1-10).

## `respectRobotsTxt` (type: `boolean`):

Skip pages the site's robots.txt disallows.

## Actor input object example

```json
{
  "articleUrls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "discoverFrom": [
    "https://techcrunch.com/feed/"
  ],
  "maxArticlesPerSite": 20,
  "maxArticles": 1000,
  "includeText": true,
  "includeMarkdown": true,
  "includeHtml": false,
  "keepLinks": false,
  "includeTables": true,
  "extractionMode": "balanced",
  "minWords": 80,
  "maxCharacters": 0,
  "skipFailed": false,
  "concurrency": 5,
  "respectRobotsTxt": true
}
```

# Actor output Schema

## `results` (type: `string`):

All results as JSON items in the run's default dataset (also downloadable as CSV/Excel from the dataset).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "articleUrls": [
        "https://en.wikipedia.org/wiki/Web_scraping"
    ],
    "discoverFrom": [
        "https://techcrunch.com/feed/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("larkmoor/article-text-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "articleUrls": ["https://en.wikipedia.org/wiki/Web_scraping"],
    "discoverFrom": ["https://techcrunch.com/feed/"],
}

# Run the Actor and wait for it to finish
run = client.actor("larkmoor/article-text-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "articleUrls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "discoverFrom": [
    "https://techcrunch.com/feed/"
  ]
}' |
apify call larkmoor/article-text-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,larkmoor/article-text-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/EXHPpIsB7hhHbxMAv/builds/oMpdq9jXpTQ3QrHGE/openapi.json