# Article Extractor: News & Blog Article Text Scraper (`fecklesspleb/article-extractor`) Actor

Article scraper for news sites and blogs: headline, publish and update dates, language, lead image and full body text without menus, ads or comments. Paste article URLs, or a section page and it finds the articles. Markdown option. Fast HTTP, robots.txt respected. $0.50 per 1,000 articles.

- **URL**: https://apify.com/fecklesspleb/article-extractor.md
- **Developed by:** [Scott Williams](https://apify.com/fecklesspleb) (community)
- **Categories:** News, AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Article Extractor: News & Blog Article Text Scraper

Extract clean, structured data from news articles and blog posts. Each article gives you:

- headline and description
- site name, language and canonical URL
- publish and update dates
- lead image
- tags as declared in the page metadata
- the full body text, with menus, ads, related links and comments removed
- a word count and an estimated reading time

**At a glance:** $0.50 per 1,000 articles, plus $0.00005 per run start. Pay per result with no subscription. You can set a maximum cost per run, export to JSON, CSV or Excel, or call it from the Apify API, Make, Zapier or n8n.

There are two ways to use it:

- **Article URLs.** Give it the article links you already have.
- **Section pages.** Give it a section page, such as a news site's technology section or a blog index. It finds the article links on that page and extracts each one.

It runs on fast plain HTTP with no browser, and robots.txt is respected (RFC 9309) on every request.

### Use cases

- **Media monitoring:** pull the latest articles from the sections you follow every hour (schedule the Actor).
- **LLM work:** feed full article text to an LLM for summaries, sentiment or topic tagging. Turn on `includeMarkdown` to keep headings and links.
- **Research datasets:** build a dataset of articles with dates and sources.
- **Link enrichment:** add titles, dates and text to a list of links shared in Slack, a newsletter or a spreadsheet.

### Input

```json
{
  "articleUrls": [{"url": "https://arstechnica.com/gadgets/2026/10/from-cheap-tool-to-premium-subscription-when-a-beloved-apps-price-skyrockets/"}],
  "sectionUrls": [{"url": "https://arstechnica.com/gadgets/"}],
  "maxArticlesPerSection": 20,
  "minWords": 100,
  "includeMarkdown": false,
  "maxConcurrency": 8,
  "maxArticles": 0
}
```

| Field | Default | What it does |
|---|---|---|
| `articleUrls` | one example page | Direct links to articles or blog posts. |
| `sectionUrls` | none | Pages that list articles. Article links anywhere on the same site are detected from the URL pattern (dates, long slugs, numeric IDs), so a section page can also yield articles from other sections of that site. |
| `maxArticlesPerSection` | `20` | How many discovered links to extract from each section page. |
| `minWords` | `100` | Pages with less body text are skipped for free. This catches index pages, video pages and paywall stubs. |
| `includeMarkdown` | `false` | Also return the body as Markdown, with headings, lists and links. |
| `maxConcurrency` | `8` (max 20) | Articles fetched in parallel. Requests to one host are spaced at least 0.5 s apart. |
| `maxArticles` | `0` | Stop after this many articles. 0 means no cap. |

### Output example

Real output from a test run (body text shortened):

```json
{
  "url": "https://arstechnica.com/gadgets/2026/10/from-cheap-tool-to-premium-subscription-when-a-beloved-apps-price-skyrockets/",
  "finalUrl": "https://arstechnica.com/gadgets/2026/10/from-cheap-tool-to-premium-subscription-when-a-beloved-apps-price-skyrockets/",
  "title": "From cheap tool to premium subscription: When a beloved app's price skyrockets",
  "description": "Huge price hikes force loyal users to reconsider their favorite apps.",
  "siteName": "Ars Technica",
  "publishedAt": "2026-10-27T15:50:19+00:00",
  "modifiedAt": "2026-10-27T17:57:01+00:00",
  "language": "en",
  "leadImageUrl": "https://cdn.arstechnica.net/wp-content/uploads/2026/10/GettyImages-1321941710-1152x648.jpg",
  "tags": [],
  "canonicalUrl": "https://arstechnica.com/gadgets/2026/10/from-cheap-tool-to-premium-subscription-when-a-beloved-apps-price-skyrockets/",
  "text": "How much are you willing to pay to keep using your favorite app?\nMaybe you’ve used it for years. Perhaps it performs a function so essential and specific that y ...",
  "wordCount": 319,
  "readingTimeMinutes": 2,
  "extractedAt": "2026-10-28T03:58:00+00:00"
}
```

`tags` is empty when a site does not declare tags in its metadata. Author bylines are not collected, so the output stays free of personal data.

### Pricing

Pay per extracted article: **$0.50 per 1,000 articles** ($0.0005 per article), plus the standard Apify run start event of $0.00005 per run.

These are free:

- section pages
- failed URLs
- pages under **Minimum words**

If a run extracts no articles at all, it writes one free status row (`recordType: "status"`) that says why, so the dataset is not left empty. The exception is when your maximum cost per run stops the run before its first article.

Set **Maximum cost per run** or `maxArticles`, and the Actor stops cleanly when that limit is reached.

### Use with AI agents / MCP

AI agents can run this Actor as a tool through [Apify's MCP server](https://docs.apify.com/platform/integrations/mcp). Add this server URL to your MCP client (Claude, Cursor, VS Code, OpenAI Agents SDK, LangChain and others) and sign in with OAuth, or send your Apify token as `Authorization: Bearer <token>`:

```text
https://mcp.apify.com?tools=fecklesspleb/article-extractor
```

The tool is named `fecklesspleb--article-extractor`. Agents can also find it with `search-actors` and run it with `call-actor`. The run returns a dataset ID; read the rows with `get-dataset-items`, and pass `fields` (for example `url,title,publishedAt,text`) to keep the response small.

Minimal tool input:

```json
{"articleUrls": [{"url": "https://en.wikipedia.org/wiki/Web_scraping"}]}
```

The output of this input (a real item from a run, `text` shortened):

```json
{
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "finalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "title": "Web scraping - Wikipedia",
  "description": null,
  "siteName": "Wikimedia Foundation, Inc.",
  "publishedAt": "2005-09-17",
  "modifiedAt": null,
  "language": "en",
  "leadImageUrl": null,
  "tags": [],
  "canonicalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "text": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. ...",
  "wordCount": 4142,
  "readingTimeMinutes": 19,
  "extractedAt": "2026-11-03T15:31:18.854793+00:00"
}
```

Tips for agents:

- Cost is $0.0005 per extracted article. Set `maxArticles`, and set `maxTotalChargeUsd` in the run options (`callOptions` in `call-actor`) to cap a run.
- With `sectionUrls`, also pass `"articleUrls": []` so the default example article is not extracted as well. `maxArticlesPerSection` (default 20) limits each section page.
- URLs that fail or have fewer than `minWords` words get no row and are free, so compare the rows you get with the URLs you sent. If no article is extracted, the dataset holds one free row with `recordType: "status"` and a `message` saying why.
- The `RUN_SUMMARY` record in the run's key-value store shows `candidateUrls`, `articles`, `failed` and `tooShort`.

### Limits

- There is no JavaScript rendering and no login, so paywalled text and browser-rendered pages are not extracted.
- Some sites block automated clients or disallow them in robots.txt. Those URLs are skipped and logged.
- On the Apify platform, URLs whose address is private, loopback or link-local (also after a redirect) are not fetched. They are skipped, logged and not charged.
- Articles split over several pages return only the first page.
- Section discovery works from URL patterns. Busy home pages can include links to other sections or to podcast and index pages; `minWords` filters out most of them.

**What happens with invalid input?** If the input is not a JSON object, or a field has the wrong type (for example a number where a list of URLs is expected, or `"abc"` for a number), the run fetches nothing and ends with one free status row: `Invalid input, nothing was fetched and no results were charged: <reason>`. No results are charged for it; the standard Apify start event still applies.

### Is this legal?

The Actor reads only public pages you point it at, identifies itself, spaces requests and obeys robots.txt. Article text is usually copyrighted. Use it for analysis, search and research in line with each publisher's terms, and do not republish it without permission.

# Changelog

This Actor's version history is a separate document: https://apify.com/fecklesspleb/article-extractor/changelog.md

# Actor input Schema

## `articleUrls` (type: `array`):

Direct links to articles or blog posts, as objects: \[{"url": "https://en.wikipedia.org/wiki/Web_scraping"}]. One row per article. A URL without http(s):// gets https:// added and duplicates are removed. When you use only sectionUrls, pass "articleUrls": \[] so the default example page is not also extracted.

## `sectionUrls` (type: `array`):

Pages that list articles, such as a news section, blog index or home page, as objects: \[{"url": "https://example.com/blog/"}]. Links on the same site are treated as articles when the URL has a /20xx/ date folder, a last segment with 3 or more hyphens, or 5 or more digits, and is not a tag, category, author, search or similar page. Section pages themselves are free.

## `maxArticlesPerSection` (type: `integer`):

How many discovered article links to extract from each section page, in page order. Default 20.

## `minWords` (type: `integer`):

Skip pages with fewer words of body text than this (index pages, video pages, paywall stubs). Default 100. Skipped pages are free and get no row.

## `includeMarkdown` (type: `boolean`):

If true, each row also gets markdown: the body with headings, lists and links. Default false; text (plain body text) is always included.

## `maxConcurrency` (type: `integer`):

Articles fetched in parallel, 1 to 20. Default 8. Requests to one host are spaced at least 0.5 s apart.

## `maxArticles` (type: `integer`):

Stop after this many extracted (charged) articles. Default 0 = no cap. Each article costs $0.0005.

## Actor input object example

```json
{
  "articleUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "sectionUrls": [],
  "maxArticlesPerSection": 20,
  "minWords": 100,
  "includeMarkdown": false,
  "maxConcurrency": 8,
  "maxArticles": 0
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "articleUrls": [
        {
            "url": "https://en.wikipedia.org/wiki/Web_scraping"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fecklesspleb/article-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "articleUrls": [{ "url": "https://en.wikipedia.org/wiki/Web_scraping" }] }

# Run the Actor and wait for it to finish
run = client.actor("fecklesspleb/article-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "articleUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ]
}' |
apify call fecklesspleb/article-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fecklesspleb/article-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/54CVSDszNebV7aylr/builds/xWAwF3RNZps0Xxn5n/openapi.json