# Article Text Extractor - News & Blog Content, Date, Image (`sophisticated_occasion/article-extractor-news-blog`) Actor

Clean article text, publish date and lead image from news and blog URLs, with boilerplate removed. The same article is charged once and monitor mode charges only new articles. Respects robots.txt. $0.00099 per article; failures free.

- **URL**: https://apify.com/sophisticated_occasion/article-extractor-news-blog.md
- **Developed by:** [HUG0 S](https://apify.com/sophisticated_occasion) (community)
- **Categories:** News, AI, SEO tools
- **Stats:** 2 total users, 1 monthly users, 91.7% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.99 / 1,000 article extracted (main text returned)s

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Article Text Extractor - News & Blog Content, Date, Image

Paste article URLs, get clean structured data back: **title, main text (Markdown or plain), publish date, lead image, site name, language, tags/categories, canonical URL, word count and reading time**. Boilerplate (menus, ads, footers, cookie banners) is removed.

### Why this one

- **Clean text, not page soup.** Title, main text (Markdown or plain), publish date, lead image, language and tags in one step, with menus, ads, footers and cookie banners removed.
- **The same article is charged once.** URLs that differ only by tracking parameters, `#anchors`, `www.` or the same canonical URL or identical text are delivered once; the duplicates are not charged.
- **Pay only for new articles when monitoring.** Set `monitorName` and run on a schedule: articles already extracted by that monitor are skipped for free.
- **Failures are free:** blocked, paywalled, non-article or too-short pages cost nothing, and robots.txt is checked for every URL.

### Quick start: example input, output and cost

Extract clean article text, publish date, lead image, tags and language from news and blog URLs. Markdown or plain text, no start fee, failures free.

**Example input**

```json
{
  "urls": [
    {
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html"
    },
    {
      "url": "https://blog.python.org/"
    }
  ]
}
```

**Cost example:** 1,000 articles x $0.00099 = **$0.99**.

### Duplicates are charged once, and only new articles on repeat runs

- The same article reached through several URLs (tracking parameters, `#anchors`, `www.` or not, the same canonical URL, or identical text) is delivered and charged once. The other URLs appear in `ERRORS` with "delivered once. Not charged."
- Set `monitorName` (for example `eu-news`) and run on a schedule: articles already extracted in an earlier run of that monitor are skipped for free, so you only pay for new ones.

### Limits

- It reads the HTML the server sends (no browser), so text, links or markup added later by JavaScript are not seen.
- Paywalled or login-only articles are not extracted (free).
- Pages that are not articles (home pages, listings) are skipped as too short (free).

### Pricing (pay per event)

- **$0.00099 per article extracted.** No start fee.
- Failed, blocked, non-article or too-short pages are **free** and listed in the ERRORS record with the reason. Your max cost per run is respected exactly.

### Good for

RAG / LLM pipelines, news monitoring, content research and summarisation, SEO content audits, building reading lists.

### Responsible use

- robots.txt is checked for every URL (RFC 9309) and pages it disallows are skipped (free). Max 1 request per second per site, honest user agent, no browser, no paywall or anti-bot bypass, no logins.
- Social networks and profile sites are refused. Emails and phone numbers in the text are redacted by default. Author bylines are only included if you switch on `includeAuthor`.
- Respect the copyright of the articles you extract; use the text in line with the publisher's terms.

Maintained by DigiVibeTech.

# Actor input Schema

## `urls` (type: `array`):

News or blog article URLs (one item per article). Up to 5,000 per run.

## `urlsText` (type: `string`):

Alternative to the list: URLs separated by spaces, commas or new lines.

## `textFormat` (type: `string`):

Main text as Markdown (headings, lists, tables) or plain text.

## `minWords` (type: `integer`):

Pages with less main text are reported as failed and not charged.

## `includeAuthor` (type: `boolean`):

Off by default (personal data). Turn on only if you have a legitimate reason to store author names.

## `redactContactDetails` (type: `boolean`):

Replace emails and phone numbers in the text with \[redacted].

## `includeComments` (type: `boolean`):

Also extract reader comments when the page has them.

## `monitorName` (type: `string`):

Set a name to skip articles already extracted in earlier runs of the same monitor (matched by canonical URL or text). Skipped articles are free. Empty = off.

## Actor input object example

```json
{
  "urls": [
    {
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html"
    },
    {
      "url": "https://blog.python.org/"
    }
  ],
  "textFormat": "markdown",
  "minWords": 50,
  "includeAuthor": false,
  "redactContactDetails": true,
  "includeComments": false
}
```

# Actor output Schema

## `results` (type: `string`):

All result items (overview view)

## `errors` (type: `string`):

Inputs that could not be processed, with the reason

## `summary` (type: `string`):

Counts of processed, billed and failed items

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        {
            "url": "https://www.rfc-editor.org/rfc/rfc9309.html"
        },
        {
            "url": "https://blog.python.org/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("sophisticated_occasion/article-extractor-news-blog").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        { "url": "https://www.rfc-editor.org/rfc/rfc9309.html" },
        { "url": "https://blog.python.org/" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("sophisticated_occasion/article-extractor-news-blog").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    {
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html"
    },
    {
      "url": "https://blog.python.org/"
    }
  ]
}' |
apify call sophisticated_occasion/article-extractor-news-blog --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,sophisticated_occasion/article-extractor-news-blog"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/UHcJc53IauLe3B5DX/builds/QRfnWI6gOgcYQxr45/openapi.json