# Bulk Article Extractor: Full Text from a CSV, Sheet or Dataset (`nerolabs/bulk-article-extractor`) Actor

Returns full text, title, author, date, publisher, image and tags for every article linked in an Apify dataset, CSV, Excel or Google Sheet, keeping your columns. Inputs: datasetId, fileUrl or articleUrls. $0.002 per article extracted; blocked or failed links free. Agent-ready via x402 and MCP.

- **URL**: https://apify.com/nerolabs/bulk-article-extractor.md
- **Developed by:** [Adam Pearce](https://apify.com/nerolabs) (community)
- **Categories:** AI, News, Automation
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.40 / 1,000 articles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk Article Extractor: Full Text from a CSV, Sheet or Dataset

**Got a list of news or blog links and need the actual stories?** Point this Actor at a Google News scraper's dataset, an RSS export, a CSV or a Google Sheet, and every row comes back with the full article text, headline, author, publish date, publisher, image, tags and word count, **next to your own columns**.

Most article extractors only take a typed-in list of URLs and hand back a separate table you then have to match up again. This one reads the list you already have, finds the link column on its own, and keeps everything you had.

### What you can do with it

- **News monitoring:** run a Google News or RSS scraper every morning, feed its dataset in here, and get the full text of every new story about your brand, competitors or market. Turn on **Only new links** and you pay only for stories you have not seen before.
- **Feed an LLM or a RAG pipeline:** choose Markdown output and get clean text with headings, lists and links kept, and no menus, cookie banners or ads.
- **Content research:** pull 500 competitor blog posts from a sitemap export into one sheet with titles, authors, dates and word counts.
- **Media lists and PR reports:** who wrote what, when, for which outlet, from a list of coverage links.

### Example output

One row per link (your own columns, such as `headline` and `source` below, are kept):

```json
{
  "headline": "tea",
  "source": "https://techcrunch.com",
  "rowNumber": 1,
  "url": "https://techcrunch.com/2026/10/17/embers-new-temperature-controlled-mug-makes-my-nightly-tea-ritual-even-better/",
  "extractStatus": "ok",
  "title": "Ember's new temperature-controlled mug makes my nightly tea ritual even better",
  "author": "Lauren Forristal",
  "authors": ["Lauren Forristal"],
  "publishedAt": "2026-10-17T15:30:00.000Z",
  "siteName": "TechCrunch",
  "publisher": "TechCrunch",
  "section": "Hardware",
  "tags": ["Ember"],
  "language": "en-US",
  "imageUrl": "https://techcrunch.com/wp-content/uploads/2026/10/Ember_Table-with-Phone-Dark-Black-10oz_030326.png?w=1200",
  "wordCount": 660,
  "readingTimeMinutes": 3,
  "paywalled": false,
  "looksLikeArticle": true,
  "pageType": "NewsArticle",
  "text": "With the nights getting chillier, I've been reaching for a cup of tea more ...",
  "statusDetail": null
}
```

Every row has an `extractStatus`, so you always know why a link did or did not return a story:

| Status | Meaning | Charged? |
|---|---|---|
| `ok` | Article text returned | Yes, once |
| `paywall_preview` | The publisher marks it subscriber-only and served only a short preview. The preview is included | No |
| `too_short` | Under the minimum word count: a video, gallery or app page | No |
| `blocked` | The site's bot protection refused the request | No |
| `robots_disallowed` | The site's robots.txt asks tools not to read this page | No |
| `http_error`, `unreachable`, `timeout`, `not_html`, `too_large`, `invalid_url`, `no_url` | The link could not be read as a web page | No |
| `duplicate` | Same link as an earlier row in this run (tracking tags like `utm_source` ignored), so it was not fetched again | No |
| `skipped_seen` | Already extracted in an earlier run (Only new links) | No |

### How it reads an article

1. The page is downloaded the way a normal browser would request it. No headless browser, so it is fast and cheap.
2. The main story is found with **Mozilla Readability**, the engine behind Firefox Reader View, which strips navigation, footers, related links and ads.
3. Author, dates, publisher, section, tags and image come from the **publisher's own structured data** (JSON-LD and Open Graph tags) wherever the page has it, and only then from the visible byline.
4. `paywalled` is true only when the publisher itself marks the article as not free. We never guess.

### Pricing

Pay per event, no subscription:

| Event | Price |
|---|---|
| Article extracted (`ok` rows only) | **$0.002** |
| Residential proxy, per article returned (only if you turn it on) | $0.01 |
| CSV or Excel export file (only when the run extracted at least one article) | $0.01 per file |
| Webhook delivered (2xx only) | $0.005 |

Examples: 1,000 articles cost **$2**. A daily monitor that finds 50 new stories a day costs about **$3 a month**, because with Only new links on, the stories it already has are skipped for free. Blocked links, paywall previews, duplicates and failed links are never charged.

### Input

Give it **one** of:

- **Dataset:** the output of another Actor, picked from the list.
- **File or Google Sheet URL:** CSV, TSV, Excel, JSON or JSON Lines. A normal Google Sheets share link works when the sheet is set to "Anyone with the link can view".
- **Article URLs:** a plain list, for a quick test.
- **Rows:** inline JSON.

The link column is found automatically (`url`, `link`, `articleUrl` and similar, preferring links that have a real path over publisher homepages). Set **Article URL field** to choose it yourself.

Useful options: **Text output** (plain, plus Markdown, plus cleaned HTML, or metadata only), **Minimum words**, **Only new links** with a memory name per feed, **Which rows to keep**, CSV or Excel export, a named dataset to build a growing archive, and a webhook for Slack, Zapier, Make or n8n.

### Use it from an AI agent

The Actor is pay per event with limited permissions, so AI agents can call it through Apify's MCP server or pay per run through x402. Pass `articleUrls` (a list of links) or `datasetId`; read `text`, `title`, `author` and `publishedAt` from the results, and check `extractStatus` before using a row.

### FAQ

**Will it work on every news site?**
Most news sites, blogs and company newsrooms, yes. A few very large publishers block all automated readers whatever you do; those rows come back `blocked` and cost nothing. A proxy only helps when a site blocks by address or country, not when it checks the browser itself.

**Does it get around paywalls?**
No. It reads what the publisher serves to a normal visitor. When a subscriber-only article serves just a short preview, you get the preview free, marked `paywall_preview`.

**Is this legal?**
It reads publicly available pages, one link at a time, at most two at once per site, and honours robots.txt by default. What you do with the text is up to you: the articles belong to their publishers, so check your use (summaries, monitoring, research and internal analysis are the usual ones).

**Why is my dataset in a different order?**
Rows are saved as each site answers, so a slow site does not hold up the rest. Every row carries `rowNumber`, its position in your input, to sort by.

**Can it find the article links for me?**
No, it reads links you already have. Pair it with a Google News, RSS or sitemap scraper, and with our **Dataset Filter & Transform** to tidy the results.

If this saved you copying articles out by hand, a short review on the Apify Store helps a lot. Questions or a site that should work and does not? Open an issue on the Issues tab and we will look at it the same day.

# Actor input Schema

## `datasetId` (type: `string`):

An Apify dataset with one row per article link, for example the output of a Google News, RSS or search scraper. Every original column is kept and the article text, author, date and metadata are added alongside. Use the picker rather than typing an ID.

## `fileUrl` (type: `string`):

A public link to a CSV, TSV, Excel, JSON or JSON Lines file with one row per article link. A normal Google Sheets link works: share it as 'Anyone with the link can view'. Used when no dataset is given.

## `fileFormat` (type: `string`):

Leave on 'Detect automatically' unless the link has no file extension and the server reports the wrong content type.

## `sheetName` (type: `string`):

Which sheet to read from an Excel workbook. Defaults to the first sheet.

## `articleUrls` (type: `array`):

A plain list of article links for a quick run with no spreadsheet. Use the dataset, file or Google Sheet inputs above to keep your own columns next to the results.

## `data` (type: `array`):

Rows as inline JSON, an alternative to a dataset or file. Each object needs a field holding the article link.

## `urlField` (type: `string`):

The column holding the article link. Left empty it is detected automatically, preferring columns named url, link or articleUrl that hold links with a real path over a column of publisher homepages.

## `textFormat` (type: `string`):

'Plain text' returns the story with paragraph breaks. 'Markdown' adds a markdown column with headings, lists and links kept, the friendliest shape for an LLM or RAG pipeline. 'HTML' adds the cleaned article HTML. 'None' returns metadata only (title, author, date, image, word count).

## `minWords` (type: `integer`):

Pages with less article text than this are marked 'too_short' and not charged: video pages, galleries, paywall stubs and app shells. Lower it for very short news briefs.

## `maxCharsPerArticle` (type: `integer`):

Cuts the returned text at this many characters so one enormous page cannot produce an unusable row. The row records whether it was cut.

## `keep` (type: `string`):

Filtering happens after each link is read. 'Only problems' is the quick way to find dead or blocked links in a large list.

## `keepOriginalFields` (type: `boolean`):

Keep every column from the input row next to the extracted fields, so results line up with your own data. Turn off for article fields only.

## `onlyNewUrls` (type: `boolean`):

Remember every link extracted under the name below and skip it on later runs, not charged and not written to the results again (the summary counts them). Made for daily news monitoring: feed the same growing scraper output in every day and pay only for new stories.

## `stateName` (type: `string`):

Separate monitors keep separate memories. Use one name per feed or project, for example 'competitor-blogs'. Letters, numbers and dashes.

## `respectRobotsTxt` (type: `boolean`):

Skip pages a site's robots.txt asks automated tools not to read (marked 'robots_disallowed', not charged). Rules aimed only at other bots, such as AI crawlers, do not apply. Turn off only for sites you have permission to read.

## `proxyConfiguration` (type: `object`):

Off by default and usually not needed. Some sites block by address or by country; Apify residential proxy can help there and adds $0.01 per article that comes back. It does not get past blocks that check the browser itself.

## `concurrency` (type: `integer`):

How many links to read in parallel. Never more than two at a time go to any one site, whatever this is set to.

## `requestTimeoutSecs` (type: `integer`):

How long to wait for one page before giving up. A timed-out page is never charged.

## `maxPageMb` (type: `integer`):

Pages bigger than this are skipped and not charged.

## `maxItems` (type: `integer`):

A hard ceiling on how many rows are read from the input, as a safety net on a large dataset.

## `exportFormats` (type: `array`):

Also write the results as a downloadable CSV or Excel file, linked from the run's output. $0.01 per file, charged only when the run extracted at least one article. Excel cells are cut at 32,000 characters; the dataset and CSV keep the full text.

## `outputDatasetName` (type: `string`):

Also append every kept row to a named dataset that persists across runs, building one growing article archive. Not charged again.

## `webhookUrl` (type: `string`):

POST the run summary to this URL when the run finishes, for Slack, Zapier, Make, n8n or your own API. Charged only on a confirmed 2xx response.

## Actor input object example

```json
{
  "fileFormat": "auto",
  "articleUrls": [
    "https://en.wikipedia.org/wiki/Web_scraping",
    "https://www.paulgraham.com/greatwork.html",
    "https://techcrunch.com/2026/10/17/embers-new-temperature-controlled-mug-makes-my-nightly-tea-ritual-even-better/"
  ],
  "textFormat": "plain",
  "minWords": 50,
  "maxCharsPerArticle": 100000,
  "keep": "all",
  "keepOriginalFields": true,
  "onlyNewUrls": false,
  "stateName": "default",
  "respectRobotsTxt": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "concurrency": 5,
  "requestTimeoutSecs": 30,
  "maxPageMb": 10,
  "exportFormats": []
}
```

# Actor output Schema

## `results` (type: `string`):

Every original row with the article's full text, title, author, dates, publisher, image, tags and status added.

## `articleSummary` (type: `string`):

Status counts, articles extracted and not charged, word totals, export links and warnings.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "articleUrls": [
        "https://en.wikipedia.org/wiki/Web_scraping",
        "https://www.paulgraham.com/greatwork.html",
        "https://techcrunch.com/2026/10/17/embers-new-temperature-controlled-mug-makes-my-nightly-tea-ritual-even-better/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("nerolabs/bulk-article-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "articleUrls": [
        "https://en.wikipedia.org/wiki/Web_scraping",
        "https://www.paulgraham.com/greatwork.html",
        "https://techcrunch.com/2026/10/17/embers-new-temperature-controlled-mug-makes-my-nightly-tea-ritual-even-better/",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("nerolabs/bulk-article-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "articleUrls": [
    "https://en.wikipedia.org/wiki/Web_scraping",
    "https://www.paulgraham.com/greatwork.html",
    "https://techcrunch.com/2026/10/17/embers-new-temperature-controlled-mug-makes-my-nightly-tea-ritual-even-better/"
  ]
}' |
apify call nerolabs/bulk-article-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nerolabs/bulk-article-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/oXv5RwFHkYQVMFSPW/builds/0mONpdaZ5mD5AvY5k/openapi.json