# Sitemap News Article Extractor (`snappytools/smart-sitemap-article-extractor`) Actor

This Actor reads one or more public XML sitemaps, discovers article URLs (following nested sitemap indexes to a bounded depth), downloads each public article page and extracts clean, readable content using a lightweight Readability-style heuristic.

- **URL**: https://apify.com/snappytools/smart-sitemap-article-extractor.md
- **Developed by:** [Snappy](https://apify.com/snappytools) (community)
- **Categories:** News
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.10 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Sitemap News Article Extractor – article text and metadata from any XML sitemap

Give it one or more public XML sitemap links and it reads the article pages listed in them. You get back one row per article, with the title, clean plain text, word count, language, description and the sitemap it came from. Author, publish date and last-modified date are included when the page or sitemap provides them.

It follows nested sitemap indexes to a bounded depth, so you can start from a site's main `sitemap.xml`. You can narrow the pages with a URL filter such as `/blog/`. Schedule it from the Apify Console to re-check a site on a regular basis.

### Why this one

- **No browser, no login.** It makes plain HTTP requests to public pages, so runs are quick and predictable.
- **Clean text, not raw HTML.** It pulls out the readable article body using a Readability-style approach.
- **Stable IDs.** Each row has an `id` (a hash of the URL), so you can join and deduplicate across runs.
- **Filters that save money.** Use **URL substring filter** and **Minimum text length** to skip pages you don't want.
- **Sitemap indexes handled.** Nested sitemaps are followed for you.
- **Pay per result.** $2.10 per 1,000 results, with no start fee, and runs with no results cost nothing.

### Quick start

```json
{
  "sitemapUrls": ["https://www.apify.com/sitemap.xml"],
  "maxArticles": 5,
  "minTextLength": 200
}
```

In a test run on 27 Oct 2026 this returned 5 pages, each with title, text, word count and language.

### What you can use it for

| Who | What for |
| --- | --- |
| AI and data engineers | Feed clean article text into RAG pipelines and vector databases |
| Marketing and PR teams | Monitor what publishers and competitors put out |
| SEO specialists | Audit titles, descriptions and word counts across a site |
| Researchers | Build text datasets from public articles in one section of a site |
| Content teams | Make an inventory of your own or a partner's content |

### How to use it

1. Click **Try for free**.
2. Add one or more links under **Sitemap URLs**, for example `https://www.apify.com/sitemap.xml`. Use public https sitemaps (urlset or sitemapindex).
3. Set **Maximum articles** (1 to 500, default 20), for example `50`.
4. Optionally set **URL substring filter** to keep only matching URLs, for example `/blog/`.
5. Optionally set **Minimum text length** to skip thin pages (default 200 characters).
6. Click **Start**.

To keep a content feed fresh, set the run on a schedule and export the dataset each time.

### What data you get

Each row is one article page.

| Field | What it is | Example |
| --- | --- | --- |
| `id` | Stable hash of the URL | `f56239f8394443ac49195075d5fa66a7bd7218c6` |
| `url` | Article page address | `https://apify.com/actors` |
| `title` | Page title | `Automate anything with Apify Actors · Apify` |
| `author` | Author, when the page provides one | `null` |
| `publishedAt` | Publish date, when found | `null` |
| `lastModified` | Last-modified date from the sitemap, when given | `null` |
| `description` | Page description | `Actors are web data automations that power your AI and operations.` |
| `language` | Page language | `en` |
| `text` | Clean plain text of the article | `Actors are web data automations...` |
| `wordCount` | Words in the extracted text | `383` |
| `sitemapUrl` | Sitemap the link was found in | `https://apify.com/sitemap/pages.xml` |
| `scrapedAt` | When the page was fetched | `2026-10-27T09:20:08.866Z` |

### Pricing

| Event | Price |
| --- | --- |
| Result (each row written to the dataset) | $0.0021 |

| Run | Cost |
| --- | --- |
| 100 results | $0.21 |
| 1,000 results | $2.10 |
| 10,000 results | $21.00 |

There is no start fee. Runs with no results cost nothing.

### Good to know

- **Limits.** Up to 500 articles per run, 2 levels of sitemap nesting, 50 sitemap files and a 20-second timeout per request.
- **Data source.** Public sitemaps and public article pages fetched over HTTPS at run time.
- **No JavaScript rendering.** Content loaded only in the browser may be missing.
- **Heuristic extraction.** Author and date depend on common meta tags and JSON-LD, so they are often empty.
- **No access tricks.** It never logs in and does not bypass paywalls, CAPTCHAs or access controls.
- **Your use of the text.** Respect each publisher's terms and copyright when reusing content.
- **Not affiliated.** This tool isn't affiliated with or endorsed by the sites you extract from.

### FAQ

**Can I start from a sitemap index?** Yes. Nested sitemaps are followed to a bounded depth.

**How do I get only blog posts?** Set **URL substring filter** to `/blog/`.

**Why are some pages missing?** Pages with less text than **Minimum text length** are skipped. Lower it, or set it to `0`, to keep them.

### Getting the data out

Download results as Excel, CSV or JSON from the Output tab. Send them to Google Sheets, Slack, Zapier, Make or webhooks from the **Integrations** tab. Every run is also available through the Apify API, and you can schedule runs from **Schedules**.

### Questions or problems

Open an issue on the **Issues** tab and include the run link. I reply within one business day.

### Related tools

- [Public RSS Feed Monitor](https://apify.com/snappytools/rss-feed-monitor): watch RSS feeds for new items
- [Hacker News Search and Who-is-Hiring Extractor](https://apify.com/snappytools/hackernews-algolia-scraper): search Hacker News and pull who-is-hiring posts
- [Google News RSS Topic Scraper](https://apify.com/snappytools/google-news-rss-scraper): get Google News results for a topic

# Actor input Schema

## `sitemapUrls` (type: `array`):

Public https sitemap.xml URLs (urlset or sitemapindex) to read article links from.

## `maxArticles` (type: `integer`):

Upper bound of article pages fetched and pushed to the dataset (1-500).

## `urlFilter` (type: `string`):

Optional substring; only sitemap URLs containing it are extracted, e.g. /blog/.

## `minTextLength` (type: `integer`):

Skip pages whose extracted article text is shorter than this many characters.

## Actor input object example

```json
{
  "sitemapUrls": [
    "https://www.apify.com/sitemap.xml"
  ],
  "maxArticles": 5,
  "minTextLength": 200
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sitemapUrls": [
        "https://www.apify.com/sitemap.xml"
    ],
    "maxArticles": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("snappytools/smart-sitemap-article-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "sitemapUrls": ["https://www.apify.com/sitemap.xml"],
    "maxArticles": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("snappytools/smart-sitemap-article-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sitemapUrls": [
    "https://www.apify.com/sitemap.xml"
  ],
  "maxArticles": 5
}' |
apify call snappytools/smart-sitemap-article-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,snappytools/smart-sitemap-article-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/MqgXnlBTTJhOOqvJj/builds/0ZT9Y47Ms0wnu4YS2/openapi.json