# AP News Articles Scraper (`automation-lab/ap-news-articles-scraper`) Actor

Search AP News and export complete public articles with headlines, URLs, bylines, dates, images, sections, summaries, keywords, and full body text.

- **URL**: https://apify.com/automation-lab/ap-news-articles-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** News
- **Stats:** 2 total users, 1 monthly users, 60.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## AP News Articles Scraper

Search AP News and export complete public article records for recurring topic research, media monitoring, and downstream analysis.

**AP News Articles Scraper** turns AP News search results, topic hubs, and direct article URLs into structured data. Each saved record includes the headline, canonical URL, summary, author byline, publication and modification dates, image, sections, keywords, and full article body.

Use it when you need repeatable **AP news articles** data instead of manually opening and copying stories.

### What does AP News Articles Scraper do?

The Actor supports three discovery routes:

1. Search AP News with one or more keyword queries.
2. Discover stories from public AP News topic or hub URLs.
3. Extract a supplied public AP News article URL directly.

For every discovered article, it opens the article page and reads public structured metadata plus the rendered server-side article body.

Results are deduplicated by canonical article URL and saved to the run's default dataset.

### Who is this AP News scraper for?

- **Media monitoring teams** tracking coverage of organizations, policies, people, or events.
- **Researchers** assembling article corpora with dates, sections, authors, and full text.
- **Public relations teams** reviewing recurring topic coverage.
- **Journalists and editors** finding AP coverage across queries and sections.
- **Data analysts** loading structured news records into spreadsheets, warehouses, or notebooks.
- **AI teams** preparing attributable public text for permitted search, classification, or summarization workflows.

The Actor extracts source records. It does not determine whether AP News is biased, score sentiment, or make factuality judgments.

### Why use this Actor?

- Search and article-detail extraction are combined in one run.
- Full body text is included with useful publication metadata.
- Multiple queries can be monitored in one scheduled task.
- Query results are interleaved so a small limit can represent multiple tracked topics.
- Direct article URLs provide a simple route for known stories.
- Date filters apply consistently to both discovered and supplied articles.
- No login, browser, or residential proxy is required for the current public route.
- Typed dataset output works with Apify integrations and APIs.

### What AP News article data is extracted?

| Field | Type | Description |
| --- | --- | --- |
| `articleId` | string | Stable identifier taken from the canonical article URL |
| `url` | string | Canonical AP News article URL |
| `headline` | string | Published article headline |
| `summary` | string or null | Public AP News description or summary |
| `authors` | array | Credited author names and public profile URLs |
| `byline` | string or null | Comma-separated author display names |
| `publishedAt` | string or null | ISO 8601 publication time |
| `modifiedAt` | string or null | ISO 8601 modification time |
| `imageUrl` | string or null | Primary article image URL |
| `sections` | string\[] | AP News section labels |
| `keywords` | string\[] | Keywords exposed in article metadata |
| `body` | string | Full public article text with paragraph breaks |
| `query` | string or null | Search query that discovered the article |
| `discoveryUrl` | string | Search, hub, or direct URL used for discovery |
| `fetchedAt` | string | Time the Actor fetched the article |

Some AP stories have no credited individual author, image, summary, or section. Those fields can be null or empty while the headline and body remain useful.

### How to scrape AP News articles

1. Open the Actor in Apify Console.
2. Enter one or more terms under **Search queries**.
3. Optionally add AP News search, hub, or article URLs.
4. Choose the maximum number of articles.
5. Adjust the search-page limit if you need deeper discovery.
6. Optionally add inclusive publication-date boundaries.
7. Click **Start**.
8. Open the **Dataset** tab to preview, download, or integrate the records.

Start with a small `maxItems` value while refining a query. Increase it only after confirming that the results match your monitoring scope.

### Input parameters

#### `queries`

An array of AP News keyword searches.

```json
{
  "queries": ["climate change", "renewable energy"]
}
```

Queries are sent to AP News separately. Results are deduplicated and interleaved during article extraction.

#### `startUrls`

An array of public HTTPS URLs on `apnews.com`.

Supported URL types include:

- Article: `https://apnews.com/article/...`
- Search: `https://apnews.com/search?q=...`
- Topic or hub: `https://apnews.com/hub/...`

Other domains and non-HTTPS URLs fail validation.

#### `maxItems`

Maximum complete article records to save.

- Default: `20`
- Minimum: `1`
- Maximum: `1000`

The Actor stops saving as soon as this accepted-result limit is reached.

#### `maxSearchPages`

Maximum result pages to inspect for each paginated search input.

- Default: `3`
- Minimum: `1`
- Maximum: `20`

AP News pages may contain duplicate links or fewer usable articles than their visible card count.

#### `publishedAfter` and `publishedBefore`

Optional inclusive ISO 8601 date or date-time filters.

```json
{
  "publishedAfter": "2026-01-01",
  "publishedBefore": "2026-12-31T23:59:59Z"
}
```

The same boundaries are applied to articles found through queries, hubs, search URLs, and direct article URLs.

### Input examples

Search one topic:

```json
{
  "queries": ["climate change"],
  "startUrls": [],
  "maxItems": 5,
  "maxSearchPages": 1
}
```

Extract a topic hub:

```json
{
  "queries": [],
  "startUrls": [
    { "url": "https://apnews.com/hub/climate-and-environment" }
  ],
  "maxItems": 5,
  "maxSearchPages": 1
}
```

Build a multi-topic monitoring feed:

```json
{
  "queries": ["climate change", "renewable energy"],
  "startUrls": [],
  "maxItems": 10,
  "maxSearchPages": 2
}
```

### Output example

A current result has this shape:

```json
{
  "articleId": "0123456789abcdef0123456789abcdef",
  "url": "https://apnews.com/article/sample-topic-0123456789abcdef0123456789abcdef",
  "headline": "Officials announce a new environmental initiative",
  "summary": "The initiative sets out new targets and a timeline for implementation.",
  "authors": [
    {
      "name": "Sample Reporter",
      "url": "https://apnews.com/author/sample-reporter"
    }
  ],
  "byline": "Sample Reporter",
  "publishedAt": "2026-01-15T12:00:00Z",
  "modifiedAt": "2026-01-15T13:30:00Z",
  "imageUrl": "https://dims.apnews.com/dims4/default/sample/image.jpg",
  "sections": ["Climate"],
  "keywords": ["environment", "policy"],
  "body": "The initiative was announced on Wednesday.\n\nOfficials described the next steps.",
  "query": "climate policy",
  "discoveryUrl": "https://apnews.com/search?q=climate%20policy",
  "fetchedAt": "2026-01-15T14:00:00.000Z"
}
```

The example is anonymized, but its keys and value types match current Actor output.

### How much does it cost to extract AP News articles?

The Actor uses pay-per-event pricing:

- A **$0.005 run-start fee** is charged once per run.
- Each complete saved article emits one **Item extracted** event.
- The current BRONZE article price is **$0.009732 per article**, with lower unit prices on higher platform tiers.
- Failed, duplicate, rejected, or date-filtered records do not emit an article event.

At the current BRONZE price, a run that saves 10 complete articles emits one **$0.005 start event** and 10 **$0.009732 article events**. A 100-article run emits the same one-time start event and 100 article events. Multiply the active item price by the number of saved records, then add the one-time start event.

Higher account tiers use the lower per-item values shown in the live pricing panel. Always check that panel for your account tier before starting a run.

### Scheduling recurring AP News monitoring

Create an Apify Task with stable queries and schedule it hourly, daily, or weekly.

A practical workflow is:

1. Run the Task on a schedule.
2. Send each dataset to a webhook or storage integration.
3. Deduplicate downstream records by `articleId` or `url`.
4. Compare `modifiedAt` to identify updates.
5. Route new records to a dashboard, alert, or analysis job.

The Actor does not maintain a cross-run seen-item database. Each run reports the public records available for its input at that time.

### Export AP News data

The default dataset can be downloaded as:

- JSON
- CSV
- Excel
- XML
- RSS
- JSONL

For spreadsheet work, export the overview view. For NLP or archival processing, use JSON or JSONL so arrays and full body text retain their structure.

### API usage with cURL

Start the Actor synchronously and return dataset items:

```bash
curl "https://api.apify.com/v2/acts/automation-lab~ap-news-articles-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["renewable energy"],
    "maxItems": 10,
    "maxSearchPages": 1
  }'
```

Keep your Apify token in an environment variable rather than source code.

### API usage with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('automation-lab/ap-news-articles-scraper').call({
  queries: ['renewable energy'],
  maxItems: 10,
  maxSearchPages: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map(({ headline, url, publishedAt }) => ({ headline, url, publishedAt })));
```

### API usage with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])

run = client.actor('automation-lab/ap-news-articles-scraper').call(run_input={
    'queries': ['renewable energy'],
    'maxItems': 10,
    'maxSearchPages': 1,
})

for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item['headline'], item['url'])
```

### Use AP News Articles Scraper with MCP

Add this Actor to Claude Code through Apify MCP:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/ap-news-articles-scraper"
```

#### Claude Desktop setup

Claude Desktop can use this MCP JSON configuration. Add the same server entry to its MCP settings:

#### Cursor and VS Code setup

Cursor and VS Code can use the equivalent MCP server URL in their respective MCP settings:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/ap-news-articles-scraper"
    }
  }
}
```

Example prompts:

- "Search AP News for renewable energy and return five complete recent articles."
- "Extract the latest articles from the AP Climate and Environment hub."
- "Create a structured AP News dataset for climate change and clean energy coverage."

### Integration ideas

#### Google Sheets and Airtable

Send each run's dataset to a table for editorial review, source tracking, or lightweight monitoring.

#### Webhooks and automation platforms

Trigger Zapier, Make, or a custom webhook when a scheduled run completes. Filter downstream on query, section, or publication date.

#### Data warehouses

Load JSONL into BigQuery, Snowflake, or another warehouse. Use the stable `articleId` as a natural deduplication key.

#### Search and retrieval

Index `headline`, `summary`, and `body` in a permitted internal search system. Retain `url` and publication metadata for attribution.

#### NLP analysis

Apply your own classification, entity extraction, topic modeling, or sentiment methods. The Actor supplies source text and metadata but does not make analytical claims.

### Tips for reliable results

- Use focused phrases instead of extremely broad single words.
- Start with one result page and inspect relevance before increasing depth.
- Use multiple queries when monitoring related but distinct terminology.
- Keep `maxItems` above the number of queries so interleaving can represent each query.
- Use direct article URLs when you already know the stories required.
- Store canonical `url` or `articleId` values to deduplicate scheduled runs.
- Use date filters for bounded research windows, not as a guarantee that AP search sorts every result chronologically.

### Limits and failure behavior

- Only public HTTPS URLs on `apnews.com` are accepted.
- The Actor does not bypass logins, paywalls, CAPTCHAs, or private newsroom systems.
- AP News may change its page structure or result ordering.
- Some articles do not expose an individual author or every optional metadata field.
- Deleted or unavailable articles cannot be extracted.
- Pagination depth is bounded by `maxSearchPages`.
- Article output is bounded by `maxItems`.
- Transient rate limits and server errors receive a small number of retries with backoff.
- Invalid input fails with a non-zero run status and an actionable message.
- A legitimate search with no current matches finishes successfully with an empty dataset.
- If AP returns article candidates but none can be parsed, the run fails instead of silently reporting success.

### Responsible use and legality

AP News content may be protected by copyright and other rights. You are responsible for your use of the data and for complying with applicable laws, contracts, source terms, and platform policies.

Good practices include:

- Collect only public information needed for a legitimate purpose.
- Respect attribution and retain canonical source URLs.
- Avoid republishing full copyrighted articles without permission.
- Use conservative limits and schedules.
- Do not use the Actor to harass people, spread misinformation, or create deceptive content.
- Review legal requirements for archival, AI training, redistribution, and commercial reuse in your jurisdiction.

This Actor is an independent extraction tool and is not affiliated with or endorsed by The Associated Press.

### Troubleshooting

#### Why is my dataset empty?

Check the query directly on AP News and remove restrictive date filters. A valid query can naturally have no current matches.

#### Why did a supplied URL fail validation?

The Actor accepts only public HTTPS URLs whose hostname is `apnews.com` or `www.apnews.com`. Redirector, AMP mirror, and third-party news URLs are not accepted.

#### Why are some bylines null?

Some AP pages credit no individual author in their public structured metadata. The Actor preserves that absence rather than inventing a name.

#### Why did I receive fewer records than `maxItems`?

`maxItems` is a ceiling, not a promise. AP may expose fewer unique article links, date filters may exclude records, and unavailable article pages are skipped.

### Frequently asked questions

#### Can I scrape a single AP News article?

Yes. Add its canonical `https://apnews.com/article/...` URL to `startUrls` and leave `queries` empty.

#### Can I search several topics in one run?

Yes. Supply multiple `queries`. Extraction interleaves their candidates to avoid letting the first query consume a small result limit.

#### Does the Actor extract full article text?

Yes, when the full public story body is present on the AP News article page.

#### Does it assess AP News bias?

No. It extracts article records that researchers may use in their own methodology. It does not provide bias, sentiment, or factuality scores.

#### Does it require a proxy?

No proxy is used by the current implementation. It relies on public server-rendered AP News pages.

#### Can I schedule it?

Yes. Save the input as an Apify Task and add a schedule. Deduplicate records downstream by `articleId` or `url`.

#### Can I filter by exact publication dates?

Use `publishedAfter` and `publishedBefore`. The boundaries are inclusive and accept ISO 8601 dates or date-times.

### Related automation-lab Actors

- [GDELT Global News Search Scraper](https://apify.com/automation-lab/gdelt-global-news-search-scraper) for multi-source global news discovery through GDELT.
- [Naver News Search Scraper](https://apify.com/automation-lab/naver-news-search-scraper) for structured Korean-market news search results.
- [Finviz Stock Quote & News Scraper](https://apify.com/automation-lab/finviz-stock-quote-news-scraper) for ticker-focused market headlines and quote context.

Choose this Actor when AP News is the required source and complete public article details are the primary output.

### Support

If a run fails, include the run URL, sanitized input, and the affected AP News URL when reporting the issue. Do not post your API token or private integration credentials.

# Actor input Schema

## `queries` (type: `array`):

Keywords or topics to search on AP News, for example climate change or artificial intelligence.

## `startUrls` (type: `array`):

Optional AP News article, search, or topic/hub URLs. Direct article URLs are extracted immediately; listing URLs are used for discovery.

## `maxItems` (type: `integer`):

Maximum number of complete article records to save.

## `maxSearchPages` (type: `integer`):

Maximum AP News result pages to inspect for each query or paginated listing URL.

## `publishedAfter` (type: `string`):

Optional inclusive ISO 8601 date or date-time filter applied to every discovered or supplied article.

## `publishedBefore` (type: `string`):

Optional inclusive ISO 8601 date or date-time filter applied to every discovered or supplied article.

## Actor input object example

```json
{
  "queries": [
    "climate change"
  ],
  "startUrls": [],
  "maxItems": 5,
  "maxSearchPages": 1
}
```

# Actor output Schema

## `overview` (type: `string`):

Default dataset containing complete AP News article records.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "climate change"
    ],
    "startUrls": [],
    "maxItems": 5,
    "maxSearchPages": 1
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/ap-news-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["climate change"],
    "startUrls": [],
    "maxItems": 5,
    "maxSearchPages": 1,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/ap-news-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "climate change"
  ],
  "startUrls": [],
  "maxItems": 5,
  "maxSearchPages": 1
}' |
apify call automation-lab/ap-news-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/ap-news-articles-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/fUguC7ZBGdlcaIwMT/builds/VGaAsxb3YPdOwUmkL/openapi.json