AP News Articles Scraper

Search AP News and export complete public article records for recurring topic research, media monitoring, and downstream analysis.

AP News Articles Scraper turns AP News search results, topic hubs, and direct article URLs into structured data. Each saved record includes the headline, canonical URL, summary, author byline, publication and modification dates, image, sections, keywords, and full article body.

Use it when you need repeatable AP news articles data instead of manually opening and copying stories.

What does AP News Articles Scraper do?

The Actor supports three discovery routes:

1. Search AP News with one or more keyword queries.
2. Discover stories from public AP News topic or hub URLs.
3. Extract a supplied public AP News article URL directly.

For every discovered article, it opens the article page and reads public structured metadata plus the rendered server-side article body.

Results are deduplicated by canonical article URL and saved to the run's default dataset.

Who is this AP News scraper for?

• Media monitoring teams tracking coverage of organizations, policies, people, or events.
• Researchers assembling article corpora with dates, sections, authors, and full text.
• Public relations teams reviewing recurring topic coverage.
• Journalists and editors finding AP coverage across queries and sections.
• Data analysts loading structured news records into spreadsheets, warehouses, or notebooks.
• AI teams preparing attributable public text for permitted search, classification, or summarization workflows.

The Actor extracts source records. It does not determine whether AP News is biased, score sentiment, or make factuality judgments.

Why use this Actor?

• Search and article-detail extraction are combined in one run.
• Full body text is included with useful publication metadata.
• Multiple queries can be monitored in one scheduled task.
• Query results are interleaved so a small limit can represent multiple tracked topics.
• Direct article URLs provide a simple route for known stories.
• Date filters apply consistently to both discovered and supplied articles.
• No login, browser, or residential proxy is required for the current public route.
• Typed dataset output works with Apify integrations and APIs.

What AP News article data is extracted?

Field	Type	Description
articleId	string	Stable identifier taken from the canonical article URL
url	string	Canonical AP News article URL
headline	string	Published article headline
summary	string or null	Public AP News description or summary
authors	array	Credited author names and public profile URLs
byline	string or null	Comma-separated author display names
publishedAt	string or null	ISO 8601 publication time
modifiedAt	string or null	ISO 8601 modification time
imageUrl	string or null	Primary article image URL
sections	string[]	AP News section labels
keywords	string[]	Keywords exposed in article metadata
body	string	Full public article text with paragraph breaks
query	string or null	Search query that discovered the article
discoveryUrl	string	Search, hub, or direct URL used for discovery
fetchedAt	string	Time the Actor fetched the article

Some AP stories have no credited individual author, image, summary, or section. Those fields can be null or empty while the headline and body remain useful.

How to scrape AP News articles

1. Open the Actor in Apify Console.
2. Enter one or more terms under Search queries.
3. Optionally add AP News search, hub, or article URLs.
4. Choose the maximum number of articles.
5. Adjust the search-page limit if you need deeper discovery.
6. Optionally add inclusive publication-date boundaries.
7. Click Start.
8. Open the Dataset tab to preview, download, or integrate the records.

Start with a small maxItems value while refining a query. Increase it only after confirming that the results match your monitoring scope.

Input parameters

queries

An array of AP News keyword searches.

{
  "queries": ["climate change", "renewable energy"]
}

Queries are sent to AP News separately. Results are deduplicated and interleaved during article extraction.

startUrls

An array of public HTTPS URLs on apnews.com.

Supported URL types include:

• Article: https://apnews.com/article/...
• Search: https://apnews.com/search?q=...
• Topic or hub: https://apnews.com/hub/...

Other domains and non-HTTPS URLs fail validation.

maxItems

Maximum complete article records to save.

• Default: 20
• Minimum: 1
• Maximum: 1000

The Actor stops saving as soon as this accepted-result limit is reached.

maxSearchPages

Maximum result pages to inspect for each paginated search input.

• Default: 3
• Minimum: 1
• Maximum: 20

AP News pages may contain duplicate links or fewer usable articles than their visible card count.

publishedAfter and publishedBefore

Optional inclusive ISO 8601 date or date-time filters.

{
  "publishedAfter": "2026-01-01",
  "publishedBefore": "2026-12-31T23:59:59Z"
}

The same boundaries are applied to articles found through queries, hubs, search URLs, and direct article URLs.

Input examples

Search one topic:

{
  "queries": ["climate change"],
  "startUrls": [],
  "maxItems": 5,
  "maxSearchPages": 1
}

Extract a topic hub:

{
  "queries": [],
  "startUrls": [
    { "url": "https://apnews.com/hub/climate-and-environment" }
  ],
  "maxItems": 5,
  "maxSearchPages": 1
}

Build a multi-topic monitoring feed:

{
  "queries": ["climate change", "renewable energy"],
  "startUrls": [],
  "maxItems": 10,
  "maxSearchPages": 2
}

Output example

A current result has this shape:

{
  "articleId": "0123456789abcdef0123456789abcdef",
  "url": "https://apnews.com/article/sample-topic-0123456789abcdef0123456789abcdef",
  "headline": "Officials announce a new environmental initiative",
  "summary": "The initiative sets out new targets and a timeline for implementation.",
  "authors": [
    {
      "name": "Sample Reporter",
      "url": "https://apnews.com/author/sample-reporter"
    }
  ],
  "byline": "Sample Reporter",
  "publishedAt": "2026-01-15T12:00:00Z",
  "modifiedAt": "2026-01-15T13:30:00Z",
  "imageUrl": "https://dims.apnews.com/dims4/default/sample/image.jpg",
  "sections": ["Climate"],
  "keywords": ["environment", "policy"],
  "body": "The initiative was announced on Wednesday.\n\nOfficials described the next steps.",
  "query": "climate policy",
  "discoveryUrl": "https://apnews.com/search?q=climate%20policy",
  "fetchedAt": "2026-01-15T14:00:00.000Z"
}

The example is anonymized, but its keys and value types match current Actor output.

How much does it cost to extract AP News articles?

The Actor uses pay-per-event pricing:

• A $0.005 run-start fee is charged once per run.
• Each complete saved article emits one Item extracted event.
• The current BRONZE article price is $0.009732 per article, with lower unit prices on higher platform tiers.
• Failed, duplicate, rejected, or date-filtered records do not emit an article event.

At the current BRONZE price, a run that saves 10 complete articles emits one $0.005 start event and 10 $0.009732 article events. A 100-article run emits the same one-time start event and 100 article events. Multiply the active item price by the number of saved records, then add the one-time start event.

Higher account tiers use the lower per-item values shown in the live pricing panel. Always check that panel for your account tier before starting a run.

Scheduling recurring AP News monitoring

Create an Apify Task with stable queries and schedule it hourly, daily, or weekly.

A practical workflow is:

1. Run the Task on a schedule.
2. Send each dataset to a webhook or storage integration.
3. Deduplicate downstream records by articleId or url.
4. Compare modifiedAt to identify updates.
5. Route new records to a dashboard, alert, or analysis job.

The Actor does not maintain a cross-run seen-item database. Each run reports the public records available for its input at that time.

Export AP News data

The default dataset can be downloaded as:

• JSON
• CSV
• Excel
• XML
• RSS
• JSONL

For spreadsheet work, export the overview view. For NLP or archival processing, use JSON or JSONL so arrays and full body text retain their structure.

API usage with cURL

Start the Actor synchronously and return dataset items:

curl "https://api.apify.com/v2/acts/automation-lab~ap-news-articles-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["renewable energy"],
    "maxItems": 10,
    "maxSearchPages": 1
  }'

Keep your Apify token in an environment variable rather than source code.

API usage with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('automation-lab/ap-news-articles-scraper').call({
  queries: ['renewable energy'],
  maxItems: 10,
  maxSearchPages: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map(({ headline, url, publishedAt }) => ({ headline, url, publishedAt })));

API usage with Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])

run = client.actor('automation-lab/ap-news-articles-scraper').call(run_input={
    'queries': ['renewable energy'],
    'maxItems': 10,
    'maxSearchPages': 1,
})

for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item['headline'], item['url'])

Use AP News Articles Scraper with MCP

Add this Actor to Claude Code through Apify MCP:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/ap-news-articles-scraper"

Claude Desktop setup

Claude Desktop can use this MCP JSON configuration. Add the same server entry to its MCP settings:

Cursor and VS Code setup

Cursor and VS Code can use the equivalent MCP server URL in their respective MCP settings:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/ap-news-articles-scraper"
    }
  }
}

Example prompts:

• "Search AP News for renewable energy and return five complete recent articles."
• "Extract the latest articles from the AP Climate and Environment hub."
• "Create a structured AP News dataset for climate change and clean energy coverage."

Integration ideas

Google Sheets and Airtable

Send each run's dataset to a table for editorial review, source tracking, or lightweight monitoring.

Webhooks and automation platforms

Trigger Zapier, Make, or a custom webhook when a scheduled run completes. Filter downstream on query, section, or publication date.

Data warehouses

Load JSONL into BigQuery, Snowflake, or another warehouse. Use the stable articleId as a natural deduplication key.

Search and retrieval

Index headline, summary, and body in a permitted internal search system. Retain url and publication metadata for attribution.

NLP analysis

Apply your own classification, entity extraction, topic modeling, or sentiment methods. The Actor supplies source text and metadata but does not make analytical claims.

Tips for reliable results

• Use focused phrases instead of extremely broad single words.
• Start with one result page and inspect relevance before increasing depth.
• Use multiple queries when monitoring related but distinct terminology.
• Keep maxItems above the number of queries so interleaving can represent each query.
• Use direct article URLs when you already know the stories required.
• Store canonical url or articleId values to deduplicate scheduled runs.
• Use date filters for bounded research windows, not as a guarantee that AP search sorts every result chronologically.

Limits and failure behavior

• Only public HTTPS URLs on apnews.com are accepted.
• The Actor does not bypass logins, paywalls, CAPTCHAs, or private newsroom systems.
• AP News may change its page structure or result ordering.
• Some articles do not expose an individual author or every optional metadata field.
• Deleted or unavailable articles cannot be extracted.
• Pagination depth is bounded by maxSearchPages.
• Article output is bounded by maxItems.
• Transient rate limits and server errors receive a small number of retries with backoff.
• Invalid input fails with a non-zero run status and an actionable message.
• A legitimate search with no current matches finishes successfully with an empty dataset.
• If AP returns article candidates but none can be parsed, the run fails instead of silently reporting success.

Responsible use and legality

AP News content may be protected by copyright and other rights. You are responsible for your use of the data and for complying with applicable laws, contracts, source terms, and platform policies.

Good practices include:

• Collect only public information needed for a legitimate purpose.
• Respect attribution and retain canonical source URLs.
• Avoid republishing full copyrighted articles without permission.
• Use conservative limits and schedules.
• Do not use the Actor to harass people, spread misinformation, or create deceptive content.
• Review legal requirements for archival, AI training, redistribution, and commercial reuse in your jurisdiction.

This Actor is an independent extraction tool and is not affiliated with or endorsed by The Associated Press.

Troubleshooting

Why is my dataset empty?

Check the query directly on AP News and remove restrictive date filters. A valid query can naturally have no current matches.

Why did a supplied URL fail validation?

The Actor accepts only public HTTPS URLs whose hostname is apnews.com or www.apnews.com. Redirector, AMP mirror, and third-party news URLs are not accepted.

Why are some bylines null?

Some AP pages credit no individual author in their public structured metadata. The Actor preserves that absence rather than inventing a name.

Why did I receive fewer records than maxItems?

maxItems is a ceiling, not a promise. AP may expose fewer unique article links, date filters may exclude records, and unavailable article pages are skipped.

Frequently asked questions

Can I scrape a single AP News article?

Yes. Add its canonical https://apnews.com/article/... URL to startUrls and leave queries empty.

Can I search several topics in one run?

Yes. Supply multiple queries. Extraction interleaves their candidates to avoid letting the first query consume a small result limit.

Does the Actor extract full article text?

Yes, when the full public story body is present on the AP News article page.

Does it assess AP News bias?

No. It extracts article records that researchers may use in their own methodology. It does not provide bias, sentiment, or factuality scores.

Does it require a proxy?

No proxy is used by the current implementation. It relies on public server-rendered AP News pages.

Can I schedule it?

Yes. Save the input as an Apify Task and add a schedule. Deduplicate records downstream by articleId or url.

Can I filter by exact publication dates?

Use publishedAfter and publishedBefore. The boundaries are inclusive and accept ISO 8601 dates or date-times.

Related automation-lab Actors

• GDELT Global News Search Scraper for multi-source global news discovery through GDELT.
• Naver News Search Scraper for structured Korean-market news search results.
• Finviz Stock Quote & News Scraper for ticker-focused market headlines and quote context.

Choose this Actor when AP News is the required source and complete public article details are the primary output.

Support

If a run fails, include the run URL, sanitized input, and the affected AP News URL when reporting the issue. Do not post your API token or private integration credentials.