# Fox News Articles Scraper (`scrapyx/foxnews-articles-scraper`) Actor

Collects Fox News articles from 38 sections with headline, full body text, author, keywords, word count, section and lead image. Reads the publisher feeds for instant full text and the archive sitemaps for 1.6M URLs back to 2014, with date and keyword filtering.

- **URL**: https://apify.com/scrapyx/foxnews-articles-scraper.md
- **Developed by:** [Ibnu Adzim](https://apify.com/scrapyx) (community)
- **Categories:** News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.10 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Fox News Articles Scraper

Collects articles from **foxnews.com** across all 38 of its sections, with the
headline, the complete body text, the author, keywords, word count, section,
publish and modified dates, and the lead image.

Two public surfaces are used, and every row says which one produced it:

- **Publisher feeds** — 13 sections have a Fox News syndication feed that
  already contains the complete article text. Those sections cost **one
  request for 25 full articles**.
- **Sitemaps** — the other 25 sections, deep archive runs and date-filtered
  runs read Fox's own sitemaps and then each article's page. The archive holds
  roughly **1.67 million article URLs going back to November 2014**.

No login, no API key, no browser. Plain HTTP, Python only.

### What you get

One dataset, three kinds of row, told apart by `recordType`.

**`ARTICLE`** — one per article: URL, headline, description, full body text
(and the body's original HTML when it came from a feed), publish and modified
dates, authors, section list, keywords, word count, lead image, and whether
Fox marks it free to read.

**`SEARCH_SUMMARY`** — one per requested section. It reports which surface was
used, how many URLs were discovered, how many rows came back, how many article
pages were fetched, how many archive partitions were read or skipped, and how
many rows each filter dropped. If a number looks wrong, this row tells you why.

**`ERROR`** — one per input or URL that failed, so every request you make maps
to at least one row and a failure can never look like an empty result.

### Sections

`latest` means no section filter.

These 13 have a publisher feed, so they return 25 full articles for one
request: `latest`, `politics`, `world`, `us`, `opinion`, `entertainment`,
`sports`, `science`, `tech`, `health`, `lifestyle`, `travel`, `media`.

These 25 exist only in the sitemaps and need one page fetch per article:
`analysis`, `area-51`, `auto`, `culture`, `deals`, `faith-values`, `family`,
`food-drink`, `forum`, `fox-news`, `games`, `great-outdoors`, `midterms-2018`,
`official-polls`, `outkick`, `outkick-analysis`, `outkick-betting`,
`outkick-culture`, `outkick-shows`, `outkick-sports`, `real-estate`, `story`,
`transcript`, `true-crime`, `uncategorized`, `weather`.

A section outside that list is refused before any request is sent, rather than
quietly returning nothing.

### Examples

Twenty-five full politics articles, one request:

```json
{ "sections": ["politics"], "maxItemsPerSection": 25 }
```

Three sections at once, ten each:

```json
{ "sections": ["politics", "world", "tech"], "maxItemsPerSection": 10 }
```

A section with no feed — body comes from the article page:

```json
{ "sections": ["outkick-sports"], "maxItemsPerSection": 20 }
```

Everything published in July 2026 that mentions a term:

```json
{
  "sections": ["politics"],
  "dateFrom": "2026-07-01",
  "dateTo": "2026-07-31",
  "includeArchive": true,
  "maxArchivePages": 3,
  "keywords": ["tariff"],
  "maxItemsPerSection": 100
}
```

Fast headline sweep, no article pages fetched at all:

```json
{ "sections": ["latest"], "maxItemsPerSection": 150, "includeFullBody": false }
```

### Limits, stated plainly

**Feed rows carry no author.** Fox's feeds leave the author element empty on
every item of every feed. The feed gives you the headline, description, full
body, publish date, image and topic tags. Author, keywords, word count and
modified date exist only on the article page. Set `enrichFeedArticles` to true
to fetch those pages and fill the gap — it costs one request per article, which
is the whole reason it is off by default.

**The keyword filter runs here, not at Fox.** No public Fox surface offers a
working search parameter, so keywords narrow what a run already retrieved. To
search wider, widen the run with `includeArchive` and `maxArchivePages`.

**A feed holds exactly 25 articles.** Asking for more than 25 from a feed
section without `includeArchive` cannot return more than 25.

**Archive partitions are large.** Each is 10,000 URLs in a multi-megabyte file.
One is fetched only when the rolling 48-hour window cannot already satisfy your
request, and partitions that fall entirely outside a date range are skipped
without being downloaded. `maxArchivePages` is your cost control.

**Undated rows are dropped inside a date range.** If a date bound is set and an
article carries no usable date, it is excluded rather than included on a guess.

**Videos are a separate Actor.** Fox's video pages carry a VideoObject, not an
article body. They have their own scraper rather than being folded in here with
half the columns empty.

### Blocking and proxies

No bot-mitigation gate was found on any Fox News surface during development:
the feed host, the sitemaps and the article pages all answered cold across
four different TLS fingerprints, with no challenge, no cookie and no token.
The Actor therefore defaults to Apify's shared **datacenter** proxy, which is
included in your plan at no extra cost. Residential is available in the proxy
editor if you ever need it; Apify bills it per gigabyte.

The Actor rotates TLS fingerprints and retries with exponential backoff on
transport errors, and if a proxy group your account cannot use is requested, it
falls back to a direct connection instead of failing the run.

### Politeness

`robots.txt` at foxnews.com sets no `Crawl-delay` and names no AI-crawler
group. It disallows the site search, the wires and the printer-friendly paths,
none of which this Actor touches: it reads the publisher feeds, the sitemaps
that `robots.txt` itself advertises, and public article pages. Request starts
are paced by `minRequestInterval`, which defaults to a quarter of a second.

# Actor input Schema

## `sections` (type: `array`):

Which Fox News sections to pull, each returned with its own summary row. 'latest' means no section filter: Fox's own cross-section feed, or the unfiltered sitemap when an archive or date range is requested. Sections marked 'RSS feed + sitemap' have a publisher feed that already contains the full article text, so they cost one request; the other 25 exist only in the sitemaps and need one page fetch per article. Unknown values are refused before any request rather than silently returning nothing.

## `maxItemsPerSection` (type: `integer`):

Cap on articles returned for each section, applied after the date, section and keyword filters. The RSS surface holds exactly 25 articles per section, so asking for more than 25 without turning on 'Include archive' cannot return more than 25. Sections are returned newest first, so raising this reaches further back in time.

## `includeFullBody` (type: `boolean`):

For sitemap-sourced articles, fetch each article's own page and read its JSON-LD for the complete body text, author, keywords, word count and section. Turn this off for a fast headline-and-URL run. Articles coming from an RSS feed always include their full body at no extra cost, so this setting does not affect them.

## `includeArchive` (type: `boolean`):

Walk Fox's paginated archive sitemaps as well as the rolling ~48-hour news sitemap. The archive holds roughly 1.67 million article URLs in 167 partitions of 10,000, newest first, reaching back to November 2014. A partition is a multi-megabyte download, so one is only fetched when the fresh window cannot already fill your item cap or date range; the summary row reports exactly how many were fetched and how many were skipped. Turning this on switches every requested section to the sitemap surface.

## `maxArchivePages` (type: `integer`):

How many 10,000-URL archive partitions to download per section when 'Include archive' is on, newest first. Each partition is a multi-megabyte XML file, so this is the real cost control for deep runs. Partitions that fall entirely before 'Published from' are skipped without being downloaded.

## `dateFrom` (type: `string`):

Optional lower bound on the publish date, inclusive, as YYYY-MM-DD. Setting it switches every section to the sitemap surface, because the RSS feeds carry only the newest 25 articles and cannot answer a date range. Articles with no usable date are excluded whenever a bound is set.

## `dateTo` (type: `string`):

Optional upper bound on the publish date, inclusive, as YYYY-MM-DD. Same surface switch as 'Published from'.

## `keywords` (type: `array`):

Optional. Keep only articles whose headline, description or body text contains one of these words, matched case-insensitively. This is filtering done by this Actor after fetching, not a search sent to Fox News: none of Fox's public surfaces offers a working search parameter, so the filter narrows what a run already retrieved rather than widening what it can reach.

## `maxConcurrency` (type: `integer`):

Upper bound on requests in flight at once across every section in this run.

## `minRequestInterval` (type: `number`):

Shared pacing floor across every request this run makes. Fox News sets no Crawl-delay in robots.txt, so this default is this portfolio's own politeness baseline rather than a site requirement.

## `proxyConfiguration` (type: `object`):

Apify Proxy on the shared datacenter pool. That is the default because it is included in your plan at no extra cost and this target needs nothing more: no bot-mitigation gate was seen on any Fox News surface during testing. If you ever see blocks or empty results, switch the group here to Residential, which Apify bills per gigabyte.

## Actor input object example

```json
{
  "sections": [
    "latest"
  ],
  "maxItemsPerSection": 25,
  "includeFullBody": true,
  "includeArchive": false,
  "maxArchivePages": 1,
  "keywords": [],
  "maxConcurrency": 5,
  "minRequestInterval": 0.25,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `items` (type: `string`):

One row per scraped record. See the dataset's default view for field definitions.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapyx/foxnews-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("scrapyx/foxnews-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call scrapyx/foxnews-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrapyx/foxnews-articles-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/zSqmf47tfrqZV9TYw/builds/fMlcoPMAIehwQvxP7/openapi.json