ESPN News Scraper

ESPN News Scraper reads public ESPN league news feeds and writes normalized article records to an Apify dataset. It uses direct HTTPS requests to ESPN's public site API and, when requested, one bounded request to each accessible non-premium ESPN article page. It does not bypass logins, paywalls, CAPTCHAs, geographic restrictions, device checks, WAFs, or other access controls.

The default public feed is the ESPN NBA news API. The Actor supports 16 public league codes: nba, nfl, mlb, nhl, mls, wnba, ncaam, ncaaw, ncaaf, epl, laliga, ucl, bundesliga, seriea, ligue1, and ligamx.

Input

• leagues (required array): one or more supported league codes.
• teams (optional array): public team abbreviations or ESPN numeric IDs. At most 25 are accepted.
• fetchFullBody (optional boolean): include cleaned visible body text only when a public, non-premium article page is accessible.
• maxItems (optional integer): maximum article records across all requested feeds, from 1 to 10,000. Each feed request is capped at 100 articles.

Unknown fields, invalid league codes, malformed filters, and out-of-range limits are rejected. No proxy, login, email, private endpoint, or browser-bypass input is accepted.

Dataset contract

Normal rows contain business article data and source provenance only:

{
  "recordType": "espn-news-article",
  "sourceName": "ESPN public site API",
  "sourceKind": "public-espn-api",
  "sourceUrl": "https://site.api.espn.com/apis/site/v2/sports/basketball/nba/news?limit=10",
  "articleId": "49854792",
  "league": "nba",
  "sport": "basketball",
  "articleType": "article",
  "headline": "Public article headline",
  "description": "Public article summary when supplied",
  "published": "2026-09-07T10:45:59.000Z",
  "url": "https://www.espn.com/nba/story/_/id/49854792/example",
  "premium": false,
  "scrapedAt": "2026-09-08T00:00:00.000Z"
}

Optional business fields include lastModified, byline, allowComments, selected images, categories, keywords, videoCount, public team fields, inputTeam, and bodyText.

Operational issue details are written to the run logs. No diagnostic records, errors, samples, or placeholders are returned in the dataset or key-value store. A run that saved article records but encountered a later feed, body, or optional-output issue has status: "SUCCEEDED" and dataStatus: "PARTIAL" in its run summary.

Run artifacts

OUTPUT_SUMMARY and its compatibility alias OUTPUT contain run-level counts and status. SOURCE_METADATA records the public source boundary and whether the run used a local fixture. Issue details remain in the run logs; the summary contains only numeric diagnostic counts.

Run status and runStatus are SUCCEEDED or FAILED. dataStatus is COMPLETE, PARTIAL, NO_RESULTS, or FAILED. A useful partial result remains SUCCEEDED with dataStatus: "PARTIAL"; a clean empty feed is SUCCEEDED with NO_RESULTS; a run with an actual failure and no saved article record is FAILED with dataStatus: "FAILED". itemCount counts saved article rows, diagnosticCount counts operational issues, and diagnosticItems is always zero. Other counts include duplicate articles removed, attempted feeds, and body pages collected.

Local verification

npm ci
npm test
npm run lint
npm run schema
npm run validate

For deterministic QA, run the Actor from a disposable copy with input such as:

{
  "leagues": ["nba"],
  "teams": ["FBK"],
  "fetchFullBody": true,
  "maxItems": 2
}

For local QA, the contract tests load an ESPN-shaped fixture under the Actor directory; fixture output is not live evidence. A public run uses the ESPN feed and reports its observed source URL. If ESPN returns an access-control response, the Actor logs the issue and does not invent replacement content.

Cost and responsible use

Metadata-only runs use one sequential public feed request per league/filter. Body mode adds one bounded article-page request per selected article. Start with a small limit and respect ESPN's terms, robots/access signals, applicable law, and the rights of people represented in public content.