New York Times Archive Index

Enumerate the New York Times archive by date — every article published on a given day, with URL, headline, date and section. The archive reaches back to 1970, so you can build a complete index of a month, a year or a decade.

Why Use This Actor?

• Complete daily coverage. Each archive day lists everything the Times published that day — roughly 130–290 articles, including pieces that never reach a section front or an RSS feed.
• Deep history. Verified working back to 1970: 1970-04-22 returned 245 articles, 1990-07-12 returned 294, 2000-01-03 returned 213.
• Headline included. You get the full headline in the index, so you can filter a corpus before deciding which articles are worth fetching.
• Section filtering. Pull only business, world, technology — parsed from the article URL, so it works on old articles too.
• Corpus building. Point it at a date range and get a clean, deduplicated URL list to feed a research pipeline or a downstream extractor.

What It's Good For

• Media research — how did coverage of a topic change across years?
• Corpus construction — build a dated, sectioned index before selective extraction.
• Archival monitoring — a reproducible daily list of what was published.
• Backfilling — fill gaps that recency-based feeds and section fronts can't reach.

Two Modes

Mode	What you get	Reliability
index (default)	URL, headline, published date, section, subsection	High — the archive pages answer plain HTTP consistently
articles	Everything above plus byline, timestamps, keywords and body text	Variable — see the honest note below

About articles mode

NYT article pages are much more tightly gated than the archive pages. During testing all 16 curl_cffi fingerprints tried were refused, and only one Firefox fingerprint got through — and even that stops working once an IP has made a burst of requests, after which that address receives sustained refusals.

So articles mode is real but rate-limited:

• Use a residential proxy and keep batches modest.
• When a body can't be fetched, the actor keeps the index row and sets _articleError — it never drops the article and never fabricates text.
• For large corpora, run index mode first (fast and reliable), then fetch bodies in controlled batches.

Input

Parameter	Type	Default	Description
mode	string	index	index or articles.
dateFrom	string	6 days ago	First day, YYYY-MM-DD. Clamped to 1970.
dateTo	string	today	Last day, YYYY-MM-DD.
sections	array	all	Section filter from the URL path, e.g. ["business","world"].
maxItems	integer	500	Cap across the whole range.
proxyConfiguration	object	none	Optional for index; recommended for articles.

Example — a month of business coverage

{
  "mode": "index",
  "dateFrom": "2026-07-01",
  "dateTo": "2026-07-31",
  "sections": ["business"],
  "maxItems": 5000
}

Output

Index row:

{
  "url": "https://www.nytimes.com/2026/08/24/world/canada/us-tariffs-trade-economy.html",
  "headline": "U.S. Tariffs Could Price Canadian Firms Out of U.S. and Threaten Thousands of Jobs",
  "publishedDate": "2026-08-24",
  "archiveDate": "2026-08-25",
  "section": "world",
  "subsection": "canada",
  "source": "The New York Times",
  "_mode": "index",
  "_scrapedAt": "2026-08-26T13:20:41.512Z"
}

articles mode adds:

{
  "authors": ["Ian Austen"],
  "publishedAt": "2026-08-24T09:00:12-04:00",
  "updatedAt": "2026-08-24T14:31:02-04:00",
  "description": "Standfirst text...",
  "keywords": ["International Trade and World Market", "Customs (Tariff)"],
  "content": "Article text...",
  "contentChars": 4820,
  "paragraphCount": 22,
  "isTruncated": false
}

Field reference

Field	Type	Description
publishedDate	string	Date from the article URL.
archiveDate	string	Archive day the article was listed under. These can differ by a day for late-evening stories.
section / subsection	string	Parsed from the URL path.
contentChars	integer	Body length; 0 when the body wasn't fetched.
isTruncated	boolean	true when the fetched body was unusually short.
_articleError	string	Present in articles mode when the body couldn't be retrieved — the index row is still returned.
_error	string	Present on failures (blocked, not_found, unexpected_shape, no_results, http_*).

Known Limits

• archiveDate and publishedDate can differ by one day. The archive groups by publication cycle, not wall-clock midnight. Both are returned so you can pick.
• Older days are bigger. Pre-2010 days often carry 200–300 items including briefs and market tables; budget maxItems accordingly.
• Non-article URLs appear. Crosswords, recipes and interactives are listed by the archive too. Filter on section if you want editorial only.
• articles mode is throttled per IP — see the note above. This is a property of the source, not a bug in the actor.
• Index mode returns no body. That is deliberate: it is the fast, reliable layer.

Scope & Compliance

• Public content only. The actor requests pages the same way an ordinary anonymous visitor's browser does. It uses no login, no subscriber credentials, no cookies from a paid account, and does not attempt to obtain content the publisher withholds from anonymous visitors.
• Subscriber-only material is not retrieved. Where only a headline or intro is served to anonymous visitors, that is what the actor returns, flagged via isTruncated.
• No security control is defeated. Ordinary HTTPS requests with a browser-accurate TLS fingerprint. No CAPTCHA solving, no forged authentication.
• Copyright stays with the publisher. Output is intended for research, monitoring, archiving and analysis. You are responsible for using it in line with the publisher's Terms of Service and copyright law — republishing article text is generally not permitted.
• Rate limits are respected. Requests are paced between days and between articles.

Related Actors

Actor	What it covers
nytimes-scraper	NYT article extraction from URLs — pair it with this actor's index
washingtonpost-scraper	The Washington Post
reuters-scraper	Reuters
ft-scraper	Financial Times