MIT Technology Review Scraper

Extract emerging-technology journalism from MIT Technology Review — headline, byline, authors, publication date, section, tags, lead image and article text — from technologyreview.com. Give it URLs, or let it pull the latest stories from any section.

Why Use This Actor?

• Complete article text. Returns the full body the site renders for a story. Measured on 5,548-5,716 characters across two articles.
• Clean, typed output. Authors as a list, ISO timestamps, section and tags as separate fields — ready for a database or a notebook with no post-processing.
• Discovery built in. mode: "latest" walks the sections you name (and its public RSS feeds) and collects recent article URLs, so you don't have to supply a list.
• Honest about what it got. Every row carries contentChars and isTruncated, so a stub is never silently mistaken for a full article.
• No browser. Plain HTTP requests, so runs are fast and cheap and stay well under 256 MB of RAM.

How It Works

The actor reads the article's JSON-LD metadata first — the most stable container a publisher offers — and falls back to the rendered paragraphs when the site doesn't ship articleBody. Requests carry a browser-accurate TLS fingerprint so no browser engine is needed; if one fingerprint stops being accepted, the actor advances through a ladder of alternatives instead of failing the run.

Input

Parameter	Type	Default	Description
mode	string	article	article scrapes only the URLs you pass. latest discovers recent articles from sections first.
startUrls	array	–	MIT Technology Review article URLs.
sections	array	["artificial-intelligence", "climate-change", "biotechnology"]	Sections to pull from in latest mode.
maxItems	integer	50	Maximum articles per run.
proxyConfiguration	object	none	Optional. The site answers plain HTTP from most IPs; a residential proxy helps if you hit rate limits at volume.

Example input

{
  "mode": "latest",
  "sections": ["artificial-intelligence", "climate-change", "biotechnology"],
  "maxItems": 25
}

Output

One row per article:

{
  "url": "https://www.technologyreview.com/...",
  "source": "MIT Technology Review",
  "title": "Article headline",
  "description": "Standfirst or summary",
  "authors": ["Jane Doe"],
  "publishedAt": "2026-08-24T16:00:57.673Z",
  "updatedAt": "2026-08-24T18:22:10.004Z",
  "section": "Technology",
  "tags": ["ai", "policy"],
  "leadImage": "https://...",
  "content": "Full article text, paragraphs separated by blank lines...",
  "contentChars": 14891,
  "paragraphCount": 33,
  "isTruncated": false,
  "_source": "json_ld",
  "_scrapedAt": "2026-08-26T12:04:11.113Z"
}

Field reference

Field	Type	Description
title	string	Article headline.
authors	string[]	Bylined authors. Empty for unsigned pieces.
publishedAt / updatedAt	string	ISO-8601 timestamps.
section	string	Primary section.
tags	string[]	Editorial tags or keywords.
content	string	Article body text, paragraphs separated by blank lines.
contentChars	integer	Length of content. Use it to filter out stubs.
paragraphCount	integer	Number of prose blocks recovered.
isTruncated	boolean	true when the body came back unusually short.
_source	string	Which layer supplied the body — json_ld or paragraphs.
_error	string	Present only on failures (not_found, blocked, http_*).

Known Limits

• Some items are genuinely short — briefs, photo essays and podcast pages carry little prose. Check contentChars rather than assuming a failure.
• Interactive features and live blogs use custom embeds that hold little text, so expect a low contentChars there.
• Discovery is recency-based. latest mode reaches current section fronts and public feeds, not a historical archive; to go back in time, supply URLs.
• Volume. At a few hundred articles per run a residential proxy is recommended to avoid rate limiting.

Scope & Compliance

• Public content only. This actor requests pages the same way an ordinary anonymous visitor's browser does. It uses no login, no subscriber credentials, no cookies from a paid account, and does not attempt to obtain content the publisher withholds from anonymous visitors.
• Subscriber-only material is not retrieved. Where the site serves only a headline, summary or intro to anonymous visitors, that is what the actor returns — and it flags the result via isTruncated so your pipeline can tell the difference.
• No security control is defeated. The actor sends ordinary HTTPS requests with a browser-accurate TLS fingerprint so it does not need a browser engine. It solves no CAPTCHAs and forges no authentication.
• Copyright stays with the publisher. Output is intended for research, monitoring, archiving and analysis. You are responsible for using it in line with the publisher's Terms of Service, copyright law and any applicable data-protection rules — republishing article text is generally not permitted.
• Rate limits are respected. Requests are paced and retried politely; the actor is not designed for denial-of-service-scale traffic.

Related Actors

Actor	Source
nytimes-scraper	New York Times
washingtonpost-scraper	The Washington Post
reuters-scraper	Reuters
ft-scraper	Financial Times
forbes-scraper	Forbes