# MIT Technology Review Scraper (`xtracto/mit-tech-review-scraper`) Actor

Extract MIT Technology Review articles: headline, byline, authors, date, section, tags and the article text as served to anonymous visitors. HTTP-only - no browser, no login.

- **URL**: https://apify.com/xtracto/mit-tech-review-scraper.md
- **Developed by:** [Farhan Febrian Nauval](https://apify.com/xtracto) (community)
- **Categories:** News, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $15.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## MIT Technology Review Scraper

Extract emerging-technology journalism from MIT Technology Review — headline, byline, authors, publication date, section, tags, lead image and article text — from **technologyreview.com**. Give it URLs, or let it pull the latest stories from any section.

### Why Use This Actor?

- **Complete article text.** Returns the full body the site renders for a story. Measured on 5,548-5,716 characters across two articles.
- **Clean, typed output.** Authors as a list, ISO timestamps, section and tags as separate fields — ready for a database or a notebook with no post-processing.
- **Discovery built in.** `mode: "latest"` walks the sections you name (and its public RSS feeds) and collects recent article URLs, so you don't have to supply a list.
- **Honest about what it got.** Every row carries `contentChars` and `isTruncated`, so a stub is never silently mistaken for a full article.
- **No browser.** Plain HTTP requests, so runs are fast and cheap and stay well under 256 MB of RAM.

### How It Works

The actor reads the article's JSON-LD metadata first — the most stable container a publisher offers — and falls back to the rendered paragraphs when the site doesn't ship `articleBody`. Requests carry a browser-accurate TLS fingerprint so no browser engine is needed; if one fingerprint stops being accepted, the actor advances through a ladder of alternatives instead of failing the run.

### Input

| Parameter | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `article` | `article` scrapes only the URLs you pass. `latest` discovers recent articles from `sections` first. |
| `startUrls` | array | – | MIT Technology Review article URLs. |
| `sections` | array | `["artificial-intelligence", "climate-change", "biotechnology"]` | Sections to pull from in `latest` mode. |
| `maxItems` | integer | `50` | Maximum articles per run. |
| `proxyConfiguration` | object | none | Optional. The site answers plain HTTP from most IPs; a residential proxy helps if you hit rate limits at volume. |

#### Example input

```json
{
  "mode": "latest",
  "sections": ["artificial-intelligence", "climate-change", "biotechnology"],
  "maxItems": 25
}
```

### Output

One row per article:

```json
{
  "url": "https://www.technologyreview.com/...",
  "source": "MIT Technology Review",
  "title": "Article headline",
  "description": "Standfirst or summary",
  "authors": ["Jane Doe"],
  "publishedAt": "2026-08-24T16:00:57.673Z",
  "updatedAt": "2026-08-24T18:22:10.004Z",
  "section": "Technology",
  "tags": ["ai", "policy"],
  "leadImage": "https://...",
  "content": "Full article text, paragraphs separated by blank lines...",
  "contentChars": 14891,
  "paragraphCount": 33,
  "isTruncated": false,
  "_source": "json_ld",
  "_scrapedAt": "2026-08-26T12:04:11.113Z"
}
```

#### Field reference

| Field | Type | Description |
|---|---|---|
| `title` | string | Article headline. |
| `authors` | string\[] | Bylined authors. Empty for unsigned pieces. |
| `publishedAt` / `updatedAt` | string | ISO-8601 timestamps. |
| `section` | string | Primary section. |
| `tags` | string\[] | Editorial tags or keywords. |
| `content` | string | Article body text, paragraphs separated by blank lines. |
| `contentChars` | integer | Length of `content`. Use it to filter out stubs. |
| `paragraphCount` | integer | Number of prose blocks recovered. |
| `isTruncated` | boolean | `true` when the body came back unusually short. |
| `_source` | string | Which layer supplied the body — `json_ld` or `paragraphs`. |
| `_error` | string | Present only on failures (`not_found`, `blocked`, `http_*`). |

### Known Limits

- **Some items are genuinely short** — briefs, photo essays and podcast pages carry little prose. Check `contentChars` rather than assuming a failure.
- **Interactive features and live blogs** use custom embeds that hold little text, so expect a low `contentChars` there.
- **Discovery is recency-based.** `latest` mode reaches current section fronts and public feeds, not a historical archive; to go back in time, supply URLs.
- **Volume.** At a few hundred articles per run a residential proxy is recommended to avoid rate limiting.

### Scope & Compliance

- **Public content only.** This actor requests pages the same way an ordinary anonymous visitor's browser does. It uses **no login, no subscriber credentials, no cookies from a paid account**, and does not attempt to obtain content the publisher withholds from anonymous visitors.
- **Subscriber-only material is not retrieved.** Where the site serves only a headline, summary or intro to anonymous visitors, that is what the actor returns — and it flags the result via `isTruncated` so your pipeline can tell the difference.
- **No security control is defeated.** The actor sends ordinary HTTPS requests with a browser-accurate TLS fingerprint so it does not need a browser engine. It solves no CAPTCHAs and forges no authentication.
- **Copyright stays with the publisher.** Output is intended for research, monitoring, archiving and analysis. You are responsible for using it in line with the publisher's Terms of Service, copyright law and any applicable data-protection rules — republishing article text is generally not permitted.
- **Rate limits are respected.** Requests are paced and retried politely; the actor is not designed for denial-of-service-scale traffic.

### Related Actors

| Actor | Source |
|---|---|
| [`nytimes-scraper`](https://apify.com/xtracto/nytimes-scraper) | New York Times |
| [`washingtonpost-scraper`](https://apify.com/xtracto/washingtonpost-scraper) | The Washington Post |
| [`reuters-scraper`](https://apify.com/xtracto/reuters-scraper) | Reuters |
| [`ft-scraper`](https://apify.com/xtracto/ft-scraper) | Financial Times |
| [`forbes-scraper`](https://apify.com/xtracto/forbes-scraper) | Forbes |

# Actor input Schema

## `mode` (type: `string`):

article = scrape only the URLs you provide. latest = discover recent articles from the chosen sections first.

## `startUrls` (type: `array`):

MIT Technology Review article URLs to scrape.

## `sections` (type: `array`):

Sections to pull recent articles from when mode is 'latest'. Examples: artificial-intelligence, climate-change, biotechnology.

## `maxItems` (type: `integer`):

Maximum number of articles to scrape in one run.

## `proxyConfiguration` (type: `object`):

Optional. The site answers plain HTTP from most IPs; a residential proxy helps if you hit rate limits at volume.

## Actor input object example

```json
{
  "mode": "latest",
  "sections": [
    "artificial-intelligence",
    "climate-change"
  ],
  "maxItems": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `title` (type: `string`):

Title of the item.

## `authors` (type: `string`):

Author names. List of values.

## `publishedAt` (type: `string`):

Publication timestamp, ISO 8601.

## `section` (type: `string`):

Section or category the item belongs to.

## `contentChars` (type: `string`):

Length of the body text in characters. Whole number.

## `url` (type: `string`):

Direct link to the scraped item.

## `_scrapedAt` (type: `string`):

UTC timestamp of the scrape, ISO 8601.

## `tags` (type: `string`):

Tags or keywords. List of values.

## `content` (type: `string`):

Main body text.

## `paragraphCount` (type: `string`):

Number of prose blocks recovered. Whole number.

## `isTruncated` (type: `string`):

True when only a short preview was available. Boolean value.

## `canonicalUrl` (type: `string`):

Canonical URL reported by the source.

## `source` (type: `string`):

Name of the source the row came from.

## `description` (type: `string`):

Short description or summary.

## `updatedAt` (type: `string`):

Last-updated timestamp, ISO 8601.

## `leadImage` (type: `string`):

Main image URL.

## `_source` (type: `string`):

Which extraction strategy produced the row.

## `_error` (type: `string`):

Set only on diagnostic rows - why that target produced no data.

## `_message` (type: `string`):

Human-readable explanation of the error.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "latest",
    "sections": [
        "artificial-intelligence",
        "climate-change"
    ],
    "maxItems": 10,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("xtracto/mit-tech-review-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "latest",
    "sections": [
        "artificial-intelligence",
        "climate-change",
    ],
    "maxItems": 10,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("xtracto/mit-tech-review-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "latest",
  "sections": [
    "artificial-intelligence",
    "climate-change"
  ],
  "maxItems": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call xtracto/mit-tech-review-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,xtracto/mit-tech-review-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7OjaP84pc3nwAt3jW/builds/vl7Zbk1hK9JS4aQGB/openapi.json