# Techcrunch News Scraper (`real-estate-api/techcrunch-news-scraper`) Actor

TechCrunch News Scraper is a Python tool designed to extract latest tech headlines, article bodies, author details, publication dates, and tags from TechCrunch. It exports structured startup and tech news into clean CSV or JSON formats for media monitoring and market research.

- **URL**: https://apify.com/real-estate-api/techcrunch-news-scraper.md
- **Developed by:** [real-estate-api](https://apify.com/real-estate-api) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

![TechCrunch News Scraper for Startup Headlines and Articles](https://images.bigibyte.com/cdn-cgi/image/width=800,quality=80,format=auto/apify-actors-covers/techcrunch-news-scraper-cover.png)

## TechCrunch News Scraper for Startup Headlines and Articles

TechCrunch News Scraper collects recent stories from TechCrunch's public RSS feed and supported WordPress posts source. Each dataset row includes a headline, article link, author, publication date, categories, story identifier, and short summary.

Export results to JSON, CSV, or Excel for startup monitoring, company research, keyword tracking, and scheduled news collection. The RSS feed provides recent items, while the posts source supports pagination for collecting a larger set of stories.

### Use Cases

- **Startup monitoring:** Track recent TechCrunch headlines on a schedule.
- **Keyword filtering:** Find stories mentioning a company, product, or funding round.
- **Research notes:** Export headlines, links, publication dates, and authors.
- **Historical sampling:** Collect more stories using the paginated posts source.
- **News pipelines:** Pass structured headline data to downstream workflows.

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `feedUrls` | Array | RSS feed and posts source | Supported TechCrunch source URLs. Other hosts or unsupported paths produce error rows. |
| `keyword` | String | Empty | Optional filter applied to headlines and short summaries. |
| `maxItems` | Integer | `100` | Maximum number of stories to save per run, from 1 to 1000. |
| `timeoutSecs` | Integer | `300` | Overall run time limit in seconds. |
| `requestTimeoutSecs` | Integer | `30` | Timeout for each download in seconds. |

### Supported Sources

The actor supports these two TechCrunch sources:

- **RSS feed:** Provides recent stories, without backward pagination.
- **WordPress posts source:** Supports page-based collection, allowing more stories to be collected up to the configured item limit.

The posts source processes up to 100 items per page. It continues through subsequent pages until the item cap is reached, a page is empty, or a short page is returned.

### Example Input

Replace the placeholders with the supported TechCrunch source URLs.

```json
{
  "feedUrls": [
    "TECHCRUNCH_RSS_FEED_URL",
    "TECHCRUNCH_WORDPRESS_POSTS_URL"
  ],
  "keyword": "",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}
```

### What Data Does This Actor Extract?

Each story is saved as a separate dataset row. The headline is stored in `itemTitle`, and the short blurb is stored in `summary`. HTML is removed from the summary, which is then trimmed to a maximum of 400 characters.

The actor does not collect full article bodies. RSS category labels are preserved as labels, while category values from the posts source remain in the form supplied by that source.

#### Example Output

```json
{
  "itemTitle": "Example TechCrunch Story Headline",
  "link": "ARTICLE_URL",
  "author": "Example Author",
  "pubDate": "Sun, 11 Oct 2026 21:14:40 +0000",
  "categories": [
    "AI",
    "Gadgets"
  ],
  "guid": "STORY_IDENTIFIER",
  "summary": "A short summary of the story.",
  "scrapedAt": "2026-10-12T00:00:00+00:00"
}
```

#### Output Fields

| Field | Type | Description |
|---|---|---|
| `itemTitle` | String | Story headline. |
| `link` | String or null | Article URL. |
| `author` | String or null | Author name, when available. |
| `pubDate` | String or null | Publication date provided by the source. |
| `categories` | Array | Category labels or category identifiers supplied by the source. |
| `guid` | String | Story identifier used for deduplication. |
| `summary` | String or null | Short, plain-text story summary. |
| `scrapedAt` | String | UTC timestamp when the row was collected. |
| `error` | String | Error description for a failed input. |
| `input` | String | URL or value associated with the error. |

#### Error Handling and Deduplication

- Invalid URLs, failed downloads, and HTML error pages produce error rows instead of automatically failing the entire run.
- Valid sources can still be processed when another source fails.
- Duplicate `guid` values are skipped.
- Missing publication dates or summaries are stored as `null` when the source does not provide them.

### How It Works

1. Reads the source list, optional keyword, item limit, and timeout settings.
2. Validates source URLs against the supported TechCrunch paths.
3. Downloads and parses RSS XML or posts data in JSON format.
4. Extracts story metadata and removes HTML from summary text.
5. Applies the optional keyword filter and skips duplicate story identifiers.
6. Continues through posts pages when applicable, stopping at the configured item cap or the end of available results.
7. Saves story rows and error details to the dataset.

#### Retries and Request Limits

- Schedule requests to the same source approximately 5–15 minutes apart.
- The actor waits about one second between requests and limits concurrent downloads.
- HTTP 429 and 5xx responses are retried up to three times, with delays of approximately 2, 4, and 8 seconds.
- A suitable `Retry-After` value is respected when provided, subject to the configured retry behavior.
- The overall run time and individual download timeout are controlled by `timeoutSecs` and `requestTimeoutSecs`.

### Limits and Terms

This actor collects headlines, article links, publication dates, authors, categories, and short summaries. It does not store full article bodies.

The RSS feed only contains recent stories. Use the paginated posts source when you need a larger collection, subject to the maximum item limit and the content available from the source.

Review TechCrunch's applicable terms before reusing exported content. Copyright, attribution, commercial-use, and other restrictions may apply. You are responsible for ensuring that your use of the data complies with relevant requirements.

### FAQ

#### Can I export results to CSV or JSON?

Yes. Apify dataset results can be downloaded as JSON, CSV, or Excel.

#### Why does the RSS feed only return recent stories?

The RSS feed provides the latest items and does not support backward pagination. Use the supported posts source for a larger collection, or schedule recurring RSS runs and compare `guid` values.

#### What happens if a source URL is incorrect?

The actor saves an error row containing the failed input. Other valid sources can still be processed.

#### Does the actor download full articles?

No. It saves headline metadata and short summaries rather than full article text.

#### How many stories can one run collect?

The `maxItems` setting defaults to `100` and supports a maximum of `1000` stories per run.

#### How can I avoid duplicate stories across scheduled runs?

Store the `guid` values from previous exports and compare them with new results before processing stories.

### Trademark Notice

TechCrunch and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the source of the data and does not imply affiliation, sponsorship, endorsement, or partnership.

### Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with TechCrunch or its owners. Users are responsible for complying with applicable source terms, rate limits, attribution requirements, and licensing restrictions.

### Is Web Scraping Legal?

Collecting information that is already public is generally legal. Scraping is not the same thing as hacking, and there is no single law that bans it. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.

This actor collects information from public pages. It does not log in and it does not bypass access controls.

For a longer explanation, see Apify's article [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/).

# Actor input Schema

## `feedUrls` (type: `array`):

Public TechCrunch URLs to read. Use https://techcrunch.com/feed/ for the RSS feed, or https://techcrunch.com/wp-json/wp/v2/posts with page and per_page for the WordPress posts endpoint. Other hosts are saved as error rows.

## `keyword` (type: `string`):

Optional client-side filter. Keeps rows whose headline or short summary contains this text, case insensitive. Leave empty to keep every item.

## `maxItems` (type: `integer`):

Maximum number of stories to save in one run. Values are clamped between 1 and 1000.

## `timeoutSecs` (type: `integer`):

Overall run time limit in seconds. The actor stops requesting more pages when this limit is reached.

## `requestTimeoutSecs` (type: `integer`):

Per-request HTTP timeout in seconds for each feed or posts page download.

## `proxyConfiguration` (type: `object`):

Optional proxy settings. The default sends requests directly, with no Apify proxy. Turn on a proxy only if you need one.

## Actor input object example

```json
{
  "feedUrls": [
    "https://techcrunch.com/feed/",
    "https://techcrunch.com/wp-json/wp/v2/posts?per_page=2&_fields=id,date,link,title,excerpt,categories"
  ],
  "keyword": "",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `headlines` (type: `string`):

Default dataset of TechCrunch headline rows.

## `overview` (type: `string`):

Table of headlines, authors, dates, links, and source URLs.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://techcrunch.com/feed/",
        "https://techcrunch.com/wp-json/wp/v2/posts?per_page=2&_fields=id,date,link,title,excerpt,categories"
    ],
    "keyword": "",
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("real-estate-api/techcrunch-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "feedUrls": [
        "https://techcrunch.com/feed/",
        "https://techcrunch.com/wp-json/wp/v2/posts?per_page=2&_fields=id,date,link,title,excerpt,categories",
    ],
    "keyword": "",
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("real-estate-api/techcrunch-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    "https://techcrunch.com/feed/",
    "https://techcrunch.com/wp-json/wp/v2/posts?per_page=2&_fields=id,date,link,title,excerpt,categories"
  ],
  "keyword": "",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call real-estate-api/techcrunch-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,real-estate-api/techcrunch-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/2zS02HkzC5pnOsmmu/builds/36NIHonTxu0nGuGAi/openapi.json