# TechCrunch Scraper (`rl1987/techcrunch-scraper`) Actor Scrape TechCrunch articles by category or search query, with full article content, authors, and publish dates. - **URL**: https://apify.com/rl1987/techcrunch-scraper.md - **Developed by:** [R.L.](https://apify.com/rl1987) (community) - **Categories:** News - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 article scrapeds This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## TechCrunch Scraper ### What does TechCrunch Scraper do? **TechCrunch Scraper** extracts articles from [TechCrunch](https://techcrunch.com), the leading technology and startup news site. It supports two modes: browsing a **category** (e.g. AI, Startups, Security) or running a **search query** across the whole site. For every article found, it fetches the full page and extracts the clean article text, title, author(s), category tags, and publish date. Run it on demand, on a schedule via the Apify platform's scheduler, or integrate it into a pipeline through the API — with automatic proxy rotation to keep scraping reliable at scale. ### Why use TechCrunch Scraper? - **Media monitoring** — track coverage of your company, competitors, or industry as soon as it's published. - **Market and trend research** — pull structured data on startups, funding rounds, and technology trends for analysis. - **Content aggregation** — feed a newsletter, dashboard, or NLP pipeline with clean, structured tech news. - **Lead generation** — identify newly funded startups and the people behind them. ### How to use TechCrunch Scraper 1. Click **Try for free** or **Run** on the Actor page. 2. Choose a **Mode**: `category` to browse a topic (e.g. Artificial Intelligence, Startups) or `search` to look up a keyword. 3. Fill in the **Categories** or **Search queries** field depending on the mode. 4. Set **Max articles per category/query** (or `0` for no limit). 5. Click **Start** and wait for the run to finish. 6. Open the **Dataset** tab to view, filter, and export the results (JSON, CSV, Excel, and more). ### Input | Field | Type | Description | |---|---|---| | `mode` | string | `category` or `search`. | | `categories` | array | TechCrunch category slugs to scrape (used when `mode` is `category`), e.g. `artificial-intelligence`, `startups`, `security`. | | `searchQueries` | array | Search terms to look up on techcrunch.com (used when `mode` is `search`). | | `maxItems` | integer | Max articles per category/query. `0` = unlimited. | | `proxyConfiguration` | object | Proxy settings for requests to techcrunch.com. Apify Proxy is recommended. | See the **Input** tab for the full schema and defaults. ### Output Each dataset item represents one article: ```json { "url": "https://techcrunch.com/2026/07/25/librarians-are-hosting-viral-avoiding-ai-workshops/", "title": "Librarians are hosting viral 'Avoiding AI' workshops for people who are fed up with Big Tech", "authors": ["Amanda Silberling"], "categories": ["AI"], "publishedAt": "2026-07-25", "excerpt": "Librarians across the U.S. are teaching patrons how to opt out of AI features...", "content": "\"Everybody's on their phone at my program!\" jokes Charlie Bailey...", "sourceType": "category", "sourceTerm": "artificial-intelligence" } ```` You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. #### Data table | Field | Description | |---|---| | `url` | Canonical article URL | | `title` | Article headline | | `authors` | Byline author name(s) | | `categories` | TechCrunch category tags | | `publishedAt` | Publish date (ISO format) | | `excerpt` | Short summary/dek | | `content` | Full extracted article text | | `sourceType` | `category` or `search` | | `sourceTerm` | The category slug or search query that produced this item | ### How much does it cost to scrape TechCrunch? Cost scales with the number of articles scraped. A run of a few hundred articles typically uses a small fraction of the free monthly Apify platform credits. ### Tips and advanced options - Lower `maxItems` for quick previews; raise it (or set `0`) for full backfills. - Combine multiple `categories` or `searchQueries` in one run — results for each are pushed to the dataset as they complete. - Use Apify's [Scheduler](https://docs.apify.com/platform/schedules) to run this Actor daily for ongoing monitoring. ### FAQ, disclaimers, and support This Actor scrapes only publicly available pages on techcrunch.com. Use it in accordance with TechCrunch's Terms of Service and applicable laws in your jurisdiction; it is intended for research, monitoring, and archival purposes. Site markup may change over time, which can affect extraction accuracy — please report issues via the Actor's **Issues** tab. Need a custom version or a different news source? Reach out and we can help build a tailored solution. # Actor input Schema ## `mode` (type: `string`): Scrape by category listing or by search query. ## `categories` (type: `array`): TechCrunch category slugs to scrape (used when Mode = Browse category). See https://techcrunch.com/category/ for the full list. ## `searchQueries` (type: `array`): Search terms to look up on techcrunch.com (used when Mode = Search). ## `maxItems` (type: `integer`): Maximum number of articles to scrape for each category or search query. 0 = no limit (scrape all available pages). ## `proxyConfiguration` (type: `object`): Proxy settings used for all requests to techcrunch.com. ## Actor input object example ```json { "mode": "category", "categories": [ "artificial-intelligence" ], "searchQueries": [ "openai" ], "maxItems": 20, "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `articles` (type: `string`): Scraped TechCrunch articles with title, URL, authors, categories, publish date, excerpt, and full content. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("rl1987/techcrunch-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("rl1987/techcrunch-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call rl1987/techcrunch-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=rl1987/techcrunch-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "TechCrunch Scraper", "description": "Scrape TechCrunch articles by category or search query, with full article content, authors, and publish dates.", "version": "0.1", "x-build-id": "bS4Y0xsGY6fu5A05s" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/rl1987~techcrunch-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-rl1987-techcrunch-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/rl1987~techcrunch-scraper/runs": { "post": { "operationId": "runs-sync-rl1987-techcrunch-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/rl1987~techcrunch-scraper/run-sync": { "post": { "operationId": "run-sync-rl1987-techcrunch-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "mode" ], "properties": { "mode": { "title": "Mode", "enum": [ "category", "search" ], "type": "string", "description": "Scrape by category listing or by search query.", "default": "category" }, "categories": { "title": "Categories", "type": "array", "description": "TechCrunch category slugs to scrape (used when Mode = Browse category). See https://techcrunch.com/category/ for the full list.", "items": { "type": "string", "enum": [ "artificial-intelligence", "apps", "biotech-health", "climate", "cloud-computing", "commerce", "crypto", "enterprise", "fintech", "gadgets", "gaming", "government-policy", "hardware", "media-entertainment", "privacy", "robotics", "security", "social", "space", "startups", "transportation", "venture" ], "enumTitles": [ "Artificial Intelligence", "Apps", "Biotech & Health", "Climate", "Cloud Computing", "Commerce", "Crypto", "Enterprise", "Fintech", "Gadgets", "Gaming", "Government & Policy", "Hardware", "Media & Entertainment", "Privacy", "Robotics", "Security", "Social", "Space", "Startups", "Transportation", "Venture" ] }, "default": [ "artificial-intelligence" ] }, "searchQueries": { "title": "Search queries", "type": "array", "description": "Search terms to look up on techcrunch.com (used when Mode = Search).", "default": [ "openai" ], "items": { "type": "string" } }, "maxItems": { "title": "Max articles per category/query", "minimum": 0, "type": "integer", "description": "Maximum number of articles to scrape for each category or search query. 0 = no limit (scrape all available pages).", "default": 20 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy settings used for all requests to techcrunch.com.", "default": { "useApifyProxy": true } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```