# TechCrunch Articles Scraper (`solidcode/techcrunch-scraper`) Actor \[πŸ’° $2.49 / 1K] Search and scrape TechCrunch articles by keyword. Extract headlines, article URLs, authors, categories, and publication dates as clean structured data β€” no API key needed. Filter by date, sort by newest or relevance, and export to JSON, CSV, or Excel. - **URL**: https://apify.com/solidcode/techcrunch-scraper.md - **Developed by:** [SolidCode](https://apify.com/solidcode) (community) - **Categories:** News, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.49 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## TechCrunch Articles Scraper Search TechCrunch by keyword and pull clean, structured article records at scale β€” headlines, direct article URLs, excerpts, author name and profile link, category name and category-page link, featured image, publish date, and last-modified timestamp. A single term like `openai` surfaces 3,600+ matching articles from a ~261,000-article public archive. Built for journalists, market researchers, competitive-intelligence analysts, and content teams tracking tech news who need a ready-to-use TechCrunch dataset without copying stories out of the site one page at a time. ### Why This Scraper? - **The full ~261,000-article TechCrunch archive** β€” search the entire public catalog, not just the front page. A single keyword like `openai` returns 3,600+ matching articles in one run. - **Author attribution on every article** β€” both the writer's name *and* their clickable TechCrunch author profile URL, so you can trace a byline or build a per-author feed. - **Category taxonomy with links** β€” each article carries its section name (Fundraising, AI, Security…) plus the category-page URL, not just a bare label. - **Precise date-range windowing** β€” `publishedAfter` and `publishedBefore` box results to an exact YYYY-MM-DD range for time-boxed coverage studies and event tracking. - **Newest-first or Most-Relevant ordering** β€” sort a keyword's matches chronologically for a live feed, or by relevance to surface the strongest matches first. - **Batch keywords in one run** β€” pass an array of terms (`artificial intelligence`, `startup funding`, `cybersecurity`) and each is searched independently, with the matching keyword stamped on every row. - **Featured image, excerpt, and last-modified timestamp per article** β€” a ready-to-render card for each story, plus a `modifiedAt` field to catch updated posts. - **Latest-feed mode** β€” leave keywords empty to stream TechCrunch's newest published articles in chronological order, no search term required. ### Use Cases **Market & Competitive Intelligence** - Track every TechCrunch mention of a competitor, product, or funding round - Monitor coverage of a specific sector (AI, fintech, climate) over time - Map which topics TechCrunch covers most heavily in a given quarter - Build a briefing feed of the latest tech-industry stories **Media Monitoring & PR** - Catch new articles that name your company, founders, or executives - Measure share of voice against competitors across TechCrunch coverage - Archive press clips by author, category, and publish date for reports - Alert stakeholders when a story about your space goes live **Content & SEO Research** - Analyze which authors and categories dominate a topic - Study headline and excerpt patterns for high-coverage keywords - Curate topical newsletters from a keyword's newest matches - Pull featured-image and excerpt data to power content roundups **Trend & Timeline Analysis** - Chart a story's coverage arc with a keyword plus a tight date window - Compare article volume for competing technologies across years - Build a dated corpus of a keyword's coverage for longitudinal study - Feed clean article metadata into sentiment or topic-modeling pipelines **Lead & Company Research** - Surface funding and launch stories to spot emerging companies early - Follow a single author's beat by profile URL to track their coverage - Compile a dated list of startups featured under a given category ### Getting Started #### Simple Keyword Search The minimum input β€” one or more keywords: ```json { "keywords": ["artificial intelligence"], "maxResults": 100 } ```` #### Time-Boxed Coverage Track a story's coverage arc within an exact date window, newest first: ```json { "keywords": ["openai"], "publishedAfter": "2025-01-01", "publishedBefore": "2025-06-30", "sortBy": "date", "maxResults": 500 } ``` #### Multi-Keyword, Most Relevant Batch several topics in one run, ordered by relevance: ```json { "keywords": ["startup funding", "cybersecurity", "electric vehicles"], "sortBy": "relevance", "maxResults": 300 } ``` ### Input Reference #### What to Search | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `keywords` | string\[] | `["artificial intelligence"]` | Words or phrases to search TechCrunch for β€” one per line, each searched separately. Leave empty to collect the latest published articles instead. | #### Options | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `maxResults` | integer | `100` | Maximum articles per keyword. Three keywords with a limit of 100 returns up to 300 articles. Set to `0` for no limit. | | `sortBy` | select | `Newest First` | Order matches before collection: **Newest First** (chronological) or **Most Relevant**. Latest-feed mode is always newest first. | | `publishedAfter` | string | β€” | Only collect articles published on or after this date (`YYYY-MM-DD`, e.g. `2025-01-01`). Leave empty for no lower limit. | | `publishedBefore` | string | β€” | Only collect articles published on or before this date (`YYYY-MM-DD`, e.g. `2025-12-31`). Leave empty for no upper limit. | ### Output Each record is one TechCrunch article with clean, flat fields: ```json { "id": 2891234, "title": "OpenAI launches new reasoning model with major benchmark gains", "url": "https://techcrunch.com/2025/03/18/openai-launches-new-reasoning-model/", "slug": "openai-launches-new-reasoning-model", "excerpt": "The company says the new model outperforms its predecessor on math and coding tasks while cutting latency.", "authorName": "Kyle Wiggers", "authorUrl": "https://techcrunch.com/author/kyle-wiggers/", "categoryName": "AI", "categoryUrl": "https://techcrunch.com/category/artificial-intelligence/", "imageUrl": "https://techcrunch.com/wp-content/uploads/2025/03/openai-model.jpg", "publishedDate": "2025-03-18", "publishedAt": "2025-03-18T14:32:05", "modifiedAt": "2025-03-18T16:10:41", "searchKeyword": "openai", "scrapedAt": "2026-07-03T13:45:12.284917+00:00" } ``` #### Article Fields | Field | Type | Description | |-------|------|-------------| | `id` | number | Unique TechCrunch article identifier | | `title` | string | Article headline | | `url` | string | Direct link to the article on TechCrunch | | `slug` | string | URL slug for the article | | `excerpt` | string | Short article summary, plain text | | `imageUrl` | string | null | Featured image URL | #### Author & Category | Field | Type | Description | |-------|------|-------------| | `authorName` | string | null | Article author's name | | `authorUrl` | string | null | Link to the author's TechCrunch profile page | | `categoryName` | string | null | Article category / section name (e.g. "AI", "Fundraising") | | `categoryUrl` | string | null | Link to the category's TechCrunch page | #### Dates & Provenance | Field | Type | Description | |-------|------|-------------| | `publishedDate` | string | Publication date, `YYYY-MM-DD` | | `publishedAt` | string | Full publication timestamp (ISO 8601) | | `modifiedAt` | string | Last-modified timestamp (ISO 8601) β€” catches updated posts | | `searchKeyword` | string | null | The keyword that matched this article. `null` in latest-feed mode | | `scrapedAt` | string | ISO 8601 timestamp of when the record was captured | ### Tips for Best Results - **Leave keywords empty for the latest feed** β€” no search term needed; the scraper streams TechCrunch's newest published articles in chronological order, ideal for a rolling tech-news monitor. - **Combine a keyword with a tight date window** β€” pair a term with `publishedAfter` and `publishedBefore` to isolate a single story's coverage arc instead of pulling the whole archive. - **Widen date boundaries by a day for exact cutoffs** β€” `publishedAfter`/`publishedBefore` filter on TechCrunch's own publication timezone, so articles right at the edge of your range can shift by a few hours; add a day of buffer if you need airtight precision. - **`maxResults` is per keyword** β€” three keywords at a limit of 100 returns up to 300 rows. Budget your run accordingly, and use `0` only when you truly want everything. - **Start small to validate** β€” set `maxResults` to 25–50 on your first run to confirm the fields and keyword match your needs, then scale up. - **Use Most Relevant for research, Newest First for monitoring** β€” relevance surfaces the strongest matches for a topic study; newest-first is best for feeds and alerts. - **Filter noise with specific phrases** β€” a precise phrase like `startup funding` returns tighter results than a single broad word, cutting incidental mentions. - **Follow a beat by author** β€” every row carries the author's profile URL, so you can pivot from a keyword search to tracking one writer's coverage. ### Pricing **From $2.49 per 1,000 results** β€” pay only for the articles you receive, cheaper than comparable TechCrunch scrapers on the store. Bronze, Silver, and Gold subscribers pay progressively less; the table below shows total cost at each discount tier. | Results | No discount | Bronze | Silver | Gold | |---------|-------------|--------|--------|------| | 100 | $0.30 | $0.28 | $0.27 | $0.25 | | 1,000 | $2.95 | $2.80 | $2.65 | $2.49 | | 10,000 | $29.50 | $28.00 | $26.50 | $24.90 | | 100,000 | $295.00 | $280.00 | $265.00 | $249.00 | A "result" is any article row in the output dataset. There are no compute or time-based charges β€” you pay per result, plus a small fixed per-run start fee. Platform fees depend on your Apify plan. ### Integrations Export data in JSON, CSV, Excel, XML, or RSS. Connect to 1,500+ apps via: - **Zapier** / **Make** / **n8n** β€” Workflow automation - **Google Sheets** β€” Direct spreadsheet export - **Slack** / **Email** β€” Notifications on new results - **Webhooks** β€” Trigger custom APIs on run completion - **Apify API** β€” Full programmatic access ### Legal & Ethical Use This actor is designed for legitimate media monitoring, market research, and journalistic work. Users are responsible for complying with applicable laws and TechCrunch's Terms of Service. Respect copyright when redistributing headlines or summaries, and do not use collected data for spam, harassment, or any illegal purpose. # Actor input Schema ## `keywords` (type: `array`): Words or phrases to search TechCrunch for. Add one per line β€” each is searched separately (for example 'artificial intelligence', 'startup funding', 'cybersecurity'). Leave empty to collect the latest TechCrunch articles. ## `maxResults` (type: `integer`): Maximum number of articles to collect for each keyword. For example, 3 keywords with a limit of 100 returns up to 300 articles. Set to 0 for no limit (collect everything available). ## `sortBy` (type: `string`): Choose how matching articles are ordered before collection. ## `publishedAfter` (type: `string`): Only collect articles published on or after this date. Use the format YYYY-MM-DD, for example 2025-01-01. Leave empty for no lower limit. Filtering uses TechCrunch's own publication timezone, so articles right at the boundary may shift by a few hours in either direction β€” widen the range by a day if you need an exact cutoff. ## `publishedBefore` (type: `string`): Only collect articles published on or before this date. Use the format YYYY-MM-DD, for example 2025-12-31. Leave empty for no upper limit. Filtering uses TechCrunch's own publication timezone, so articles right at the boundary may shift by a few hours in either direction β€” widen the range by a day if you need an exact cutoff. ## Actor input object example ```json { "keywords": [ "artificial intelligence" ], "maxResults": 100, "sortBy": "date" } ``` # Actor output Schema ## `overview` (type: `string`): Table of scraped articles with key fields β€” title, author, category, publication date, matched keyword, and URL. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "keywords": [ "artificial intelligence" ], "maxResults": 100, "sortBy": "date" }; // Run the Actor and wait for it to finish const run = await client.actor("solidcode/techcrunch-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "keywords": ["artificial intelligence"], "maxResults": 100, "sortBy": "date", } # Run the Actor and wait for it to finish run = client.actor("solidcode/techcrunch-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "keywords": [ "artificial intelligence" ], "maxResults": 100, "sortBy": "date" }' | apify call solidcode/techcrunch-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=solidcode/techcrunch-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "TechCrunch Articles Scraper", "description": "[πŸ’° $2.49 / 1K] Search and scrape TechCrunch articles by keyword. Extract headlines, article URLs, authors, categories, and publication dates as clean structured data β€” no API key needed. Filter by date, sort by newest or relevance, and export to JSON, CSV, or Excel.", "version": "1.0", "x-build-id": "BgdfXstLQLp7894dN" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/solidcode~techcrunch-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-solidcode-techcrunch-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/solidcode~techcrunch-scraper/runs": { "post": { "operationId": "runs-sync-solidcode-techcrunch-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/solidcode~techcrunch-scraper/run-sync": { "post": { "operationId": "run-sync-solidcode-techcrunch-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "keywords": { "title": "Search Keywords", "type": "array", "description": "Words or phrases to search TechCrunch for. Add one per line β€” each is searched separately (for example 'artificial intelligence', 'startup funding', 'cybersecurity'). Leave empty to collect the latest TechCrunch articles.", "items": { "type": "string" } }, "maxResults": { "title": "Max Results Per Keyword", "minimum": 0, "type": "integer", "description": "Maximum number of articles to collect for each keyword. For example, 3 keywords with a limit of 100 returns up to 300 articles. Set to 0 for no limit (collect everything available).", "default": 100 }, "sortBy": { "title": "Sort Order", "enum": [ "date", "relevance" ], "type": "string", "description": "Choose how matching articles are ordered before collection.", "default": "date" }, "publishedAfter": { "title": "Published After", "pattern": "^\\d{4}-\\d{2}-\\d{2}$", "type": "string", "description": "Only collect articles published on or after this date. Use the format YYYY-MM-DD, for example 2025-01-01. Leave empty for no lower limit. Filtering uses TechCrunch's own publication timezone, so articles right at the boundary may shift by a few hours in either direction β€” widen the range by a day if you need an exact cutoff." }, "publishedBefore": { "title": "Published Before", "pattern": "^\\d{4}-\\d{2}-\\d{2}$", "type": "string", "description": "Only collect articles published on or before this date. Use the format YYYY-MM-DD, for example 2025-12-31. Leave empty for no upper limit. Filtering uses TechCrunch's own publication timezone, so articles right at the boundary may shift by a few hours in either direction β€” widen the range by a day if you need an exact cutoff." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```