# Techreviewer Agency Directory Scraper (`automation-lab/techreviewer-agency-directory-scraper`) Actor Extract public Techreviewer agency listings for prospecting, vendor research, and competitor mapping. - **URL**: https://apify.com/automation-lab/techreviewer-agency-directory-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Business - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Techreviewer Agency Directory Scraper Export public agency records from Techreviewer category and location directories. ### What does it do? This scraper turns public Techreviewer agency-directory pages into structured data. It is designed for repeatable vendor research and agency prospecting. ### Who is it for? #### B2B sales and partnerships teams Build category-specific agency lead lists with website, location, team-size, rate, and service signals. Export the results to a CRM, prioritize accounts, and retain the profile URL as evidence for each prospect. #### Procurement and vendor-management teams Compare providers in a defined Techreviewer category or geography. Use ratings, project-size signals, hourly-rate bands, and listed services to build a reproducible vendor shortlist instead of reviewing pages manually. #### Agency strategy and competitive-intelligence teams Map competitors by service category or location, then repeat the same directory input later to track coverage changes. Deduplicated profile URLs and `scrapedAt` timestamps make snapshots easy to compare. #### Market-research and data-operations teams Maintain a public vendor database with normalized agency records. Keep `sourceUrl` and `profileUrl` alongside exports so analysts can trace every record back to its public directory listing. ### Why use it? Manual directory review is slow and hard to reproduce. This actor returns one normalized record per agency with the source URL retained. ### Supported source pages Use public Techreviewer category or location directory URLs. Example: `https://techreviewer.co/top-web-development-companies` Do not provide company profile URLs as start URLs. ### Data returned | Field | Meaning | | --- | --- | | `rank` | Visible directory position | | `agencyName` | Agency name | | `profileUrl` | Techreviewer profile URL | | `website` | Public agency website link | | `location` | Visible location text | | `hourlyRate` | Visible hourly-rate band | | `employeeBand` | Visible team-size band | | `minimumProjectSize` | Visible project-size signal | | `rating` | Visible directory rating | | `services` | Listed service allocation labels | | `description` | Public agency description | | `sourceUrl` | Directory page that supplied the record | | `scrapedAt` | UTC extraction timestamp | ### Quick start 1. Add a public Techreviewer directory URL. 2. Keep the initial item limit low. 3. Run the actor. 4. Export the dataset as JSON, CSV, or Excel. ### Input: directory URLs `startUrls` accepts one or more category or location directory pages. Each URL is crawled independently. Duplicate agency profile URLs are emitted once per run. ### Input: maximum agencies `maxItems` limits total unique records across all URLs. Use 25 for a quick validation run. Increase it when your workflow is verified. ### Input: maximum pages `maxPages` controls pagination per directory URL. A value of 1 fetches only the supplied directory page. ### Output quality Fields visible in the directory are extracted as provided. Unavailable fields are `null` rather than guessed. Service labels remain arrays so multiple services are preserved. ### Pagination Techreviewer listing pagination is followed up to `maxPages`. The actor stops early when the directory has no next page. ### Pricing ### How much does it cost to scrape Techreviewer agency directories? The actor charges a small one-time run fee and per returned agency record. Per-record pricing uses Apify usage tiers. Check the actor pricing panel for the current tier price before a large run. ### Practical workflow: prospecting Start with a category such as web development. Export agency websites, locations, and employee bands. Use those signals to qualify outreach lists in your CRM. ### Practical workflow: vendor discovery Run targeted country or category directories. Compare rating, project-size, rate, and service signals. Keep `sourceUrl` in your research sheet for auditability. ### Practical workflow: competitor mapping Run a core category and a geographic variation. Deduplicate on `profileUrl`. Track repeated snapshots by `scrapedAt`. ### Integrations Send exported results to a CRM for agency lead enrichment. Use Make or Zapier to load dataset rows into a spreadsheet. Use the Apify API for scheduled vendor-list refreshes. ### API usage: Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/techreviewer-agency-directory-scraper').call({ startUrls: [{ url: 'https://techreviewer.co/top-web-development-companies' }], maxItems: 25, maxPages: 1, }); console.log(await client.dataset(run.defaultDatasetId).listItems()); ```` ### API usage: Python ```python from apify_client import ApifyClient client = ApifyClient() run = client.actor('automation-lab/techreviewer-agency-directory-scraper').call(run_input={ 'startUrls': [{'url': 'https://techreviewer.co/top-web-development-companies'}], 'maxItems': 25, 'maxPages': 1, }) print(client.dataset(run['defaultDatasetId']).list_items().items) ``` ### API usage: cURL ```bash curl -X POST 'https://api.apify.com/v2/acts/automation-lab~techreviewer-agency-directory-scraper/runs?token=TOKEN' \ -H 'content-type: application/json' \ -d '{"startUrls":[{"url":"https://techreviewer.co/top-web-development-companies"}],"maxItems":25,"maxPages":1}' ``` ### MCP Add the actor-scoped HTTP tool in Claude Code: ```bash claude mcp add --transport http techreviewer-agencies https://mcp.apify.com?tools=automation-lab/techreviewer-agency-directory-scraper ``` This scoped connection exposes this actor's automation tool without requiring a local npx server. Claude Desktop configuration: ```json {"mcpServers":{"apify":{"command":"npx","args":["-y","@apify/mcp-server"]}}} ``` #### Example prompt workflows - “Use the Techreviewer Agency Directory Scraper MCP tool to extract 25 agencies from the web development directory, then return agency name, website, location, and hourly rate.” - “Run the Techreviewer Agency Directory Scraper for Germany and summarize the highest-rated agencies with their profile URLs.” - “Create a 100-record Techreviewer vendor map from web-development and UK directory pages; retain source URLs for every result.” MCP tools: `https://mcp.apify.com?tools=automation-lab/techreviewer-agency-directory-scraper` ### Legality and responsible use Only collect public data you are authorized to use. Respect Techreviewer terms, applicable privacy rules, and recipient preferences. Do not use personal contact data for unlawful or unsolicited outreach. ### Limitations The actor extracts fields rendered on public directory pages. Directory layouts and availability can change without notice. Rate limits or anti-bot responses may reduce coverage. ### Tips Use a narrow location directory for focused lists. Preserve `profileUrl` as your stable Techreviewer reference. Use `maxPages` and `maxItems` to control scope and cost. ### Troubleshooting: no results Confirm the supplied URL is a public Techreviewer listing page. Review the run log for a non-200 response or a layout change. ### Troubleshooting: fewer results than expected The actor deduplicates repeated profile URLs. A low `maxItems` or `maxPages` also stops collection intentionally. ### FAQ: does it scrape private contact details? No. It is limited to public agency-directory data rendered by Techreviewer. ### FAQ: can I use more than one directory? Yes. Add multiple URLs in `startUrls`; duplicate profiles are removed. ### FAQ: are all fields always populated? No. Techreviewer may omit rate, team-size, rating, or website fields for some listings. ### Related scrapers For a different directory source, review Automation Lab agency and lead scrapers. Choose the source that matches your buyer’s research workflow. ### Changelog See [CHANGELOG](.actor/CHANGELOG.md) for user-visible release notes. ### Support Include the input URL and a run-log excerpt when reporting a layout issue. This makes source-specific fixes faster to verify. # Actor input Schema ## `startUrls` (type: `array`): One or more public Techreviewer category or location listing URLs. Do not add company profile URLs. ## `maxItems` (type: `integer`): Stop after this many unique agency records. Start small while validating a new workflow. ## `maxPages` (type: `integer`): How many paginated listing pages to visit for each start URL. ## Actor input object example ```json { "startUrls": [ { "url": "https://techreviewer.co/top-web-development-companies" } ], "maxItems": 20, "maxPages": 1 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://techreviewer.co/top-web-development-companies" } ], "maxItems": 20, "maxPages": 1 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/techreviewer-agency-directory-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://techreviewer.co/top-web-development-companies" }], "maxItems": 20, "maxPages": 1, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/techreviewer-agency-directory-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://techreviewer.co/top-web-development-companies" } ], "maxItems": 20, "maxPages": 1 }' | apify call automation-lab/techreviewer-agency-directory-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/techreviewer-agency-directory-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Techreviewer Agency Directory Scraper", "description": "Extract public Techreviewer agency listings for prospecting, vendor research, and competitor mapping.", "version": "0.1", "x-build-id": "nXurSIENai1lQkTCJ" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~techreviewer-agency-directory-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-techreviewer-agency-directory-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~techreviewer-agency-directory-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-techreviewer-agency-directory-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~techreviewer-agency-directory-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-techreviewer-agency-directory-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "🔗 Techreviewer directory URLs", "type": "array", "description": "One or more public Techreviewer category or location listing URLs. Do not add company profile URLs.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxItems": { "title": "🏢 Maximum agencies", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Stop after this many unique agency records. Start small while validating a new workflow.", "default": 20 }, "maxPages": { "title": "📄 Maximum directory pages per URL", "minimum": 1, "maximum": 500, "type": "integer", "description": "How many paginated listing pages to visit for each start URL.", "default": 1 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```