Techreviewer Agency Directory Scraper

Export public agency records from Techreviewer category and location directories.

What does it do?

This scraper turns public Techreviewer agency-directory pages into structured data.

It is designed for repeatable vendor research and agency prospecting.

Who is it for?

B2B sales and partnerships teams

Build category-specific agency lead lists with website, location, team-size, rate, and service signals. Export the results to a CRM, prioritize accounts, and retain the profile URL as evidence for each prospect.

Procurement and vendor-management teams

Compare providers in a defined Techreviewer category or geography. Use ratings, project-size signals, hourly-rate bands, and listed services to build a reproducible vendor shortlist instead of reviewing pages manually.

Agency strategy and competitive-intelligence teams

Map competitors by service category or location, then repeat the same directory input later to track coverage changes. Deduplicated profile URLs and scrapedAt timestamps make snapshots easy to compare.

Market-research and data-operations teams

Maintain a public vendor database with normalized agency records. Keep sourceUrl and profileUrl alongside exports so analysts can trace every record back to its public directory listing.

Why use it?

Manual directory review is slow and hard to reproduce.

This actor returns one normalized record per agency with the source URL retained.

Supported source pages

Use public Techreviewer category or location directory URLs.

Example: https://techreviewer.co/top-web-development-companies

Do not provide company profile URLs as start URLs.

Data returned

Field	Meaning
rank	Visible directory position
agencyName	Agency name
profileUrl	Techreviewer profile URL
website	Public agency website link
location	Visible location text
hourlyRate	Visible hourly-rate band
employeeBand	Visible team-size band
minimumProjectSize	Visible project-size signal
rating	Visible directory rating
services	Listed service allocation labels
description	Public agency description
sourceUrl	Directory page that supplied the record
scrapedAt	UTC extraction timestamp

Quick start

1. Add a public Techreviewer directory URL.
2. Keep the initial item limit low.
3. Run the actor.
4. Export the dataset as JSON, CSV, or Excel.

Input: directory URLs

startUrls accepts one or more category or location directory pages.

Each URL is crawled independently.

Duplicate agency profile URLs are emitted once per run.

Input: maximum agencies

maxItems limits total unique records across all URLs.

Use 25 for a quick validation run.

Increase it when your workflow is verified.

Input: maximum pages

maxPages controls pagination per directory URL.

A value of 1 fetches only the supplied directory page.

Output quality

Fields visible in the directory are extracted as provided.

Unavailable fields are null rather than guessed.

Service labels remain arrays so multiple services are preserved.

Pagination

Techreviewer listing pagination is followed up to maxPages.

The actor stops early when the directory has no next page.

Pricing

How much does it cost to scrape Techreviewer agency directories?

The actor charges a small one-time run fee and per returned agency record.

Per-record pricing uses Apify usage tiers.

Check the actor pricing panel for the current tier price before a large run.

Practical workflow: prospecting

Start with a category such as web development.

Export agency websites, locations, and employee bands.

Use those signals to qualify outreach lists in your CRM.

Practical workflow: vendor discovery

Run targeted country or category directories.

Compare rating, project-size, rate, and service signals.

Keep sourceUrl in your research sheet for auditability.

Practical workflow: competitor mapping

Run a core category and a geographic variation.

Deduplicate on profileUrl.

Track repeated snapshots by scrapedAt.

Integrations

Send exported results to a CRM for agency lead enrichment.

Use Make or Zapier to load dataset rows into a spreadsheet.

Use the Apify API for scheduled vendor-list refreshes.

API usage: Node.js

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/techreviewer-agency-directory-scraper').call({
  startUrls: [{ url: 'https://techreviewer.co/top-web-development-companies' }],
  maxItems: 25,
  maxPages: 1,
});
console.log(await client.dataset(run.defaultDatasetId).listItems());

API usage: Python

from apify_client import ApifyClient
client = ApifyClient()
run = client.actor('automation-lab/techreviewer-agency-directory-scraper').call(run_input={
    'startUrls': [{'url': 'https://techreviewer.co/top-web-development-companies'}],
    'maxItems': 25, 'maxPages': 1,
})
print(client.dataset(run['defaultDatasetId']).list_items().items)

API usage: cURL

curl -X POST 'https://api.apify.com/v2/acts/automation-lab~techreviewer-agency-directory-scraper/runs?token=TOKEN' \
  -H 'content-type: application/json' \
  -d '{"startUrls":[{"url":"https://techreviewer.co/top-web-development-companies"}],"maxItems":25,"maxPages":1}'

MCP

Add the actor-scoped HTTP tool in Claude Code:

$claude mcp add --transport http techreviewer-agencies https://mcp.apify.com?tools=automation-lab/techreviewer-agency-directory-scraper

This scoped connection exposes this actor's automation tool without requiring a local npx server.

Claude Desktop configuration:

{"mcpServers":{"apify":{"command":"npx","args":["-y","@apify/mcp-server"]}}}

Example prompt workflows

• “Use the Techreviewer Agency Directory Scraper MCP tool to extract 25 agencies from the web development directory, then return agency name, website, location, and hourly rate.”
• “Run the Techreviewer Agency Directory Scraper for Germany and summarize the highest-rated agencies with their profile URLs.”
• “Create a 100-record Techreviewer vendor map from web-development and UK directory pages; retain source URLs for every result.”

MCP tools: https://mcp.apify.com?tools=automation-lab/techreviewer-agency-directory-scraper

Legality and responsible use

Only collect public data you are authorized to use.

Respect Techreviewer terms, applicable privacy rules, and recipient preferences.

Do not use personal contact data for unlawful or unsolicited outreach.

Limitations

The actor extracts fields rendered on public directory pages.

Directory layouts and availability can change without notice.

Rate limits or anti-bot responses may reduce coverage.

Tips

Use a narrow location directory for focused lists.

Preserve profileUrl as your stable Techreviewer reference.

Use maxPages and maxItems to control scope and cost.

Troubleshooting: no results

Confirm the supplied URL is a public Techreviewer listing page.

Review the run log for a non-200 response or a layout change.

Troubleshooting: fewer results than expected

The actor deduplicates repeated profile URLs.

A low maxItems or maxPages also stops collection intentionally.

FAQ: does it scrape private contact details?

No. It is limited to public agency-directory data rendered by Techreviewer.

FAQ: can I use more than one directory?

Yes. Add multiple URLs in startUrls; duplicate profiles are removed.

FAQ: are all fields always populated?

No. Techreviewer may omit rate, team-size, rating, or website fields for some listings.

Related scrapers

For a different directory source, review Automation Lab agency and lead scrapers.

Choose the source that matches your buyer’s research workflow.

Changelog

See .actor/CHANGELOG.md for user-visible release notes.

Support

Include the input URL and a run-log excerpt when reporting a layout issue.

This makes source-specific fixes faster to verify.