# GoodFirms Agency Directory Scraper (`ahmed_jasarevic/goodfirms-agency-scraper`) Actor

Extract B2B agencies and IT service providers from GoodFirms.co: name, rating, client reviews, hourly rate, employees, founded year, location, services, industry/client focus, website, phone, email and social links. For B2B lead-gen, agency list building and market research.

- **URL**: https://apify.com/ahmed\_jasarevic/goodfirms-agency-scraper.md
- **Developed by:** [Ahmed Jasarevic](https://apify.com/ahmed_jasarevic) (community)
- **Categories:** Lead generation, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## GoodFirms Agency Directory Scraper

### What does GoodFirms Agency Directory Scraper do?

This Actor extracts **B2B agencies and IT service providers from [GoodFirms.co](https://www.goodfirms.co)** — one of the largest B2B research and review platforms (150,000+ company profiles). It walks GoodFirms **directory listing pages** (e.g. `/directory/platforms/top-web-design-companies`) and **company profile pages** (`/company/<slug>`), pulling the same lead-gen data shape as Clutch and The Manifest: company name, rating, client reviews, hourly rate, employee count, founded year, location, services, industry/client focus, website, phone, email and social links.

The Actor runs on the **Apify platform**, so you get API access, scheduling, proxy rotation, monitoring and CSV/JSON/Excel export out of the box. It uses a **real browser (Playwright) + residential proxies** to pass GoodFirms' Cloudflare managed challenge automatically.

### Why use GoodFirms Agency Directory Scraper?

- **Build agency lists for outreach** — collect hundreds of vetted agencies with contact details, ratings and reviews in one run.
- **Market research** — analyze service focus, industry focus, client size focus, hourly rate bands and team sizes across a category.
- **Competitive intelligence** — track ratings, review counts and positioning of competitors over time.
- **Same data shape as Clutch/The Manifest** — drop-in replacement for your existing B2B lead-gen pipeline.

### How to use GoodFirms Agency Directory Scraper

1. Open the Actor in Apify Console and go to the **Input** tab.
2. Paste one or more **GoodFirms directory URLs** (e.g. `https://www.goodfirms.co/directory/platforms/top-web-design-companies`), or paste **direct company profile URLs** (e.g. `https://www.goodfirms.co/company/unified-infotech`).
3. Set `maxResults` (how many companies to save) and `maxPages` (how deep to paginate each directory).
4. Click **Start**. The Actor will walk the listings, follow each company to its profile, and save enriched records to the dataset.
5. Download the results as **JSON, CSV, HTML or Excel** from the Output tab, or pull them via the Apify API.

### Input

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `directoryUrls` | string\[] | `[]` | GoodFirms directory listing URLs to walk page-by-page. |
| `profileUrls` | string\[] | `[]` | Direct company profile URLs to scrape (use instead of directories for a target list). |
| `maxPages` | integer | `5` | Max listing pages per directory (~20 companies per page). |
| `maxResults` | integer | `100` | Max companies to save. Free users are capped at 10. |
| `scrapeProfiles` | boolean | `true` | Follow each company to its profile page for enriched data. |
| `proxy` | object | Residential | Apify proxy config. Residential is strongly recommended. |

Example input:

```json
{
  "directoryUrls": ["https://www.goodfirms.co/directory/platforms/top-web-design-companies"],
  "maxPages": 3,
  "maxResults": 50,
  "scrapeProfiles": true
}
```

### Output

Each dataset record is one company. Example:

```json
{
  "name": "Unified Infotech",
  "profileUrl": "https://www.goodfirms.co/company/unified-infotech",
  "website": "https://www.unifiedinfotech.net/",
  "rating": "4.9",
  "reviewCount": "30",
  "location": "Kolkata, West Bengal, India",
  "city": "Kolkata",
  "country": "in",
  "streetAddress": "DN-53, STP-II Building, 1st Floor, Sector-V, SaltLake",
  "postalCode": "700091",
  "foundedYear": "2010",
  "employees": "50 - 249",
  "hourlyRate": "$50 - $99/hr",
  "services": ["Mobile App Development", "Web Development", "Software Development"],
  "industryFocus": ["Education - 10%", "Fintech - 10%"],
  "clientFocus": ["40% Small Business", "50% Midmarket Business"],
  "email": "contacts@unifiedinfotech.net",
  "phone": "+9193309 01942",
  "socialLinks": ["https://www.linkedin.com/company/unifiedinfotech"],
  "reviews": [
    {
      "title": "Absolutely magnificent",
      "body": "been a pleasure! Anuj, Pampa, Priyam...",
      "date": "2025-12-22T05:57:17Z",
      "author": "Kritvi Rana",
      "rating": "5.0"
    }
  ],
  "scrapedAt": "2026-08-14T08:00:00.000Z"
}
```

You can download the dataset in various formats such as **JSON, HTML, CSV, or Excel**.

### Data table

| Field | Description |
| --- | --- |
| `name` | Company name |
| `profileUrl` | GoodFirms profile URL |
| `website` | Company website |
| `rating` | Average review rating (1-5) |
| `reviewCount` | Number of reviews |
| `reviews` | Client reviews with title, body, date, author and rating |
| `location` / `city` / `country` / `streetAddress` / `postalCode` | Location data |
| `employees` | Team size bracket (e.g. "50 - 249") |
| `hourlyRate` | Hourly rate bracket (e.g. "$50 - $99/hr") |
| `foundedYear` | Year founded |
| `services` | Services offered |
| `industryFocus` | Industry focus with percentages |
| `clientFocus` | Client size focus with percentages |
| `email` / `phone` | Contact details from the profile |
| `socialLinks` | LinkedIn, Facebook, Twitter etc. |
| `portfolioIndustries` | Project industry breakdown from the profile portfolio analytics |

### Pricing / Cost estimation

GoodFirms is behind a **Cloudflare managed challenge**, so every page requires a real browser and a residential proxy IP. Expect roughly **1-2 compute units per 100 companies** depending on how many profiles you follow. Free users get a capped preview of 10 companies. For large runs, keep `maxResults` reasonable and use `scrapeProfiles: false` for a cheaper listing-only pass first.

### Tips or Advanced options

- **Find directory URLs**: browse GoodFirms, open a category (e.g. "Top Web Design Companies"), and copy the URL. The actor handles any `/directory/...` listing.
- **Cheaper runs**: set `scrapeProfiles: false` to get listing-level data (name, rating, location, founded, employees, website) without visiting every profile.
- **Bigger runs**: raise `maxPages` and `maxResults` — the actor hard-caps requests and concurrency so it can't run away.
- **Direct lists**: if you already have a list of company slugs, feed `profileUrls` directly — no directory walking needed.

### FAQ, disclaimers, and support

**Is scraping GoodFirms legal?** Scraping publicly available business listing data for lead-gen is generally acceptable, but respect GoodFirms' Terms of Service, robots.txt, and applicable data-protection laws (GDPR, CAN-SPAM). Do not redistribute review text wholesale.

**Known limitations**: GoodFirms rate-limits aggressively; residential proxies are required. Reviews in the dataset are the ones rendered on the profile page (typically the most recent 5-15). Contact emails/phones come from the public profile data.

For issues or feature requests, open a ticket in the **Issues** tab. Need a custom version (e.g. software-product scraping, review pagination, or website contact enrichment)? Contact the author for a tailored solution.

# Actor input Schema

## `directoryUrls` (type: `array`):

GoodFirms.co directory listing URLs to scrape, e.g. https://www.goodfirms.co/directory/platforms/top-web-design-companies or https://www.goodfirms.co/directory/cms/top-website-development-companies. The actor walks each directory page-by-page and collects every company card.

## `profileUrls` (type: `array`):

Optional: specific GoodFirms company profile URLs to scrape directly, e.g. https://www.goodfirms.co/company/unified-infotech. Use this instead of directory URLs when you already have a target list.

## `maxPages` (type: `integer`):

Maximum number of listing pages to walk per directory URL (~20 companies per page).

## `maxResults` (type: `integer`):

Maximum number of companies to save. Free users are capped at 10 regardless of this value.

## `scrapeProfiles` (type: `boolean`):

Follow each company to its GoodFirms profile page to enrich the record with description, hourly rate, employees, services, industry/client focus, contact details and client reviews. Disable for faster/cheaper runs (listing-only data).

## `proxy` (type: `object`):

Apify Proxy with Residential IPs is strongly recommended - goodfirms.co sits behind a Cloudflare managed challenge and datacenter IPs get flagged fast.

## Actor input object example

```json
{
  "directoryUrls": [
    "https://www.goodfirms.co/directory/platforms/top-web-design-companies"
  ],
  "profileUrls": [],
  "maxPages": 5,
  "maxResults": 100,
  "scrapeProfiles": false,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "directoryUrls": [
        "https://www.goodfirms.co/directory/platforms/top-web-design-companies"
    ],
    "profileUrls": [],
    "proxy": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("ahmed_jasarevic/goodfirms-agency-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "directoryUrls": ["https://www.goodfirms.co/directory/platforms/top-web-design-companies"],
    "profileUrls": [],
    "proxy": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("ahmed_jasarevic/goodfirms-agency-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "directoryUrls": [
    "https://www.goodfirms.co/directory/platforms/top-web-design-companies"
  ],
  "profileUrls": [],
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call ahmed_jasarevic/goodfirms-agency-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ahmed_jasarevic/goodfirms-agency-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/GVhU4xJgUYlFdIStY/builds/vYh5haFtztfpfy44e/openapi.json