GoodFirms Agency Directory Scraper

What does GoodFirms Agency Directory Scraper do?

This Actor extracts B2B agencies and IT service providers from GoodFirms.co — one of the largest B2B research and review platforms (150,000+ company profiles). It walks GoodFirms directory listing pages (e.g. /directory/platforms/top-web-design-companies) and company profile pages (/company/<slug>), pulling the same lead-gen data shape as Clutch and The Manifest: company name, rating, client reviews, hourly rate, employee count, founded year, location, services, industry/client focus, website, phone, email and social links.

The Actor runs on the Apify platform, so you get API access, scheduling, proxy rotation, monitoring and CSV/JSON/Excel export out of the box. It uses a real browser (Playwright) + residential proxies to pass GoodFirms' Cloudflare managed challenge automatically.

Why use GoodFirms Agency Directory Scraper?

• Build agency lists for outreach — collect hundreds of vetted agencies with contact details, ratings and reviews in one run.
• Market research — analyze service focus, industry focus, client size focus, hourly rate bands and team sizes across a category.
• Competitive intelligence — track ratings, review counts and positioning of competitors over time.
• Same data shape as Clutch/The Manifest — drop-in replacement for your existing B2B lead-gen pipeline.

How to use GoodFirms Agency Directory Scraper

1. Open the Actor in Apify Console and go to the Input tab.
2. Paste one or more GoodFirms directory URLs (e.g. https://www.goodfirms.co/directory/platforms/top-web-design-companies), or paste direct company profile URLs (e.g. https://www.goodfirms.co/company/unified-infotech).
3. Set maxResults (how many companies to save) and maxPages (how deep to paginate each directory).
4. Click Start. The Actor will walk the listings, follow each company to its profile, and save enriched records to the dataset.
5. Download the results as JSON, CSV, HTML or Excel from the Output tab, or pull them via the Apify API.

Input

Field	Type	Default	Description
directoryUrls	string[]	[]	GoodFirms directory listing URLs to walk page-by-page.
profileUrls	string[]	[]	Direct company profile URLs to scrape (use instead of directories for a target list).
maxPages	integer	5	Max listing pages per directory (~20 companies per page).
maxResults	integer	100	Max companies to save. Free users are capped at 10.
scrapeProfiles	boolean	true	Follow each company to its profile page for enriched data.
proxy	object	Residential	Apify proxy config. Residential is strongly recommended.

Example input:

{
  "directoryUrls": ["https://www.goodfirms.co/directory/platforms/top-web-design-companies"],
  "maxPages": 3,
  "maxResults": 50,
  "scrapeProfiles": true
}

Output

Each dataset record is one company. Example:

{
  "name": "Unified Infotech",
  "profileUrl": "https://www.goodfirms.co/company/unified-infotech",
  "website": "https://www.unifiedinfotech.net/",
  "rating": "4.9",
  "reviewCount": "30",
  "location": "Kolkata, West Bengal, India",
  "city": "Kolkata",
  "country": "in",
  "streetAddress": "DN-53, STP-II Building, 1st Floor, Sector-V, SaltLake",
  "postalCode": "700091",
  "foundedYear": "2010",
  "employees": "50 - 249",
  "hourlyRate": "$50 - $99/hr",
  "services": ["Mobile App Development", "Web Development", "Software Development"],
  "industryFocus": ["Education - 10%", "Fintech - 10%"],
  "clientFocus": ["40% Small Business", "50% Midmarket Business"],
  "email": "contacts@unifiedinfotech.net",
  "phone": "+9193309 01942",
  "socialLinks": ["https://www.linkedin.com/company/unifiedinfotech"],
  "reviews": [
    {
      "title": "Absolutely magnificent",
      "body": "been a pleasure! Anuj, Pampa, Priyam...",
      "date": "2025-12-22T05:57:17Z",
      "author": "Kritvi Rana",
      "rating": "5.0"
    }
  ],
  "scrapedAt": "2026-08-14T08:00:00.000Z"
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data table

Field	Description
name	Company name
profileUrl	GoodFirms profile URL
website	Company website
rating	Average review rating (1-5)
reviewCount	Number of reviews
reviews	Client reviews with title, body, date, author and rating
location / city / country / streetAddress / postalCode	Location data
employees	Team size bracket (e.g. "50 - 249")
hourlyRate	Hourly rate bracket (e.g. "$50 - $99/hr")
foundedYear	Year founded
services	Services offered
industryFocus	Industry focus with percentages
clientFocus	Client size focus with percentages
email / phone	Contact details from the profile
socialLinks	LinkedIn, Facebook, Twitter etc.
portfolioIndustries	Project industry breakdown from the profile portfolio analytics

Pricing / Cost estimation

GoodFirms is behind a Cloudflare managed challenge, so every page requires a real browser and a residential proxy IP. Expect roughly 1-2 compute units per 100 companies depending on how many profiles you follow. Free users get a capped preview of 10 companies. For large runs, keep maxResults reasonable and use scrapeProfiles: false for a cheaper listing-only pass first.

Tips or Advanced options

• Find directory URLs: browse GoodFirms, open a category (e.g. "Top Web Design Companies"), and copy the URL. The actor handles any /directory/... listing.
• Cheaper runs: set scrapeProfiles: false to get listing-level data (name, rating, location, founded, employees, website) without visiting every profile.
• Bigger runs: raise maxPages and maxResults — the actor hard-caps requests and concurrency so it can't run away.
• Direct lists: if you already have a list of company slugs, feed profileUrls directly — no directory walking needed.

FAQ, disclaimers, and support

Is scraping GoodFirms legal? Scraping publicly available business listing data for lead-gen is generally acceptable, but respect GoodFirms' Terms of Service, robots.txt, and applicable data-protection laws (GDPR, CAN-SPAM). Do not redistribute review text wholesale.

Known limitations: GoodFirms rate-limits aggressively; residential proxies are required. Reviews in the dataset are the ones rendered on the profile page (typically the most recent 5-15). Contact emails/phones come from the public profile data.

For issues or feature requests, open a ticket in the Issues tab. Need a custom version (e.g. software-product scraping, review pagination, or website contact enrichment)? Contact the author for a tailored solution.