# GoodFirms B2B Companies & Software Scraper (`haketa/goodfirms-scraper`) Actor Extract B2B agency listings (web dev, mobile, marketing, design, blockchain, AI) and software products (CRM, PM, HR…) from GoodFirms.co — with hourly rates, employees, hourly rate, services, ratings, verified reviews, and contact data. No API key required. - **URL**: https://apify.com/haketa/goodfirms-scraper.md - **Developed by:** [Haketa](https://apify.com/haketa) (community) - **Categories:** Lead generation, Developer tools, Jobs - **Stats:** 2 total users, 1 monthly users, 25.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $2.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GoodFirms Scraper — B2B Agencies, Software & Verified Reviews **Turn the world's largest B2B services directory into a ready-to-use sales pipeline.** Extract **110,000+ verified agencies** and **thousands of SaaS products** from GoodFirms — with hourly rates, employee counts, founded year, headquarters, contact emails & phones, verified client reviews, service breakdowns, and more — in clean JSON / CSV / Excel. No login. No manual exports. No copy-pasting. Bulk lead generation for sales teams, market researchers, recruiters, agency analysts, and SaaS founders. --- ### ⭐ Why use this scraper GoodFirms is one of the top three B2B agency directories on the planet (alongside Clutch and DesignRush), used by **enterprise buyers, procurement teams, and founders** to evaluate IT services, marketing, design, and software vendors. The data sitting inside it is gold for: - **Outbound sales teams** — get pre-qualified agency leads with budget signals (hourly rate range, minimum project size, employee count) **plus contact emails and phone numbers in the same record**. - **Recruiters & talent sourcers** — find specialized agencies hiring in your stack, with employee bands and tech skills surfaced. - **Market researchers & analysts** — segment the global agency landscape by service, country, employee size, and rating. - **SaaS founders** — identify trending software competitors per category, with rating, reviews, and feature lists. - **Agency owners & marketers** — benchmark yourself against competitors, build prospect lists of agencies who buy services like yours. - **Investors & M&A teams** — get a pipeline of acquisition targets sortable by region, revenue proxies, and review momentum. If you've ever paid for ZoomInfo, Apollo, Crunchbase, or Clay just to get B2B agency leads — this scraper gives you a fraction of that cost per record, with **agency-specific signals those platforms don't carry** (hourly rates, service-share breakdown, project minimums, client logos, AI tool stack). --- ### 🔑 SEO / keyword reference This scraper is a perfect fit for these workflows: **B2B lead generation**, **agency database**, **outbound sales pipeline**, **ABM (Account-Based Marketing) targeting**, **agency directory scraper**, **IT services leads**, **web development companies database**, **mobile app development agencies**, **digital marketing agency leads**, **SaaS competitive intelligence**, **agency contact extraction**, **email + phone enrichment**, **agency reviews scraping**, **GoodFirms data export**, **enterprise sales prospecting**, **lookalike targeting**, **GTM (go-to-market) intelligence**. --- ### 🎯 What you get — per agency #### 🟢 Identity & contact - **Agency name** + URL-safe slug - **Logo URL** (high-resolution) - **Agency website** (UTM-stripped — no tracking junk) - **GoodFirms profile URL** (canonical) - **Tagline** + **long description** (about / company summary) - **Email** ✉️ — public business email - **Phone** ☎️ — public business phone - **Social links** — LinkedIn, Twitter / X, Facebook, Instagram #### 🟢 Lead-gen meta (this is the gold) - **Hourly rate range** — e.g. `$25–$49/hr`, `$50–$99/hr`, `$100–$149/hr`, `$150–$199/hr`, `$200+/hr` - **Employees range** — e.g. `2–9`, `10–49`, `50–249`, `250–999`, `1,000–9,999`, `10,000+` - **Founded year** — when the agency was established - **Headquarters** — full structured address: city, state/region, country, street - **Aggregate rating** (0–5) + **total verified review count** #### 🟢 Service & focus - **Top-level service categories** — e.g. "Mobile App Development", "Web Development", "Digital Marketing", "AI", "Blockchain", "IT Services" - **Service-share breakdown** — sub-services with **percentage allocation** (e.g. "iOS – 10%, Android – 10%, Web Apps – 15%") - **Client-focus split** — Small Business / Mid-market / Enterprise share with % - **Top review highlight** — the featured client quote + author surfaced on the listing card #### 🟢 Reviews (opt-in) - Full verified reviews when `scrapeReviews=true` - Review rating, project type, project cost band, duration, reviewer name + role + company, summary, date #### 🟡 Bonus signals (when the agency claims & fills its profile — coverage varies) - Additional office locations - AI tools the team is using (GitHub Copilot, Cursor, ChatGPT, Claude, etc.) - "Why choose us" bullet points - Minimum project budget, industries served, skills list, listed clients, portfolio count #### 🟢 Per software product (when `mode=software`) - Product name, vendor (link to vendor profile), logo, website - Pricing model + starting price + free trial / free tier flags - Deployment options (Cloud, On-Premise, Mobile) + supported devices + feature list - Categories + alternatives (competing products) - Rating, reviews --- ### 💰 Real-world use cases — money-making scenarios #### 1. Build a 10,000-lead outbound list for a B2B SaaS product You sell a productivity tool aimed at agencies with 50–250 employees in the US. Run: ```json { "mode": "companies", "serviceCategorySlugs": ["web-development-agency", "digital-marketing-agency"], "countrySlugs": ["us"], "employeesFilter": "50-249", "maxRecords": 10000, "scrapeReviews": false } ```` You get **10,000 agencies** with email, phone, hourly rate, employees, HQ — ready to import into Apollo, Smartlead, Lemlist, or Salesforce. #### 2. Hand-picked enterprise sales targets You're an enterprise AE prospecting top-rated agencies for ABM. Run with quality filters: ```json { "serviceCategorySlugs": ["software-development", "mobile-app-development"], "countrySlugs": ["us", "gb", "ca"], "employeesFilter": "250-999", "hourlyRateFilter": "150-199", "minReviews": 10, "scrapeReviews": true } ``` You get the **decile of top-tier agencies** with high reviews, premium rates, full review bodies for personalized outreach. #### 3. Competitive intelligence — track an agency vertical over time You run a managed service competing with mid-market mobile agencies. Schedule a weekly run on `mobile-app-development` + `digital-marketing-agency`, compare deltas → know when competitors get new reviews, raise their rates, or grow team size. #### 4. SaaS "alternatives to X" SEO pages You sell project management software. Pull every CRM and PM software listing → use the data to build hundreds of programmatic SEO pages ("Best Asana Alternatives", "Best Trello Alternatives") with ratings, prices, feature lists. Affiliate revenue meets SEO. #### 5. Agency recruiting — find specialized teams You're a CTO looking to outsource a specific stack. Run with `serviceCategorySlugs: ["ai-development", "blockchain-development"]` + `countrySlugs: ["ua", "pl", "in"]` → get a shortlist of specialized agencies with budgets you can afford. #### 6. M\&A / investment screening You're a PE firm scouting agency acquisitions. Pull every agency with `employees 50-249`, `rating 4.5+`, `reviews 20+` across multiple service categories → instant longlist with revenue proxies (hourly rate × employees). #### 7. Market research / TAM analysis You're sizing the global web-design market for a board deck. Pull all agencies per country → segment by employees + hourly rate → compute TAM bands by region. One run replaces weeks of analyst work. #### 8. Agency lookalike modeling You have 50 happy customers — all are mid-market agencies. Pull every similar agency by employees + service mix + country → instant lookalike pipeline. *** ### ⚙️ Input — what you give | Field | Type | What it does | Example | |---|---|---|---| | `mode` | enum | `auto` / `companies` / `software` — choose which side of the directory to scrape | `companies` | | `startUrls` | array | Any GoodFirms URL: service category, country page, single profile, software detail. Mixed types OK. | `["https://www.goodfirms.co/companies/web-development-agency/in"]` | | `serviceCategorySlugs` | array | Discover agencies from these service categories | `["web-development-agency", "mobile-app-development"]` | | `countrySlugs` | array | Restrict to specific country / state / city slugs | `["us", "in", "gb", "california", "new-york"]` | | `softwareCategorySlugs` | array | Discover software products from these categories | `["crm-software", "project-management-software"]` | | `companySlugs` | array | Scrape specific agency profiles directly | `["spconsulting", "sdlc-corp"]` | | `softwareSlugs` | array | Scrape specific software products directly | `["agile-crm", "notion"]` | | `employeesFilter` | enum | Restrict to a team-size band | `"50-249"` | | `hourlyRateFilter` | enum | Restrict to an hourly-rate band | `"25-49"` | | `minReviews` | int | Minimum verified review count | `5` | | `scrapeReviews` | boolean | Include full client reviews in each record | `true` | | `scrapeProfileDetails` | boolean | Visit each profile for richer data (turn off for ultra-fast bulk discovery) | `true` | | `maxRecords` | int | Total cap (0 = unlimited) | `100` | | `maxPagesPerCategory` | int | Pagination depth per category × country combo | `5` | | `requestDelay` | int (ms) | Politeness delay between requests | `1200` | | `maxConcurrency` | int | Parallel request count | `2` | | `proxyConfiguration` | object | Proxy settings — defaults provided | *(see Apify proxy editor)* | *** ### 📤 Output — what you get Every run pushes records to a structured **dataset** that you can export from the Apify Console as: | Format | When to use | |---|---| | **JSON** | Direct API / programmatic ingestion, full nested data preserved | | **CSV** | Open in Excel, Google Sheets, import into CRM (Salesforce, HubSpot, Pipedrive) | | **XLSX** | Hand off to non-technical stakeholders (sales managers, marketing) | | **Companies view** | Compact one-row-per-agency lead-gen table (recommended for outbound) | | **Software view** | One-row-per-product table (for SaaS competitive intel) | | **Reviews view** | One row per (record × review) — perfect for sentiment analysis | The dataset is queryable via the Apify API — chain this scraper into your stack with one line. #### Example output — company record ```json { "recordType": "company", "slug": "example-agency", "profileUrl": "https://www.goodfirms.co/company/example-agency", "name": "Example Agency", "tagline": "Established in 2015, Example Agency is a trusted web development company...", "logoUrl": "https://assets.goodfirms.co/services/medium/example.png", "websiteUrl": "https://exampleagency.com/", "rating": 4.9, "reviewCount": 155, "foundedYear": 2015, "employeesRange": "250 - 999", "hourlyRateRange": "$25 - $49/hr", "headquarters": { "city": "San Francisco", "state": "California", "country": "us", "address": "44 Montgomery St" }, "contact": { "email": "hello@exampleagency.com", "phone": "+1-555-555-5555" }, "socialLinks": { "linkedin": "https://www.linkedin.com/company/example-agency", "twitter": "https://twitter.com/exampleagency", "facebook": "https://www.facebook.com/exampleagency", "instagram": null }, "topServices": ["Mobile App Development", "Web Development", "Software Development", "E-commerce Development", "Artificial Intelligence", "Digital Marketing"], "services": [ { "name": "iOS - iPhone", "percent": 10 }, { "name": "Android", "percent": 10 }, { "name": "Enterprise Apps", "percent": 4 }, { "name": "Web Apps", "percent": 10 } ], "focusBreakdown": [ { "name": "Small Business", "percent": 70 }, { "name": "Medium Business", "percent": 20 }, { "name": "Large Business", "percent": 10 } ], "topReview": { "quote": "Example Agency delivered a robust and user-friendly hospital app with professionalism.", "author": "Akansha Mishra, Software developer at MedCorp" }, "aiTools": ["GitHub Copilot", "Cursor", "ChatGPT", "Claude", "Perplexity"], "whyChoose": ["Full-cycle product engineering", "Dedicated teams, clear reporting", "Scalable systems, ongoing support"], "reviews": [], "discoveredFrom": "web-development-agency", "scrapedAt": "2026-05-19T14:00:00.000Z" } ``` #### Example output — software record ```json { "recordType": "software", "slug": "example-crm", "profileUrl": "https://www.goodfirms.co/software/example-crm", "name": "Example CRM", "vendor": { "name": "ExampleSoft Inc.", "profileUrl": "https://www.goodfirms.co/company/examplesoft" }, "rating": 4.6, "reviewCount": 312, "pricingModel": "Monthly Subscription", "startingPrice": "$15 / user / month", "freeTrial": true, "freeVersion": true, "deployment": ["Cloud", "Mobile"], "supportedDevices": ["Web", "iOS", "Android"], "features": ["Contact Management", "Sales Pipeline", "Email Marketing", "Reporting"], "categories": ["CRM Software"], "alternatives": [ { "slug": "competitor-crm", "name": "Competitor CRM" } ], "scrapedAt": "2026-05-19T14:00:00.000Z" } ``` *** ### 📊 Real-world fill rate Measured across a sample of **93 randomly-discovered agencies** with default settings: | Field group | Coverage | Notes | |---|---|---| | Identity (`name`, `logoUrl`, `websiteUrl`, `tagline`, `profileUrl`) | **100%** | UTM-stripped, ready to ship | | Aggregate rating | **100%** | Every agency has a rating | | Founded year | **99%** | Almost always present | | Employees range | **100%** | Every agency declares team size | | Hourly rate range | **100%** | Every agency declares a band | | Headquarters (city + state + country + address) | **100%** | Full structured address | | Email | **91%** | Public business email | | Phone | **98%** | Public business phone | | Top services + service-share breakdown | **97–98%** | With % allocation | | Client-focus split (SMB / Mid / Enterprise) | **97%** | With % allocation | | Top review highlight | **100%** | Featured client quote | | Description / tagline | **97% / 100%** | Long & short variants | | LinkedIn / Facebook / Twitter | **92% / 82% / 72%** | Per agency declaration | | Bonus fields (AI tools, why-choose, additional offices, min project, industries, skills) | **0–20%** | Filled only when the agency claims their profile | In short: **the lead-gen meta + contact + service breakdown is essentially guaranteed**. Bonus fields are upside when the agency has invested in their profile. *** ### 🌍 Geographic coverage Default discovery pulls a balanced global mix. Example country breakdown from a random web-development run: | Country | Share | |---|---| | United States 🇺🇸 | ~50% | | India 🇮🇳 | ~27% | | Poland 🇵🇱 | ~4% | | United Kingdom 🇬🇧 | ~4% | | Ukraine 🇺🇦 | ~3% | | Canada 🇨🇦 | ~2% | | Belgium / Germany / Spain / others | ~10% | Combine `serviceCategorySlugs` with `countrySlugs` for tight ICP segmentation. *** ### 💡 Tips & best practices #### Sales / outbound - **Start narrow, then scale.** Run with `maxRecords: 100` and your top 1–2 categories to validate the data before pulling 10,000+ leads. - **Always run `scrapeProfileDetails: true`** for outbound — the listing card alone misses email/phone/full address. - **Skip reviews for pure leadgen** (`scrapeReviews: false`) — saves time and cost, and you don't need them for cold email. - **Combine `employeesFilter` + `hourlyRateFilter` + `countrySlugs`** to land directly on your ICP segment. #### Competitive intelligence - Set `scrapeReviews: true` for full review bodies — gold for understanding competitor positioning. - Use the **reviews view** (CSV export) for sentiment analysis or LLM-based summaries. #### Programmatic SEO - Run `mode: software` with multiple software categories → use `name`, `tagline`, `pricingModel`, `startingPrice`, `features`, `alternatives` for landing pages. #### Recruiting & talent - Filter by employees + country + service — the **`topServices` and `services` breakdown** tell you exactly which tech stack each agency operates. #### Speed vs depth - `scrapeProfileDetails: false` → ultra-fast bulk discovery from listing cards only (~5× faster, cheaper). - `scrapeProfileDetails: true` (default) → full data per profile, recommended for paid outreach. #### Rate-limit safety - Keep `maxConcurrency` at 2 for the cleanest runs. Bumping to 5+ can trigger rate limits on big runs. - The default `requestDelay: 1200ms` is conservative — drop it to 800 if you're in a hurry. *** ### ❓ FAQ **Is the data fresh?** Every run pulls live data at the moment of execution. Schedule daily/weekly runs for change tracking. **Can I scrape just one agency?** Yes — pass it in `companySlugs: ["agency-slug"]` or include the full profile URL in `startUrls`. **Can I export to my CRM?** Yes — the dataset is exportable as CSV / XLSX / JSON and Apify's API lets you pipe it into HubSpot, Salesforce, Pipedrive, Apollo, Clay, Smartlead, Lemlist, or any custom system. **Does it scrape software, agencies, or both?** Both — set `mode: companies` (default), `mode: software`, or `mode: auto` (infer per URL). The dataset has a `recordType` discriminator field. **Will the same agency appear twice?** No — deduplication by slug across all discovery sources. **How do I sort by hourly rate or rating after I have the data?** Open in Excel / Google Sheets → sort by `rating`, `hourlyRateRange`, `reviewCount`, or `employeesRange`. Or query via the Apify API. **Can I get only US agencies?** Yes — set `countrySlugs: ["us"]` plus optional state slugs. **Can I limit by review count or rating?** Yes — `minReviews: 5` (server-side filter) + post-filter on rating. **Can I schedule recurring runs?** Absolutely — Apify's built-in scheduler runs this scraper on any cron expression. **Is there a free tier?** The first 100 records are free — enough to test the data on your real ICP before scaling. *** ### 💸 Pricing This scraper uses **Pay-per-result** pricing — you pay only for the records you actually receive. No subscriptions, no platform fees from us. The cheapest way to get fresh B2B agency leads at scale. The first **100 results per run are free** so you can validate the data before committing. *** ### 🚀 Related actors by the same team If you found this useful, you'll get more leverage from the rest of our B2B intelligence suite. Each one is built with the same data-quality discipline: - **DesignRush Scraper** — `haketa/designrush-scraper` 40,000+ B2B agencies with **minimum project budget**, hourly rate, employees, founded year, email, phone, full address, and inline verified reviews. Stronger on premium creative / branding agencies. - **ProductHunt Launches & Makers Scraper** — `haketa/producthunt-launches-scraper` Every Product Hunt product launch with founders, vote counts, comments, reviews, screenshots, and topic categorization. Perfect for VC sourcing and recruiter pipelines. - **Y Combinator Companies Scraper** — `haketa/ycombinator-companies-scraper` Every YC company across all batches with industry, location, team, and tags. The canonical YC alumni database. - **BuiltIn Tech Companies + Tech Stack Scraper** — `haketa/builtin-tech-companies-scraper` Tech companies + their declared tech stack. Ideal for selling developer tools and finding companies hiring in your stack. - **H1B Visa Database Scraper** — `haketa/h1b-visa-database-scraper` Visa sponsor employers + salary records. Recruiter and immigration-research gold. - **NPPES NPI Registry Scraper** — `haketa/nppes-npi-registry-scraper` Every US healthcare provider with NPI, specialty, address, and phone. Healthcare GTM in a single dataset. - **Apartments / Rent / Realtor / Domain / Trade Me / Zameen / etc.** — country-by-country real-estate scrapers in the `haketa/` namespace. Use them for real-estate lead lists and market data. - **License-board scrapers** — `haketa/california-dca-license-scraper`, `tsbp-license-scraper`, `ohio-elicense-scraper`, `illinois-idfpr-license-scraper`, `arizona-roc-contractor-license-scraper`, `colorado-professional-license-scraper`, `nc-licensing-board-for-general-contractors-scraper`, `washington-li-contractor-license-scraper`, `virginia-dpor-license-scraper`, `minnesota-dli-license-scraper`, `sam-gov-federal-contractor-scraper`, `ttb-alcohol-permittee-scraper` — every US state's licensed professional database, ready for trades / contractor / regulated-industry outreach. - **Hiring-marketplace scrapers** — `kleinanzeigen-scraper`, `marktplaats-scraper`, `mourjan-scraper`, `chotot-scraper`, `offerup-scraper`, `kijiji-scraper`, `lelong-my-scraper`, `seek-scraper`, `trademe-scraper` — global classifieds. *** ### ✅ Status & support Maintained actively. Update notes shipped via the Apify build log. For feedback, edge cases, or category requests, leave a review on the actor page — every report shapes the next release. *** ### ⚖️ Disclaimer This scraper collects **publicly available information** from a public B2B directory. The user is responsible for compliance with the source website's terms of service and any applicable data-protection laws (GDPR, CCPA) when using the scraped data — especially for outbound email or call campaigns. Cold outreach laws vary by jurisdiction. *** Built for **sales teams, market researchers, SaaS founders, recruiters, agency owners, and growth marketers** who want B2B agency data faster, cheaper, and richer than ZoomInfo or Crunchbase can deliver — without writing a line of code. # Actor input Schema ## `mode` (type: `string`): What to scrape. 'companies' = B2B agencies (web dev, mobile, marketing…). 'software' = SaaS products (CRM, PM…). 'auto' = infer from each startUrl. ## `startUrls` (type: `array`): Any GoodFirms URL — service category (/companies/web-development-agency), company detail (/company/), software category (/crm-software), or software detail (/software/). Mixed types OK. ## `serviceCategorySlugs` (type: `array`): Discover companies from these service categories. Examples: web-development-agency, mobile-app-development, digital-marketing-agency, blockchain-development, ai-development, ecommerce-development, software-development. ## `countrySlugs` (type: `array`): Limit company discovery to these countries or states/cities. Examples: us, in, gb, ua, ca, california, denver, orange-county. If empty, scrapes the global category page. ## `softwareCategorySlugs` (type: `array`): Discover software from these category pages. Examples: crm-software, project-management-software, accounting-software, hr-software. ## `companySlugs` (type: `array`): Scrape these specific company profiles directly. Slug is the part after /company/ in the URL. ## `softwareSlugs` (type: `array`): Scrape these specific software product pages directly. Slug is the part after /software/. ## `employeesFilter` (type: `string`): Restrict company listings to this employee range. Maps to GoodFirms employees\[] filter. ## `hourlyRateFilter` (type: `string`): Restrict company listings to this hourly rate bracket. Maps to GoodFirms rate\[] filter. ## `minReviews` (type: `integer`): Filter listings to companies with at least N reviews (server-side via reviews=N). ## `scrapeReviews` (type: `boolean`): Fetch verified client reviews (rating, body, reviewer, project cost/duration). Costs more requests. ## `scrapeProfileDetails` (type: `boolean`): Visit each company/software detail page to extract richer data (services breakdown, portfolio counts, contact info). Disable for fast list-only scraping. ## `maxRecords` (type: `integer`): Maximum records to push. Set to 0 for unlimited. ## `maxPagesPerCategory` (type: `integer`): Pagination cap for each category/country page (≈10 listings per page). ## `requestDelay` (type: `integer`): Delay between requests. Higher = friendlier to GoodFirms / Cloudflare. ## `maxConcurrency` (type: `integer`): Parallel request count. Keep low (1–3) to avoid Cloudflare challenges. ## `proxyConfiguration` (type: `object`): Proxy configuration. Default: DATACENTER (cheap, fast — passes GoodFirms's Cloudflare basic check in most cases). If you see persistent 403s in the log, switch to RESIDENTIAL. ## Actor input object example ```json { "mode": "auto", "startUrls": [], "serviceCategorySlugs": [ "web-development-agency" ], "countrySlugs": [], "softwareCategorySlugs": [], "companySlugs": [], "softwareSlugs": [], "employeesFilter": "", "hourlyRateFilter": "", "minReviews": 0, "scrapeReviews": false, "scrapeProfileDetails": true, "maxRecords": 100, "maxPagesPerCategory": 5, "requestDelay": 1200, "maxConcurrency": 2, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] } } ``` # Actor output Schema ## `allRecords` (type: `string`): Full dataset — choose JSON / CSV / Excel in the export menu. ## `companiesView` (type: `string`): B2B agency lead-gen view. ## `softwareView` (type: `string`): SaaS product listings. ## `reviewsView` (type: `string`): One row per (record, review). Requires scrapeReviews=true. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "auto", "startUrls": [], "serviceCategorySlugs": [ "web-development-agency" ], "countrySlugs": [], "softwareCategorySlugs": [], "companySlugs": [], "softwareSlugs": [], "scrapeReviews": false, "scrapeProfileDetails": true, "maxRecords": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] } }; // Run the Actor and wait for it to finish const run = await client.actor("haketa/goodfirms-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "auto", "startUrls": [], "serviceCategorySlugs": ["web-development-agency"], "countrySlugs": [], "softwareCategorySlugs": [], "companySlugs": [], "softwareSlugs": [], "scrapeReviews": False, "scrapeProfileDetails": True, "maxRecords": 100, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": [], }, } # Run the Actor and wait for it to finish run = client.actor("haketa/goodfirms-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "auto", "startUrls": [], "serviceCategorySlugs": [ "web-development-agency" ], "countrySlugs": [], "softwareCategorySlugs": [], "companySlugs": [], "softwareSlugs": [], "scrapeReviews": false, "scrapeProfileDetails": true, "maxRecords": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] } }' | apify call haketa/goodfirms-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=haketa/goodfirms-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GoodFirms B2B Companies & Software Scraper", "description": "Extract B2B agency listings (web dev, mobile, marketing, design, blockchain, AI) and software products (CRM, PM, HR…) from GoodFirms.co — with hourly rates, employees, hourly rate, services, ratings, verified reviews, and contact data. No API key required.", "version": "0.0", "x-build-id": "EuqAzZp3hgXzAH5Zb" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/haketa~goodfirms-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-haketa-goodfirms-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/haketa~goodfirms-scraper/runs": { "post": { "operationId": "runs-sync-haketa-goodfirms-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/haketa~goodfirms-scraper/run-sync": { "post": { "operationId": "run-sync-haketa-goodfirms-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Scraping Mode", "enum": [ "auto", "companies", "software" ], "type": "string", "description": "What to scrape. 'companies' = B2B agencies (web dev, mobile, marketing…). 'software' = SaaS products (CRM, PM…). 'auto' = infer from each startUrl.", "default": "auto" }, "startUrls": { "title": "Start URLs", "type": "array", "description": "Any GoodFirms URL — service category (/companies/web-development-agency), company detail (/company/), software category (/crm-software), or software detail (/software/). Mixed types OK.", "default": [], "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "serviceCategorySlugs": { "title": "Service Category Slugs (companies mode)", "type": "array", "description": "Discover companies from these service categories. Examples: web-development-agency, mobile-app-development, digital-marketing-agency, blockchain-development, ai-development, ecommerce-development, software-development.", "default": [], "items": { "type": "string" } }, "countrySlugs": { "title": "Country / Region Slugs (companies mode)", "type": "array", "description": "Limit company discovery to these countries or states/cities. Examples: us, in, gb, ua, ca, california, denver, orange-county. If empty, scrapes the global category page.", "default": [], "items": { "type": "string" } }, "softwareCategorySlugs": { "title": "Software Category Slugs (software mode)", "type": "array", "description": "Discover software from these category pages. Examples: crm-software, project-management-software, accounting-software, hr-software.", "default": [], "items": { "type": "string" } }, "companySlugs": { "title": "Individual Company Slugs", "type": "array", "description": "Scrape these specific company profiles directly. Slug is the part after /company/ in the URL.", "default": [], "items": { "type": "string" } }, "softwareSlugs": { "title": "Individual Software Slugs", "type": "array", "description": "Scrape these specific software product pages directly. Slug is the part after /software/.", "default": [], "items": { "type": "string" } }, "employeesFilter": { "title": "Employees Range Filter", "enum": [ "", "2-9", "10-49", "50-249", "250-999", "1000-9999", "10000+" ], "type": "string", "description": "Restrict company listings to this employee range. Maps to GoodFirms employees[] filter.", "default": "" }, "hourlyRateFilter": { "title": "Hourly Rate Filter (USD)", "enum": [ "", "<25", "25-49", "50-99", "100-149", "150-199", "200+" ], "type": "string", "description": "Restrict company listings to this hourly rate bracket. Maps to GoodFirms rate[] filter.", "default": "" }, "minReviews": { "title": "Minimum Reviews", "minimum": 0, "type": "integer", "description": "Filter listings to companies with at least N reviews (server-side via reviews=N).", "default": 0 }, "scrapeReviews": { "title": "Scrape Reviews", "type": "boolean", "description": "Fetch verified client reviews (rating, body, reviewer, project cost/duration). Costs more requests.", "default": false }, "scrapeProfileDetails": { "title": "Scrape Full Profile Details", "type": "boolean", "description": "Visit each company/software detail page to extract richer data (services breakdown, portfolio counts, contact info). Disable for fast list-only scraping.", "default": true }, "maxRecords": { "title": "Max Records", "minimum": 0, "type": "integer", "description": "Maximum records to push. Set to 0 for unlimited.", "default": 100 }, "maxPagesPerCategory": { "title": "Max Pages Per Category", "minimum": 1, "type": "integer", "description": "Pagination cap for each category/country page (≈10 listings per page).", "default": 5 }, "requestDelay": { "title": "Request Delay (ms)", "minimum": 0, "type": "integer", "description": "Delay between requests. Higher = friendlier to GoodFirms / Cloudflare.", "default": 1200 }, "maxConcurrency": { "title": "Max Concurrency", "minimum": 1, "maximum": 10, "type": "integer", "description": "Parallel request count. Keep low (1–3) to avoid Cloudflare challenges.", "default": 2 }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Proxy configuration. Default: DATACENTER (cheap, fast — passes GoodFirms's Cloudflare basic check in most cases). If you see persistent 403s in the log, switch to RESIDENTIAL." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```