Clutch Scraper — Agency Rates, Size & Services

Shortlisting a B2B agency on Clutch means opening a directory, then opening a tab per agency to find out what it charges, how small a project it will take, how many people it has, and what it actually spends its time doing. This Actor returns that as a spreadsheet. One row per agency, in Clutch's own ranking, with the rating, the review count, the minimum project size, the hourly-rate band, the headcount band, the location, the phone number and the agency's percentage service mix.

The service mix is the column worth the run. Firmographics are available in a dozen places;

45% Web Development, 45% Web Design, 10% Branding

Accepted input

directory is required and takes Clutch directory paths, one per line. A path is what follows clutch.co/ in the address bar: web-developers, it-services, agencies/digital-marketing. A pasted directory URL is accepted and reduced to its path, and a query string on it is discarded. One or two segments is the accepted shape.

maxItems caps how many agencies the run saves across every directory it was given, and defaults to 100; 0 walks each directory to its end. startPage defaults to 1 and is a resume point rather than a filter — a run continues into later pages by itself, and the log prints the page to continue from when it stops.

{
  "directory": ["web-developers", "agencies/digital-marketing"],
  "maxItems": 100,
  "startPage": 1
}

Where the path is unknown, Clutch Agency List run in directory mode returns every ranked directory path Clutch publishes, and every row of it goes straight into directory here.

Response fields

A real row, from web-developers:

{
  "position": 1,
  "name": "Lounge Lizard",
  "slug": "lounge-lizard",
  "agency_id": "23730",
  "clutch_url": "https://clutch.co/profile/lounge-lizard",
  "directory": "web-developers",
  "rating": 4.8,
  "review_count": 43,
  "rating_best": 5,
  "rating_worst": 0.5,
  "min_project_size": "$25,000+",
  "hourly_rate": "Undisclosed",
  "employees": "50 - 249",
  "location": "New York, NY",
  "service_mix": "45% Web Development, 45% Web Design, 10% Branding",
  "primary_service": "Web Development",
  "services": [
    { "share_pct": 45, "service": "Web Development" },
    { "share_pct": 45, "service": "Web Design" },
    { "share_pct": 10, "service": "Branding" }
  ],
  "telephone": "1-888-444-0110",
  "street": "112 West 34th Street 18th Floor",
  "locality": "New York",
  "region": "NY",
  "postal_code": "10120",
  "country": "US"
}

min_project_size, hourly_rate and employees are kept as the strings Clutch renders rather than parsed into numbers. The + in $25,000+ and the boundaries of $100 - $149 / hr are the information, and a numeric reading discards both.

Questions

Why did a run return fewer agencies than maxItems? Because the directory ran out. A directory is walked until it produces no agency the run has not already saved, and that point arrives sooner than the row count on the page suggests: a Clutch directory page renders 80 cards but holds far fewer distinct agencies, because promoted placements repeat inside a page and roughly thirty of them ride along on every adjacent pair of pages. Those repeats are dropped, so a run asked for 500 agencies gets 500 distinct ones or the whole directory, whichever is smaller.

Why is hourly_rate the word Undisclosed on some rows and empty on others? They mean different things and are deliberately not merged. Undisclosed is what Clutch renders when a firm declines to publish a rate, which is an answer in its own right and is worth filtering on. An empty value means no rate was read from that listing at all. Collapsing the first into the second would make a firm that withholds its rate indistinguishable from a row that came back short.

A path from the Agency List Actor was rejected. Why? Clutch publishes deeper, country- and city-scoped variants of its directories — de/it-services/msp/berlin is a real page, and most of the paths in its index are that shape. This Actor walks one- and two-segment paths. Deeper paths in the input list are skipped with a note in the log naming each one, and the rest of the list still runs; a run fails only when every path it was given is that shape.

Can more than one directory be scraped in a single run? Yes — directory takes a list, and the directories are walked in the order given. They share one maxItems budget rather than getting one each, so a run over three directories with maxItems: 100 saves 100 rows in total, filling from the first directory before it starts the second. Set maxItems to 0 when each directory should be walked to its end.

Why does a large run take minutes? Requests to Clutch are paced, and that pacing is what keeps a long walk working rather than being cut off partway. Agencies arrive roughly fifty per request, so the wall-clock time of a run scales with rows rather than with how much data each row carries. A run of a few hundred agencies is a matter of minutes; the log prints progress per directory as it goes.

What is not in a row? The agency's own website, its founding year, its long description and its review text are on the agency's profile page, not on the directory listing, and none of them are returned here. Everything above comes off the listing itself.

Related

• Clutch Agency List — Every Agency & Directory URL — every directory path Clutch publishes, which is where the directory input for this Actor comes from, and every agency profile URL with its key.