Clutch.co Scraper

Features

Clutch.co doesn't provide a very flexible or free API, but this scraper acts as an unofficial Clutch API to help you extract the data you need, when you need it, and at scale.

Clutch.co Scraper supports the following features:

• Search any keyword

• Scrape lists

• Scrape reviews

• Scrape company profile

Clutch provides a "platform of in-depth client reviews, data-driven content, and vetted market leaders". Scraping that content and extracting it in a structured format could give you invaluable business insights and an edge over the competition.

Tutorial

Check out this blog post on how to extract data from Clutch.co with unofficial Clutch API for more information on the scraper.

Bugs, fixes, updates, and changelog

This scraper is under active development. If you have any feature requests, you can create an issue from here.

Upcoming changes

• Retrieve resource details.
• Search any keyword or resources.
• Enrich the reviews part, integration of full reviews.

Input parameters

The input of this scraper should be JSON containing the list of pages on Clutch.co that should be visited. Possible fields are:

• search: (Optional) (String) Keyword that you want to search on Clutch.co. This is required when mode is defined.

• mode: (Optional) (String) Mode that you want to use the search keyword on. The values can only be: profiles and companies. This is required when search field is defined.

• startUrls: (Optional) (Array) List of Clutch.co URLs. You should only provide a list or detailed URLs.

• includeReviews: (Optional) (Boolean) Adding reviews into the profile objects is optional and by default, it is false. If you want to scrape the reviews of the companies, then you can set this option as true.

• excludePortfolio: (Optional) (Boolean) Profile objects include portfolio field by default. If you don't want to scrape the portfolios of the companies, then you can set this option as true. This can increase performance.

• maxReviewsPerCompany: (Optional) (Number) Maximum number of reviews that you want to retrieve per each company. It is only enabled when includeReviews option is enabled as well. By default, it is all. If you want to scrape the reviews of the companies but with a limited amount, then you can set this option.

• endPage: (Optional) (Number) Final number of page that you want to scrape. The default is Infinite. This applies to all search requests and startUrls individually.

• maxItems: (Optional) (Number) You can limit scraped items. This should be useful when you search through the big lists or search results.

• proxy: (Required) (Proxy Object) Proxy configuration.

• extendOutputFunction: (Optional) (String) Function that takes a JQuery handle ($) as an argument and returns an object with data.

• customMapFunction: (Optional) (String) Function that takes each object's handle as an argument and returns the object with executing the function.

This solution requires the use of proxy servers, either your own proxy servers or Apify Proxy.

Tips

When you want to scrape over a specific listing URL, just copy and paste the link as one of the startURL.

If you would like to scrape only the first page of a list, then add the link for the page and have the endPage as 1.

Please also keep in mind that the includeReviews parameter will add multiple requests per company. That's why the number of requests or CUs that are consumed might be higher if you set this option as true.

Compute unit consumption

Clutch.co Scraper is optimized to run extremely fast and scrape many listings as possible, so it forefronts all listing detail requests. If the actor doesn't get blocked very often, it will scrape 100 listings in 2 minutes and consume ~0.07-0.08 compute units.

Clutch.co Scraper input example

1{
2    "startUrls": [
3        "https://clutch.co/profile/smartsites",
4        "https://clutch.co/web-developers/freelance",
5        "https://clutch.co/profile/blue-collar-agency"
6    ],
7    "search": "api",
8    "mode": "companies",
9    "endPage": 1,
10    "maxItems": 50,
11    "includeReviews": false,
12    "customMapFunction": "(object) => { return object }"
13}

During the run

During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified.

When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page.

If you provide incorrect input to the actor, it will immediately stop with a failure state and output an explanation of what is wrong.

Clutch.co export

During the run, the actor stores results into a dataset. Each item is a separate item in the dataset.

You can manage the results in any language (Python, PHP, Node.js/NPM). See the FAQ or our API reference to learn more about getting results from this Clutch.co actor.

Scraped Clutch.co company profiles

The structure of each item in Clutch.co listings looks like this:

1{
2    "url": "https://clutch.co/profile/smartsites",
3    "summary": {
4        "name": "SmartSites",
5        "logo": "https://img.shgstatic.com/clutchco-static/image/scale/60x60/s3fs-public/logos/a33a9494d9c0f41b112e8a1b4354a3e9.png",
6        "title": "Think Web. Think Smart. 💡",
7        "rating": 5,
8        "noOfReviews": 56,
9        "description": "Outsmart the competition with best-in-class digital marketing services. With over 450 ⭐⭐⭐⭐⭐ reviews online, SmartSites is America's #1 rated digital marketing agency. Call 📞 (201) 870 6000 for a free consultation! Get more traffic. Acquire more customers. Sell more stuff. SmartSites works for businesses of all sizes. SmartSites is a Google Premier Partner and Facebook Marketing Partner. Winner of dozens website design awards and four-time Inc5000 (2017-2020) fastest growing company. Let us grow your company.Read more...",
10        "verificationStatus": "GOLD VERIFIED",
11        "minProjectSize": "$1,000+",
12        "averageHourlyRate": "$100 - $149 / hr",
13        "employees": "10 - 49",
14        "founded": "Founded 2011",
15        "addresses": [
16            {
17                "title": "headquarters",
18                "street": "45 Eisenhower Drive",
19                "locality": "Paramus",
20                "region": "NJ",
21                "postalCode": "07652",
22                "country": "United States",
23                "phone": "+1.201.870.6000"
24            }
25        ]
26    },
27    "focus": [
28        {
29            "title": "Client focus",
30            "values": [
31                {
32                    "name": "Small Business (<$10M)",
33                    "percentage": 80
34                },
35                {
36                    "name": "Midmarket ($10M - $1B)",
37                    "percentage": 20
38                }
39            ]
40        }
41    ],
42    "serviceProvided": [
43		{
44			"name": "Web Development",
45			"percent": 0.3
46		},
47		{
48			"name": "Custom Software Development",
49			"percent": 0.25
50		},
51		{
52			"name": "Mobile App Development",
53			"percent": 0.25
54		},
55		{
56			"name": "AI Development",
57			"percent": 0.2
58		}
59	],
60    "portfolio": [
61        {
62            "image": "https://static2.clutch.co/s3fs-public/portfolio/28eae3874f8afd9e23a14f332d86353d.jpeg?N66vx9xl9cXLR5H6.euqT_BkCwjjAYD8",
63            "description": "Web Design, SEO, PPC"
64        }
65    ],
66    "verification": {
67        "verificationStatus": "GOLD VERIFIED",
68        "businessEntity": {
69            "name": "Melen, LLC",
70            "status": "Active",
71            "jurisdictionOfFormation": "New Jersey",
72            "ID": "0600372607",
73            "source": "New Jersey Division of Revenue & Enterprise Services",
74            "lastUpdated": "November 1, 2020",
75            "dateOfFormation": "May 15, 2011"
76        },
77        "paymentLegalFilings": {
78            "bankruptcy": "No",
79            "taxLienFilings": "0",
80            "judgementFilings": "0",
81            "collectionsCount": "0",
82            "source": "New Jersey Division of Revenue & Enterprise Services",
83            "lastUpdated": "November 1, 2020",
84            "fullBusinessCreditReport": "https://www.smartbusinessreports.com/search2.aspx?link=1352&fn=951794204"
85        }
86    },
87    "reviews": [
88        {
89            "name": " SEO & PPC Services for Outdoor Refinishing Company ",
90            "datePublished": "May 25, 2021",
91            "project": {
92                "name": "SEO & PPC Services for Outdoor Refinishing Company",
93                "category": "SEO & PPC",
94                "size": "$10,000 to $49,999$10,000 to $49,999",
95                "length": "Sep. 2020 - Jun. 2021Sep. 2020 - Jun. 2021",
96                "description": "SmartSites provided SEO and PPC services for an outdoor refinishing company including adding backlinks and creating new content. They planned monthly activities to raise the traffic and conversions."
97            },
98            "review": {
99                "rating": 5,
100                "quality": 5,
101                "schedule": 5,
102                "cost": 5,
103                "willingToRefer": 5,
104                "comments": "The team's work resulted in increased traffic and conversions. Thanks to SmartSites' efforts, cost per conversion (CPC) went down by 25%. They provided consistent updates and showed detailed reports about the project's status. Their timeliness and remarkable work ensured the engagement's success."
105            },
106            "reviewer": {
107                "title": "Owner, General Manager, Teak & Deck",
108                "name": "Drew Isaacman",
109                "industry": "Construction",
110                "size": "1-10 Employees",
111                "location": "Carlsbad, California",
112                "reviewType": "Online Review",
113                "verified": "Verified"
114            }
115        }
116    ],
117    "websiteUrl": "https://www.smartsites.com/lp/digital-marketing-lp/?utm_source=clutch.co&utm_medium=referral"
118}

Contact

Please visit us through epctex.com to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in epctex.com. In need of support? devops@epctex.com is at your service.