Snowflake Marketplace Scraper

This project is a sophisticated scraper for the Snowflake Marketplace, designed to extract and compile data product listings. It is built with TypeScript and leverages the Crawlee framework and Cheerio library to efficiently navigate and parse the marketplace's web pages.

Features

• Fetch listings of data products available on the Snowflake Marketplace.
• Parse detailed information about each product into a structured JSON format.
• Handle pagination to collect data from multiple pages.

Input

To use the Snowflake Marketplace Scraper, you need to specify the number of results you want to retrieve. If you want all results set the limit to 99999 (currently Snowflake Marketplace has ~2200 data products as of March 2024).

⬇️ Input example

maxResults=99999

Output

The Snowflake Marketplace Scraper extracts all the browse data products from Snowflake, providing a comprehensive dataset that includes details such as product ID, name, title, description, links to Snowflake and the product's website, profile images, and descriptions. It also gathers contact information, regions where the products are available, coverage types, pricing plans, free sample data availability, categories, business needs, usage examples, refresh rates, and publication dates. This data is crucial for analyzing the offerings on the Snowflake Marketplace and can be used for market research, competitive analysis, or data acquistion stratergies.

⬆️ Output example

1{
2    "id": "GZ1M6Z1CH1F",
3    "name": "Comprehensive Company Data",
4    "title": "Established Year | Address, Revenue, Employees, Online presence | CDL, BUS Classifications | Phone, Email",
5    "description": "Maintaining a clean data can be a real challenge – especially for global organizations with thousands of customers and vendors. But the business imperative is clear. Poor data quality is considered responsible for costing organizations an average of $15 million per year.\n\napexdata helps you minimize your risks by cleaning your data from top to bottom and keeping it in pristine condition. Our technology does the heavy lifting.\n\nsmartvm automatically compares each field in your vendor master database to more than 2 billion records from hundreds of public and private sources, including 90M+ records compiled by apexanalytix during direct interaction with suppliers for Global 2000 companies.\n\nTable(s) Included:\n- Company Comprehensive Data\n\nFields Included:\n- Company Name\n- Year Established\n- Annual Revenue\n- Employee Range (Lower & Higher)\n- CDL Score\n- Payment Method & Terms\n- BUS Classification\n- Address, Emails & Phone Numbers\n- Primary Secondary & Tertiary Contacts",
6    "snowflakeLink": "https://app.snowflake.com/marketplace/listing/GZ1M6Z1CH1F",
7    "siteLink": "https://www.apexanalytix.com/data",
8    "profileImage": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAACXBIWXMAAAsTAAALEwEAmpwYAAAIRklEQVR4nO1ZCUxUSRDtmQEUHUA5xBtEURd1jbrGaACPNVmMNyiibKIJ8YgalYjGKx6LUVlBQVldxQTPxAs13iCgUcCDlah4rGfUGK9433dtXjH9988w7AyuOiZ/Kqn8rq7q/t31q6qr+gshBGkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcBWkcRYVMnU5Her2en9Z4sl2RjJSz5Nnb5zAF6K1sCH3W5CqiDQaDTfnvBEWFX7ZWrVrUsmVL8vf3N5Px9PQkLy8vbru5ubFMzZo1y31J8Bo1akR169ZVxmKcr58fz6F+H/rxvipVqjhOATrTYtq2bUv79u0jCW/evKE5c+YoMocOHaJHjx7RlClT6M6dOyxz//596tGjhzLXkCGxdPPmTWWO3Nxc8vHx4bmfPXtGr169ogEDBrBsbOyvLHPmzBkzRX5zBRhMZjthwgReUE5ODq1cuZIVAOjduzfzT548yfS7d+9o9erVVFRUxPTdu3d54REREUxjo+PGjaPU1FSmCwoKePzChQuZvnDhAjVs2JCuXbvGdK9evSp0n2/mAsKEvr6+Snv9+vW8wAULFjB97NgxpqdOncp08+bN6cWLF9zXvXt32rx5M7eXLFlCHh4e1LhxY7p06RL3QRZjTp8+zfSVK1cUWQfFCWGV0adPH9q1axd/pQcPHvAiFy1aZKaAMWPGMA1/vnHjBveNHTuWjh49qliIJYSGhvEYWJOEJ0+eKHHGoQrQmfwuIyODF4ZNYdPSxJOTk80UMGLECKbh24gBgMGDB1N2dja358+fT/Xr16fWrVtTcHAwowxya9asYZn379/zE/HEAeZPwnLzDRo0UMwZpou+7JwcppOSkswUsGzZMl5wQkKCEixhDbNnz2YaigsICCCj0UgjR46kUaNG8fi+ffsy/9SpUzR+/Hhuv3z5khXkACsQZh2urq50sqSEF/X48WN6+PCh8nVXrFjBMsXFxYr5fvz4UWlPmzaN+YjkUklqQBxo0aIFB0dATEwMy+NUAZw4Uew4CxAqK8CXSE9Ppx07drCf16tXjxYvXkz9+vVjvvRxBK6UlBTauHEjDRw40GwOd3d3HosAumHDBj5ZqlatSuHh4ZSZmUnx8fGKLAIj5sd8TZo0MZvHVpb6xS1AZ+PFeoOBjh8/bhYDLMdW5gy3V1Yt94VzBGH1ZTBFiaDhGjLLw+nw6dMniouLUzI+a6arnsPFxYX78IQbBAUFmb0P/VLGHkV4e3tTs2bNvo4ChI0vhA3DnO3xV0urQH6BLPDIkSN2L7JatWqcJqv7pBt26dKlzDL1hi9vAcJGtaf2QXXb2njJhwKQH2AD1sbK8dIqQCOTRACVVoXnnj176O3bt9ShQ4eyOdCv039OjBD/KaAuTj7H9yyLGygAx2VhYaHdcxw+fJguX75cTvmyINNZUeD/sgA/Pz+aN28eax3Fzt69e5UzGryDBw/S6NGj+by/desWnT17loYOHaqMR/KTlpZGV69epdu3b9OWLVv4JAEPpgwLgAvgC2dlZdHOnTs5ZZbje/bsyTlCbGwsJ2WQx9GJ4xc0ZBITE6mwqIj8vWuQoUETavrnLqozajrpTXOEhYXR+fPnafjw4VatzKoCDCbzmjFjBvvXpk2baNWqVdxG7i43JwFnfWpqGj19+pTp9u3bswyOUACyvXXr1nH7wIFcRYHYEE4S0EimANisXIfMIUJCQrgYAyAfgSLnzp3LMijUACi3QQclZtBPpe/I2LE7ebjo6fbde/T8+XN+nw2rEOU6kcjUrl1boZcvX84vgxXUqFGDPnz4QCUlJQp/2LBhzEdWB7pOnTocpSV/9+7dzEe+jy+N9Beb5IUHBTFPugTuFgDwezkeG4FFqNcIywEEBAYy7Wr0pJZZf1HDzHzq1O3nsgDZtavZh7VLATqTpqCACfHxlJ7+B507d44nbNOmDae6gPz8fGVM//79uW/SpElKX2BgIE2ePJm/8PXr15mP0rd69eqsAKTJUhaJlEy9Z86cyW0clfJjIDWHm/FGXV35uX379jIFBAQo83i070xtip9S7ZHTqVVT+xIqYRlY8IR/I8VFMbR27VrWPgBFjVQAApMch4sNtQJmzZrFNAIXsj48AagzUBdYKiA8vDPz4S6IG3l5eQoPCoMCSktLmZYngzUF+EXFUauCexQUHUe/TZ9G1U1xxYYShJkQojRMXL1ApKgAmKdUAAKh5EdGRiqlMDYI2Lp1q8KXi0UgxIZwCljmAeraQV68ADEfCiUZg8q5gCkGuAcGU9uie+SdsJD6RfzCvIkTJ1ZeAUajkSP/69evuURFHi+rw3bt2vHRA0BElmMHDYpRLADzIEvE7RBcYNu2bcqNkqwMAdKnpX9GR0cr/Zb5xcWLF5kHpS1dupT7ZckdaLKAH9bk0485V8hQ04/cXV3ob9MYXMHhHZGRUeyCVhQiyrlAt27d+PoKV1UodlDAYCPwURQ5SIVR68txnTp14i8YFRWlxAQoCLc9uEvELRHGwLqQQcIikpJ+N1NAaGgoLxhXaLJfrgfvx+ZxOSMrTjwP5OWRj6eR3ELaUfCSLPIKjfh3TR078h5gBZgL13Tqy1mbQVB8JlZ2vJTfv38/xwZcrqj77ZrPyj+GSqxBlOtUp74yjbVVjVn+KFG3rY1Xp6owU0BycoqZVVS0Huvr0JkpohI/WgQ5GpEfIKeHezng/YI0joK+B3TgbzPh8M07LUA4XYA0HwOE1oOgcCpAOC1AOF1AOGOAcAZB4TwFhPMYFM48QDgTIeHMBIUzFRbOWkB8ZSX8A4hKBmQhWLkQAAAAAElFTkSuQmCC",
9    "profileName": "apexanalytix",
10    "profileDescription": "apexanalytix revolutionized recovery audit with advanced analytics and the introduction of firststrike overpayment and fraud prevention software. Today, apex also leads the world in supplier management innovation with apexportal and smartvm, now the most widely used supplier onboarding and compliant master data management solution in global procure to pay processes. With over 250 clients in the Fortune 500 and Global 2000, apex is dedicated to providing companies and their suppliers with the ultimate supplier management experience.",
11    "contactInfo": "cpardo@apexanalytix.com",
12    "regionsAvailable": [
13        "AWS_CA_CENTRAL_1",
14        "AWS_US_EAST_1",
15        "AWS_US_EAST_2",
16        "AWS_US_WEST_2",
17        "AZURE_CANADACENTRAL",
18        "AZURE_CENTRALUS",
19        "AZURE_EASTUS2",
20        "AZURE_SOUTHCENTRALUS",
21        "AZURE_WESTUS2",
22        "GCP_US_CENTRAL1",
23        "GCP_US_EAST4"
24    ],
25    "coverageType": "GLOBAL",
26    "coverageBy": ["COUNTY"],
27    "isFree": false,
28    "pricingPlan": {
29        "type": "fixed",
30        "currency": "USD",
31        "priceAfterTrial": 999,
32        "daysTrial": 7
33    },
34    "freeSampleData": false,
35    "categories": ["FINANCIAL"],
36    "businessNeeds": [
37        {
38            "category": "360-Degree Customer View",
39            "description": "90M+ company records scrubbed, validated, enriched, classified and scored for accuracy\n\nPrepopulate data as new companies are brought on board to reduce the risk of keying errors\n\nValidate the accuracy of each record at the data-field level\n\nFlexible options for a one-time clean-up or continuous protection\n\n\nSupply Chain\nRemove duplicate records that can trigger overpayments\n\nMitigate noncompliance and fraud risks\n\nPrepopulate data as new suppliers are brought on board to reduce the risk of keying errors\n\nEnsure regulatory compliance\n\nImprove spend analytics for better-informed strategic sourcing decisions and contract negotiations"
40        }
41    ],
42    "isWithStandardTerms": true,
43    "usageExamples": [
44        {
45            "title": "Company Revenue and Employee Data",
46            "description": "",
47            "query": "\"Select \\\"Company Name\\\", \\\"Annual Revenue\\\", \\\"Annual Revenue Range\\\", \\\"Employee Lower Range\\\", \\\"Employee Higher Range\\\" From  DATA_SHARE.ComprehensiveCompanyData;\""
48        },
49        {
50            "title": "Company Address and Classification",
51            "description": "",
52            "query": "\"Select \\\"Company Name\\\", \\\"Primary Address\\\", \\\"Primary City\\\", \\\"Primary State\\\", \\\"Primary Postal Code\\\", \\\"Primary ISO\\\", \\\"Secondary Address\\\", \\\"Secondary City\\\", \\\"Secondary State\\\", \\\"Secondary Postal Code\\\", \\\"Secondary ISO\\\",\\n    \\\"Tertiary Address\\\", \\\"Tertiary City\\\", \\\"Tertiary State\\\", \\\"Tertiary Postal Code\\\", \\\"Tertiary ISO\\\", \\n    \\\"Primary BUS Classification Type\\\", \\\"Primary BUS Classification Number\\\", \\\"Primary BUS Classification Desc\\\", \\\"Primary BUS Classification Stdz\\\",\\n    \\\"Secondary BUS Classification Type\\\", \\\"Secondary BUS Classification Number\\\", \\\"Secondary BUS Classification Desc\\\", \\\"Secondary BUS Classification Stdz\\\",\\n    \\\"Tertiary BUS Classification Type\\\", \\\"Tertiary BUS Classification Number\\\", \\\"Tertiary BUS Classification Desc\\\", \\\"Tertiary BUS Classification Stdz\\\", \\\"DUNS Number\\\"\\n    From DATA_SHARE.ComprehensiveCompanyData;\""
53        },
54        {
55            "title": "Company Phone and Email Information",
56            "description": "",
57            "query": "\"Select \\\"Company Name\\\", \\\"Primary Phone\\\", \\\"Primary Phone Type\\\", \\\"Secondary Phone\\\", \\\"Secondary Phone Type\\\", \\\"Tertiary Phone\\\", \\\"Tertiary Phone Type\\\", \\n    \\\"Primary Email\\\", \\\"Primary Email Type\\\", \\\"Secondary Email\\\", \\\"Secondary Email Type\\\", \\\"Tertiary Email\\\", \\\"Tertiary Email Type\\\", \\n    \\\"Primary Person Name\\\", \\\"Primary Person Title\\\", \\\"Primary Person Dept\\\", \\\"Primary Person LinkedIn URL\\\", \\n    \\\"Primary Person Email 1\\\", \\\"Primary Person Email 1 Type\\\", \\\"Primary Person Email 2\\\", \\\"Primary Person Email 2 Type\\\", \\\"Primary Person Email 3\\\", \\\"Primary Person Email 3 Type\\\", \\n    \\\"Primary Person Phone 1\\\", \\\"Primary Person Phone Type 1\\\", \\\"Primary Person Phone 2\\\", \\\"Primary Person Phone Type 2\\\", \\\"Primary Person Phone 3\\\", \\\"Primary Person Phone Type 3\\\",\\n    \\\"Secondary Person Name\\\", \\\"Secondary Person Title\\\", \\\"Secondary Person Dept\\\", \\\"Secondary Person LinkedIn URL\\\", \\n    \\\"Secondary Person Email 1\\\", \\\"Secondary Person Email 1 Type\\\", \\\"Secondary Person Email 2\\\", \\\"Secondary Person Email 2 Type\\\", \\\"Secondary Person Email 3\\\", \\\"Secondary Person Email 3 Type\\\", \\n    \\\"Secondary Person Phone 1\\\", \\\"Secondary Person Phone Type 1\\\", \\\"Secondary Person Phone 2\\\", \\\"Secondary Person Phone Type 2\\\", \\\"Secondary Person Phone 3\\\", \\\"Secondary Person Phone Type 3\\\",\\n    \\\"Tertiary Person Name\\\", \\\"Tertiary Person Title\\\", \\\"Tertiary Person Dept\\\", \\\"Tertiary Person LinkedIn URL\\\", \\n    \\\"Tertiary Person Email 1\\\", \\\"Tertiary Person Email 1 Type\\\", \\\"Tertiary Person Email 2\\\", \\\"Tertiary Person Email 2 Type\\\", \\\"Tertiary Person Email 3\\\", \\\"Tertiary Person Email 3 Type\\\", \\n    \\\"Tertiary Person Phone 1\\\", \\\"Tertiary Person Phone Type 1\\\", \\\"Tertiary Person Phone 2\\\", \\\"Tertiary Person Phone Type 2\\\", \\\"Tertiary Person Phone 3\\\", \\\"Tertiary Person Phone Type 3\\\"    \\n    From DATA_SHARE.ComprehensiveCompanyData;\""
58        },
59        {
60            "title": "Company Certification Details",
61            "description": "",
62            "query": "\"Select \\\"Company Name\\\", \\\"Primary Certification Type\\\", \\\"Primary Certification Ethnic Group\\\", \\\"Primary Certification Body\\\",\\n    \\\"Secondary Certification Type\\\", \\\"Secondary Certification Ethnic Group\\\", \\\"Secondary Certification Body\\\",\\n    \\\"Tertiary Certification Type\\\", \\\"Tertiary Certification Ethnic Group\\\", \\\"Tertiary Certification Body\\\"\\n    From DATA_SHARE.ComprehensiveCompanyData;\""
63        }
64    ],
65    "refreshRate": "continuously",
66    "lastPublishedOn": "2023-06-05T07:12:24.61-07:00"
67}

Ending Note

Check out our other crawlers on our profile, Crawlr, for more tools and utilities.

Enjoy our scraper.