AWS Marketplace Scraper | Fast & Reliable

Slug: fatihtahta/aws-marketplace-scraper

Overview

AWS Marketplace Scraper collects structured AWS Marketplace listing data such as titles, descriptions, categories, vendors, pricing summaries, reviews, and key metadata in clean JSON records. It turns searches, category pages, and listing pages into consistent datasets you can use across analysis and reporting workflows. AWS Marketplace (https://aws.amazon.com/marketplace) is a central catalog for cloud software and services, making it a valuable source for product discovery, vendor benchmarking, and market sizing. Runs are automated and consistent to help you save time and maintain reliable coverage.

Why Use This Actor

• Market research & analytics: Build market maps, track category trends, and analyze pricing/positioning over time.
• Product & content teams: Identify gaps, compare positioning, and keep catalogs fresh with verified listing metadata.
• Developers & data engineering: Feed dashboards, data warehouses, and enrichment pipelines with structured listings.
• Lead gen & enrichment: Discover vendors and solutions aligned with target segments for outreach or profiling.
• Monitoring & competitive tracking: Watch new launches, updates, and changes in key categories.

Input Parameters

Provide any combination of URLs, queries, and filters. Leave optional fields empty to collect broader results.

Parameter	Type	Description	Default
startUrls	string[]	One or more AWS Marketplace URLs to scrape directly (search results, category pages, or listing pages).	–
queries	string[]	Keywords to search on AWS Marketplace (e.g., vendor names, product names, or solution categories).	–
limit	integer	Maximum listings to save per query. Minimum: 10.	50000
proxyConfiguration	object	Optional connection settings for higher stability on larger runs. Default uses Apify Proxy with RESIDENTIAL group.	Apify proxy with RESIDENTIAL group

Example Input

{
  "startUrls": [
    "https://aws.amazon.com/marketplace/search/results?searchTerms=data+warehouse"
  ],
  "queries": ["security analytics", "observability"],
  "limit": 2000,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Output

6.1 Output destination

The actor writes results to an Apify dataset as JSON records. And the dataset is designed for direct consumption by analytics tools, ETL pipelines, and downstream APIs without post-processing.

6.2 Record envelope (all items)

Every dataset item includes:

• type (string, required)
• id (number, required)
• url (string, required)

Recommended idempotency key: type + ":" + id. Use this for deduplication or upserts when the same listing appears across multiple inputs.

6.3 Examples

Example: listing (type = "listing")

{
  "type": "listing",
  "id": 100001,
  "url": "https://aws.amazon.com/marketplace/pp/prodview-eju4flv5eqmgq",
  "title": "Wazuh All-In-One Deployment",
  "source_url": "https://aws.amazon.com/marketplace/search/results?category=96c2cd16-fe69-4b18-99cc-e016c61e820c&searchTerms=wow",
  "seed_type": "url",
  "seed_value": "https://aws.amazon.com/marketplace/search/results?searchTerms=wow&category=96c2cd16-fe69-4b18-99cc-e016c61e820c",
  "short_description": "Wazuh All-In-One. Includes Wazuh server, Filebeat, Wazuh dashboard and Wazuh Indexer",
  "long_description": "Wazuh is a free, open source and enterprise-ready security monitoring solution for threat detection, integrity monitoring, incident response, and regulatory compliance. \n\nThe solution includes the Wazuh server, which is in charge of analyzing the data received from the agents, processing events through decoders and rules, and using threat intelligence to look for well-known IOCs (Indicators Of Compromise). A single Wazuh server can analyze data from hundreds or thousands of agents. Alerts generated by Wazuh are sent to Wazuh indexer, where they are indexed and stored. The unique integration between Wazuh and Wazuh dashboard provides a powerful user interface for data visualization and analysis. The server is also used to manage the agents, configuring and upgrading them remotely when necessary. Additionally, the server is capable of sending orders to the agents, for example, to trigger a response when a threat is detected.\n\nWazuh provides a security solution capable of monitoring your infrastructure, detecting threats, intrusion attempts, system anomalies, poorly configured applications, and unauthorized user actions. It also provides a framework for incident response and compliance, all in one platform.",
  "version": "v4.14.1-1",
  "categories": [
    {
      "name": "Security",
      "id": "0625e4fd-88dd-4dd9-9e57-4a0461f97fb4",
      "parent_id": "579164ab-a400-4924-9c9c-590e45580b7d"
    },
    {
      "name": "Infrastructure Software",
      "id": "579164ab-a400-4924-9c9c-590e45580b7d"
    },
    {
      "name": "Monitoring",
      "id": "45c68cc2-ccd6-426b-94bd-92a791004dc2",
      "parent_id": "2560df6f-b99a-4899-8421-07dbaa1aa9d2"
    },
    {
      "name": "DevOps",
      "id": "2560df6f-b99a-4899-8421-07dbaa1aa9d2"
    },
    {
      "name": "Device Security",
      "id": "1c8040d7-fd09-424b-8ee8-3a733c77db7f",
      "parent_id": "96c2cd16-fe69-4b18-99cc-e016c61e820c"
    },
    {
      "name": "IoT",
      "id": "96c2cd16-fe69-4b18-99cc-e016c61e820c"
    }
  ],
  "badges": [
    {
      "name": "Deployed on AWS",
      "id": "DEPLOYED_ON_AWS"
    },
    {
      "name": "Standard Contract",
      "id": "STANDARD_CONTRACT"
    }
  ],
  "fulfillment_options": [
    {
      "type_name": "Amazon Machine Image",
      "type_id": "AMAZON_MACHINE_IMAGE"
    }
  ],
  "offer_summary": {
    "pricing_models": [
      {
        "name": "Free",
        "id": "FREE"
      }
    ],
    "pricing_summary": "Starting from $0.00 to $0.00/hr for software + AWS usage fees",
    "vendors": [
      {
        "name": "Wazuh Inc.",
        "id": "4c3cda83-f4cf-4afd-9f48-18ffe4f4fe69"
      }
    ]
  },
  "reviews": [
    {
      "source": "AWS_MARKETPLACE",
      "average_rating": "3.3",
      "review_count": 5,
      "url": "https://aws.amazon.com/marketplace/reviews/reviews-list/B09J56274H"
    },
    {
      "source": "PEERSPOT",
      "average_rating": "3.8",
      "review_count": 27,
      "url": "https://www.peerspot.com/products/wazuh-reviews"
    }
  ],
  "canonical_reference": "c1090bbb-318e-49c3-be1c-feca5c1eee35",
  "base_product_id": "79ced6c9-1e2d-4f22-ada6-dc528473b3f8",
  "seller_name": "Wazuh Inc.",
  "seller_id": "4c3cda83-f4cf-4afd-9f48-18ffe4f4fe69",
  "logo_url": "https://d7umqicpi7263.cloudfront.net/img/product/420dde73-e98d-4b8b-8324-2b8cad9266ce.com/efc9eb506f1c3a7443869b8c52ef63c2",
  "listing_thumbnail_url": "https://d7umqicpi7263.cloudfront.net/img/product/420dde73-e98d-4b8b-8324-2b8cad9266ce.com/efc9eb506f1c3a7443869b8c52ef63c2",
  "pricing": {
    "offer_id": "77lvn2mqfc4s937alo40h12ig",
    "offer_name": "!undefined",
    "pricing_model": "FREE",
    "authored_date": "1970-01-01T00:00:00.000Z",
    "vendor_name": "Wazuh Inc.",
    "vendor_id": "4c3cda83-f4cf-4afd-9f48-18ffe4f4fe69",
    "terms": [
      {
        "type": "UsageBasedPricingTerm",
        "currency_code": "USD",
        "rate_cards": [
          {
            "dimension_key": "c5a.xlarge",
            "display_name": "c5a.xlarge",
            "description": "!undefined",
            "unit": "Hrs",
            "price": "0.00000000"
          }
        ]
      },
      {
        "type": "LegalTerm",
        "documents": [
          {
            "type": "StandardEula",
            "url": "https://d7umqicpi7263.cloudfront.net/eula/lJ5Dcm2p-cVioqTyAE2Wx6-MlqisHgCBt53CJuaOyJY"
          }
        ]
      },
      {
        "type": "SupportTerm",
        "refund_policy": "We do not currently support refunds."
      }
    ]
  },
  "reviews_summary": [
    {
      "source": "PEERSPOT",
      "average_rating": 3.8,
      "review_count": 27,
      "max_rating": 5,
      "url": "https://www.peerspot.com/products/wazuh-reviews"
    },
    {
      "source": "AWSMP",
      "average_rating": 3.3,
      "review_count": 5,
      "max_rating": 5,
      "url": "https://aws.amazon.com/marketplace/reviews/reviews-list/B09J56274H"
    }
  ],
  "support_resources": [
    {
      "label": "AWS_SUPPORT",
      "type": "LINK",
      "name": "AWS Infrastructure Support",
      "value": "https://aws.amazon.com/premiumsupport/"
    },
    {
      "label": "CREATOR_SUPPORT",
      "type": "TEXT",
      "value": "Wazuh has one of the largest open source security communities in the world. You can become part of it to learn from other users, participate in discussions, talk to our development team, and contribute to the project.\n\n- https://wazuh.com/community/join-us-on-slack/\n- https://groups.google.com/g/wazuh \n- https://twitter.com/wazuh \n- https://reddit.com/r/Wazuh/ \n- https://stackoverflow.com/questions/tagged/wazuh \n- https://github.com/wazuh"
    }
  ],
  "highlights": [
    "Open Source Security Platform",
    "Host Based Intrusion Detection Solution",
    "Endpoint Detection and Response"
  ]
}

Field reference

Listing fields (type = "listing")

• title (string, required): Listing title.
• source_url (string, optional): Page used as the discovery source.
• seed_type (string, optional): Seed type used for discovery (e.g., url, query).
• seed_value (string, optional): Seed value used for discovery.
• short_description (string, optional): Brief summary of the listing.
• long_description (string, optional): Full description when available.
• version (string, optional): Listed version.
• categories (array[object], optional): Category hierarchy.
  • categories.name (string, optional)
  • categories.id (string, optional)
  • categories.parent_id (string, optional)
• badges (array[object], optional): Listing badges.
  • badges.name (string, optional)
  • badges.id (string, optional)
• fulfillment_options (array[object], optional): Available fulfillment types.
  • fulfillment_options.type_name (string, optional)
  • fulfillment_options.type_id (string, optional)
• offer_summary (object, optional): High-level pricing and vendor summary.
  • offer_summary.pricing_models (array[object], optional)
  • offer_summary.pricing_models.name (string, optional)
  • offer_summary.pricing_models.id (string, optional)
  • offer_summary.pricing_summary (string, optional)
  • offer_summary.vendors (array[object], optional)
  • offer_summary.vendors.name (string, optional)
  • offer_summary.vendors.id (string, optional)
• reviews (array[object], optional): Review sources.
  • reviews.source (string, optional)
  • reviews.average_rating (string, optional)
  • reviews.review_count (number, optional)
  • reviews.url (string, optional)
• canonical_reference (string, optional): Canonical reference identifier.
• base_product_id (string, optional): Base product identifier.
• seller_name (string, optional): Seller display name.
• seller_id (string, optional): Seller identifier.
• logo_url (string, optional): Logo image URL.
• listing_thumbnail_url (string, optional): Thumbnail image URL.
• pricing (object, optional): Detailed pricing metadata.
  • pricing.offer_id (string, optional)
  • pricing.offer_name (string, optional)
  • pricing.pricing_model (string, optional)
  • pricing.authored_date (string, optional)
  • pricing.vendor_name (string, optional)
  • pricing.vendor_id (string, optional)
  • pricing.terms (array[object], optional)
  • pricing.terms.type (string, optional)
  • pricing.terms.currency_code (string, optional)
  • pricing.terms.rate_cards (array[object], optional)
  • pricing.terms.rate_cards.dimension_key (string, optional)
  • pricing.terms.rate_cards.display_name (string, optional)
  • pricing.terms.rate_cards.description (string, optional)
  • pricing.terms.rate_cards.unit (string, optional)
  • pricing.terms.rate_cards.price (string, optional)
  • pricing.terms.documents (array[object], optional)
  • pricing.terms.documents.type (string, optional)
  • pricing.terms.documents.url (string, optional)
  • pricing.terms.refund_policy (string, optional)
• reviews_summary (array[object], optional): Consolidated review stats.
  • reviews_summary.source (string, optional)
  • reviews_summary.average_rating (number, optional)
  • reviews_summary.review_count (number, optional)
  • reviews_summary.max_rating (number, optional)
  • reviews_summary.url (string, optional)
• support_resources (array[object], optional): Support links or text.
  • support_resources.label (string, optional)
  • support_resources.type (string, optional)
  • support_resources.name (string, optional)
  • support_resources.value (string, optional)
• highlights (array[string], optional): Listing highlights.

Data guarantees & handling

• Best-effort extraction: fields may vary by region/session/availability/UI experiments.
• Optional fields: null-check in downstream code.
• Deduplication: recommend type + ":" + id.

Notes & Limitations

• Respect site terms and applicable laws.
• Avoid excessive frequency and automated runs that could disrupt normal site use.
• Pricing and availability can vary by region, time, and session context.
• Validate collected data to meet your compliance requirements.

Support

For help, open an issue on the actor page in Apify Console. Include the input you used (redacted), the run ID, a clear expected vs. actual description, and a small output sample if possible.

Happy Scraping -Fatih