# Costco Product Scraper (`cirkit/costco-product-scraper`) Actor Scrape Costco product data from search URLs, category pages, and product URLs. Extract titles, prices, ratings, reviews, images, features, availability, and product URLs from Costco's search API. - **URL**: https://apify.com/cirkit/costco-product-scraper.md - **Developed by:** [Crikit](https://apify.com/cirkit) (community) - **Categories:** E-commerce, Open source - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $0.90 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### Costco Product Scraper Costco Product Scraper extracts structured product data from Costco search URLs, category URLs, product URLs, and keywords. It is built for Apify and uses Costco's own search API path instead of a browser-rendered crawl, which keeps runs fast, stable, and inexpensive. The actor is useful for price monitoring, catalog research, product availability checks, ecommerce analytics, and building clean Costco data feeds for downstream tools. The scraper accepts the same practical inputs buyers expect from a Costco scraping actor: a Costco search URL such as `https://www.costco.com/s?keyword=tshirt`, a legacy `CatalogSearch` URL, a category URL such as `https://www.costco.com/mens-clothing.html`, a direct product URL, or a list of keywords. Each result is written as one dataset item. ### What Data You Get Costco Product Scraper returns product data with fields that match or exceed the main Costco actor benchmark used during development. Dataset items include product URL, product ID, item number, title, item name, description, image URLs, list price, sale price, min and max price, currency, rating, review count, availability, stock status, membership flags, quantity restrictions, marketing features, marketing keywords, category paths, and timestamps. The output is normalized with camelCase keys so it works well in JavaScript, Python, spreadsheets, databases, and BI tools. The API data can be exported from Apify as JSON, CSV, Excel, XML, or RSS. ### Input Use `startUrls` for Costco URLs, `searchUrls` for a simple list of search or category URLs, or `keywords` for plain search terms. Use `maxItems` to cap the total dataset size and `maxItemsPerStartUrl` to cap each input URL. Use `maxSearchPagesPerStartUrl` to control pagination depth. The default input is intentionally small and valid for Apify's automatic actor tests: ```json { "startUrls": [{ "url": "https://www.costco.com/CatalogSearch?dept=All&keyword=tshirt" }], "maxItems": 10, "maxItemsPerStartUrl": 10, "maxSearchPagesPerStartUrl": 1 } ```` ### Output Example ```json { "productUrl": "https://www.costco.com/kirkland-signature-mens-pima-tee.product.4000385619.html", "name": "Kirkland Signature Men's Pima Tee", "itemNumber": "1966168", "groupId": "4000385619", "listPrice": 13.99, "currencyCode": "USD", "rating": 4.396900177001953, "reviewsCount": 194, "isInStock": true, "image": "https://bfasset.costco-static.com/..." } ``` ### How It Works Costco Product Scraper resolves every input into one of three API request types: 1. Keyword or Costco search URL: calls the Costco search API with the search query. 2. Category URL: calls the Costco navigation API route with the category path. 3. Product URL: searches the Costco API by product group or item number and emits the matching product. The actor uses Apify Proxy by default, with the Residential group and US country routing. That keeps local and cloud behavior aligned with real Costco availability and avoids many direct data center blocks. ### Usage Tips Use direct product URLs when you need one exact product. Use category URLs when you want products from a specific Costco section. Use keywords or search URLs for broader catalog discovery, and set `maxItems` to control the maximum number of charged results. For best product relevance, use concrete search terms such as `kirkland coffee`, `mens pima tee`, `vitamins`, or `laptop` instead of very broad terms. ### Notes Costco Product Scraper is designed for public Costco catalog pages and does not log in, bypass paywalls, or collect personal account data. It is a scraping API tool for public ecommerce product data. For best results, keep the proxy configuration enabled and use concrete search terms such as `tshirt`, `coffee`, `vitamins`, `laptop`, or `kirkland`. # Actor input Schema ## `startUrls` (type: `array`): Costco search, category, or product URLs. Product URLs are resolved through Costco search, while search and category URLs are read through Costco's product API. ## `searchUrls` (type: `array`): Optional simple list of Costco search or category URLs. This is compatible with common Costco scraper inputs. ## `keywords` (type: `array`): Optional Costco search keywords. Each keyword becomes https://www.costco.com/s?keyword=. Use concrete product terms such as tshirt, laptop, coffee, or vitamins. ## `maxItems` (type: `integer`): Hard cap on product records written to the dataset. ## `maxItemsPerStartUrl` (type: `integer`): Hard cap on product records discovered from each search or category start URL. ## `maxSearchPagesPerStartUrl` (type: `integer`): Maximum number of paginated Costco search/category API pages to request for each start URL. ## `scrapeProductDetails` (type: `boolean`): Compatibility option. Costco API results already include the core product details, so this does not require browser navigation. ## `proxyConfiguration` (type: `object`): Costco blocks many data center IPs. The default uses Apify Residential routed through the US. ## `maxConcurrency` (type: `integer`): Compatibility option retained for common scraper inputs. The API scraper runs paginated requests sequentially. ## `debug` (type: `boolean`): Enable verbose logs and save blocked-page evidence to the default key-value store. ## Actor input object example ```json { "startUrls": [ { "url": "https://www.costco.com/CatalogSearch?dept=All&keyword=tshirt" } ], "searchUrls": [], "keywords": [], "maxItems": 25, "maxItemsPerStartUrl": 25, "maxSearchPagesPerStartUrl": 2, "scrapeProductDetails": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" }, "maxConcurrency": 2, "debug": false } ``` # Actor output Schema ## `dataset` (type: `string`): Dataset containing all scraped Costco product records. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://www.costco.com/CatalogSearch?dept=All&keyword=tshirt" } ], "maxItems": 25, "maxItemsPerStartUrl": 25, "maxSearchPagesPerStartUrl": 2, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }; // Run the Actor and wait for it to finish const run = await client.actor("cirkit/costco-product-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://www.costco.com/CatalogSearch?dept=All&keyword=tshirt" }], "maxItems": 25, "maxItemsPerStartUrl": 25, "maxSearchPagesPerStartUrl": 2, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "US", }, } # Run the Actor and wait for it to finish run = client.actor("cirkit/costco-product-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://www.costco.com/CatalogSearch?dept=All&keyword=tshirt" } ], "maxItems": 25, "maxItemsPerStartUrl": 25, "maxSearchPagesPerStartUrl": 2, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }' | apify call cirkit/costco-product-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=cirkit/costco-product-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Costco Product Scraper", "description": "Scrape Costco product data from search URLs, category pages, and product URLs. Extract titles, prices, ratings, reviews, images, features, availability, and product URLs from Costco's search API.", "version": "0.1", "x-build-id": "MWx2Eoe3kpeK1CaT9" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/cirkit~costco-product-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-cirkit-costco-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/cirkit~costco-product-scraper/runs": { "post": { "operationId": "runs-sync-cirkit-costco-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/cirkit~costco-product-scraper/run-sync": { "post": { "operationId": "run-sync-cirkit-costco-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Costco URLs", "type": "array", "description": "Costco search, category, or product URLs. Product URLs are resolved through Costco search, while search and category URLs are read through Costco's product API.", "default": [ { "url": "https://www.costco.com/CatalogSearch?dept=All&keyword=tshirt" } ], "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "searchUrls": { "title": "Search URLs", "type": "array", "description": "Optional simple list of Costco search or category URLs. This is compatible with common Costco scraper inputs.", "default": [], "items": { "type": "string" } }, "keywords": { "title": "Keywords", "type": "array", "description": "Optional Costco search keywords. Each keyword becomes https://www.costco.com/s?keyword=. Use concrete product terms such as tshirt, laptop, coffee, or vitamins.", "default": [], "items": { "type": "string" } }, "maxItems": { "title": "Max products", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Hard cap on product records written to the dataset.", "default": 50 }, "maxItemsPerStartUrl": { "title": "Max products per start URL", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Hard cap on product records discovered from each search or category start URL.", "default": 50 }, "maxSearchPagesPerStartUrl": { "title": "Max search pages per start URL", "minimum": 1, "maximum": 50, "type": "integer", "description": "Maximum number of paginated Costco search/category API pages to request for each start URL.", "default": 3 }, "scrapeProductDetails": { "title": "Scrape product detail pages", "type": "boolean", "description": "Compatibility option. Costco API results already include the core product details, so this does not require browser navigation.", "default": true }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Costco blocks many data center IPs. The default uses Apify Residential routed through the US.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }, "maxConcurrency": { "title": "Max concurrency", "minimum": 1, "maximum": 10, "type": "integer", "description": "Compatibility option retained for common scraper inputs. The API scraper runs paginated requests sequentially.", "default": 2 }, "debug": { "title": "Debug logging", "type": "boolean", "description": "Enable verbose logs and save blocked-page evidence to the default key-value store.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```