# Walgreens Product Scraper (`shahidirfan/walgreens-product-scraper`) Actor Extract product listings, prices, discounts, availability, ratings, reviews, images, brands, categories, UPCs, and detailed product information from Walgreens. Ideal for price monitoring, inventory tracking, competitor analysis, market research, and e-commerce automation. - **URL**: https://apify.com/shahidirfan/walgreens-product-scraper.md - **Developed by:** [Shahid Irfan](https://apify.com/shahidirfan) (community) - **Categories:** E-commerce, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 83.3% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Walgreens Product Scraper Extract comprehensive product data from Walgreens.com at scale. Collect product names, prices, ratings, inventory status, and more from search results and category pages. Perfect for retail analytics, price monitoring, and market research. ### Features - **Product Search** - Search by keyword to find relevant products across all categories - **Price Extraction** - Collect regular prices, sale prices, and unit pricing - **Ratings & Reviews** - Extract average ratings and review counts for each product - **Inventory Data** - Get real-time stock status and availability information - **Product Details** - Access product IDs, SKUs, UPCs, and brand information - **Offer Details** - Capture deals, rebate messages, and product-level savings text - **Fulfillment Signals** - Collect shipping, pickup, same-day, add-to-cart, and channel availability fields - **Pagination Support** - Automatically navigate through multiple result pages - **Flexible Input** - Use keywords or direct Walgreens URLs for targeted scraping ### Use Cases #### Competitive Price Monitoring Track price changes on Walgreens for specific product categories. Compare pricing strategies, identify sale patterns, and optimize your own pricing decisions. #### Product Research & Analysis Build comprehensive product catalogs with detailed attributes. Analyze product assortments, brand distribution, and category trends across Walgreens marketplace. #### Inventory & Availability Tracking Monitor stock levels for products across different fulfillment methods. Identify trends in product availability and plan purchasing strategies accordingly. --- ### Input Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `keyword` | String | No | `"vitamin"` | Product keyword to search for on Walgreens | | `url` | String | No | -- | Direct Walgreens URL (search results or category page) | | `results_wanted` | Integer | No | `20` | Maximum number of products to collect | | `max_pages` | Integer | No | `10` | Safety cap on number of result pages to scrape | | `sort` | Select | No | `"relevance"` | Sort order for search results | | `proxyConfiguration` | Object | No | Apify Proxy | Proxy settings for reliable data collection | --- ### Output Data Each item in the dataset contains: | Field | Type | Description | |-------|------|-------------| | `productId` | String | Unique product identifier | | `title` | String | Product display name | | `brand` | String | Brand name | | `price` | String | Current price | | `salePrice` | String | Sale or discounted price | | `regularPrice` | String | Regular price | | `onSale` | Boolean | Whether the product is on sale | | `imageUrl` | String | Product image URL | | `productUrl` | String | Direct link to product page | | `category` | String | Product category | | `size` | String | Product size or volume | | `retailUnitQty` | String | Retail unit quantity | | `reviewCount` | Number | Number of customer reviews | | `averageRating` | Number | Average customer rating | | `upc` | String | Universal product code | | `gtin` | String | Global trade item number | | `storeInventoryStatus` | String | Store inventory availability | | `shippingEligible` | Boolean | Whether the product can ship | | `pickupEligible` | Boolean | Whether the product supports pickup | | `sameDayEligible` | Boolean | Whether the product supports same-day fulfillment | | `addToCart` | Boolean | Whether the product can be added to cart | | `sku` | String | Stock keeping unit | | `description` | String | Product description | | `ingredients` | Array | Ingredient groups when available | | `warnings` | String | Product warning text when available | | `offers` | Array | Product offer and promotion details | | `deal` | String | Primary visible deal or savings message | | `shippingWeight` | Number | Shipping weight in pounds | | `ageRestricted` | Boolean | Whether the item has an age restriction | | `onlineRedemption` | Object | Online coupon redemption details when available | --- ### Usage Examples #### Basic Keyword Search Search for products by keyword: ```json { "keyword": "vitamin d", "results_wanted": 50 } ```` #### Category Page Scraping Use a direct category URL for targeted scraping: ```json { "url": "https://www.walgreens.com/store/c/productlist/N=20000944", "results_wanted": 100, "max_pages": 5 } ``` #### Full Configuration Maximum control with all options: ```json { "keyword": "ibuprofen", "results_wanted": 200, "max_pages": 20, "sort": "top_sellers", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } } ``` *** ### Sample Output ```json { "productId": "prod6089370", "title": "Vitamin D3 2000 IU Softgels", "brand": "Walgreens", "price": "$8.99", "salePrice": "$6.99", "regularPrice": "$8.99", "onSale": true, "imageUrl": "https://pics.walgreens.com/prodimg/508499/155.jpg", "productUrl": "https://www.walgreens.com/store/c/vitamin-d3-2000-iu-softgels/ID=prod6089370-product", "category": "Vitamins & Supplements", "size": "200 count", "reviewCount": 1247, "averageRating": 4.5, "upc": "311917150345", "storeInventoryStatus": "In Stock", "sku": "sku6080848", "shippingEligible": true, "pickupEligible": false, "sameDayEligible": true, "deal": "Buy 1, Get 1 FREE", "addToCart": true } ``` *** ### Tips for Best Results #### Choose Effective Keywords - Start with broad search terms to discover available products - Use specific product names for targeted extraction - Try different keyword variations to cover product categories #### Optimize Collection Size - Start with `results_wanted: 20` for quick testing - Increase to 100-500 for production runs based on your needs - Adjust `max_pages` to control scraping depth #### Proxy Configuration - Use residential proxies for reliable access to Walgreens - Enable Apify Proxy in regions that match your target market - Start with default proxy settings for most use cases *** ### Integrations Connect your data with: - **Google Sheets** - Export product data for collaborative analysis - **Airtable** - Build searchable product databases - **Make** - Create automated workflows and data pipelines - **Zapier** - Trigger actions based on scraping results - **Webhooks** - Send data to custom endpoints in real time #### Export Formats - **JSON** - For developers and API integrations - **CSV** - For spreadsheet analysis and reporting - **Excel** - For business reporting and presentations - **XML** - For system integrations and data exchange *** ### Frequently Asked Questions #### How many products can I collect? You can collect thousands of products in a single run. The practical limit depends on the available results for your keyword or category URL. #### Can I scrape specific product categories? Yes. Use a category URL as the `url` parameter to scrape specific departments or product categories on Walgreens. #### What if some fields are missing? Not all products have every attribute. Fields like `upc`, `salePrice`, or `description` may be empty if the source page does not provide that information. The scraper only includes fields with available data. #### Does this work with Walgreens pharmacy products? The scraper focuses on retail products available through Walgreens.com search. Some pharmacy- restricted items may not be accessible. #### How fast is the scraping? Performance depends on your proxy quality, network conditions, and the number of results requested. Typical runs collect 20-50 products per minute with residential proxies. #### Can I schedule regular runs? Yes. Apify supports scheduling for recurring data collection. Set up daily or weekly runs to track price changes and inventory updates. *** ### Support For issues or feature requests, contact support through the Apify Console. #### Resources - [Apify Documentation](https://docs.apify.com/) - [API Reference](https://docs.apify.com/api/v2) - [Scheduling Runs](https://docs.apify.com/schedules) *** ### Legal Notice This actor is designed for legitimate data collection purposes. Users are responsible for ensuring compliance with website terms of service and applicable laws. Use data responsibly and respect rate limits. # Actor input Schema ## `url` (type: `string`): Direct Walgreens URL to scrape. Supports search results (/search/results.jsp?Ntt=...) or category pages (/store/c/productlist/N=...). If provided, keyword is ignored. ## `keyword` (type: `string`): Product keyword to search for on Walgreens (e.g., 'vitamin', 'ibuprofen', 'shampoo'). ## `results_wanted` (type: `integer`): Maximum number of products to collect. ## `max_pages` (type: `integer`): Safety cap on number of result pages to scrape. ## `sort` (type: `string`): Sort order for search results. ## `proxyConfiguration` (type: `object`): Use Apify Proxy for reliable scraping. Residential proxies recommended for Walgreens. ## Actor input object example ```json { "keyword": "vitamin", "results_wanted": 20, "max_pages": 10, "sort": "relevance", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "keyword": "vitamin" }; // Run the Actor and wait for it to finish const run = await client.actor("shahidirfan/walgreens-product-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "keyword": "vitamin" } # Run the Actor and wait for it to finish run = client.actor("shahidirfan/walgreens-product-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "keyword": "vitamin" }' | apify call shahidirfan/walgreens-product-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=shahidirfan/walgreens-product-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Walgreens Product Scraper", "description": "Extract product listings, prices, discounts, availability, ratings, reviews, images, brands, categories, UPCs, and detailed product information from Walgreens. Ideal for price monitoring, inventory tracking, competitor analysis, market research, and e-commerce automation.", "version": "0.0", "x-build-id": "ZEGmXrEVFR1dZVP7I" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/shahidirfan~walgreens-product-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-shahidirfan-walgreens-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/shahidirfan~walgreens-product-scraper/runs": { "post": { "operationId": "runs-sync-shahidirfan-walgreens-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/shahidirfan~walgreens-product-scraper/run-sync": { "post": { "operationId": "run-sync-shahidirfan-walgreens-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "url": { "title": "Start URL (optional)", "type": "string", "description": "Direct Walgreens URL to scrape. Supports search results (/search/results.jsp?Ntt=...) or category pages (/store/c/productlist/N=...). If provided, keyword is ignored." }, "keyword": { "title": "Search Keyword", "type": "string", "description": "Product keyword to search for on Walgreens (e.g., 'vitamin', 'ibuprofen', 'shampoo')." }, "results_wanted": { "title": "Maximum results", "minimum": 1, "type": "integer", "description": "Maximum number of products to collect.", "default": 20 }, "max_pages": { "title": "Maximum pages", "minimum": 1, "type": "integer", "description": "Safety cap on number of result pages to scrape.", "default": 10 }, "sort": { "title": "Sort order", "enum": [ "relevance", "price_asc", "price_desc", "top_sellers", "rating", "newest", "name_asc", "name_desc" ], "type": "string", "description": "Sort order for search results.", "default": "relevance" }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Use Apify Proxy for reliable scraping. Residential proxies recommended for Walgreens.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```