Dollar General Price Monitor

Monitor product prices, find clearance items, and discover penny items (≤ $0.01) from Dollar General. Perfect for price tracking, discount hunting, and market research.

🎯 Features

• Product Monitoring: Scrape product data including price, title, category, brand, and stock status
• Penny Item Detection: Automatically identify penny items (price ≤ $0.01)
• Clearance Tracking: Detect clearance items with discount calculations
• Price Range Filtering: Filter products by minimum and maximum price
• Flexible URL Input: Crawl category pages, search results, or individual product pages
• Proxy Rotation: Built-in proxy support to avoid blocking

📊 Output Data

Each scraped product includes:

Field	Type	Description
url	string	Product page URL
productId	string	Unique product identifier
title	string	Product name
price	number	Current price in USD
originalPrice	number	Original price before discount
discount	number	Discount percentage (0-100)
isClearance	boolean	Whether the product is on clearance
isPennyItem	boolean	Whether price ≤ $0.01
category	string	Product category path
brand	string	Product brand name
inStock	boolean	Whether the product is in stock
imageUrl	string	Product image URL
scrapedAt	string	Timestamp (ISO 8601)

🚀 Quick Start

Input Parameters

Parameter	Type	Required	Default	Description
start_urls	array	✅	-	URLs to start crawling from
maxResults	number	❌	100	Maximum products to scrape (0 = unlimited)
includePennyItemsOnly	boolean	❌	false	Only scrape penny items
includeClearanceOnly	boolean	❌	false	Only scrape clearance items
minPrice	number	❌	0	Minimum price in USD
maxPrice	number	❌	-	Maximum price in USD (null = no limit)
proxyConfig	object	❌	-	Proxy configuration (recommended)
requestDelay	number	❌	1.0	Delay between requests (seconds)

Example Input

{
  "start_urls": [
    {"url": "https://www.dollargeneral.com/clearance.html"}
  ],
  "maxResults": 100,
  "includePennyItemsOnly": false,
  "includeClearanceOnly": true,
  "minPrice": 0,
  "maxPrice": 50
}

Example Output

{
  "url": "https://www.dollargeneral.com/p/example-product/123456.html",
  "productId": "123456",
  "title": "Example Product Name",
  "price": 0.01,
  "originalPrice": 5.99,
  "discount": 99.83,
  "isClearance": true,
  "isPennyItem": true,
  "category": "Home > Kitchen",
  "brand": "BrandName",
  "inStock": true,
  "imageUrl": "https://www.dollargeneral.com/product-image.jpg",
  "scrapedAt": "2026-03-09T12:00:00.000Z"
}

💰 Pricing

This Actor uses Pay-Per-Event (PPE) pricing:

Event	Price	Description
actor-start	$0.01	Actor initialization
page-scraped	$0.001	Each page crawled
result-generated	$0.01	Each product extracted

Estimated cost: $1-5 per 100 products (varies by configuration)

🔧 Use Cases

1. Price Tracking: Monitor price changes over time
2. Penny Item Hunting: Find extreme discounts (≤ $0.01)
3. Clearance Monitoring: Track clearance sales
4. Market Research: Analyze product pricing and availability
5. Inventory Tracking: Monitor stock status

📝 Tips

• Use Proxies: Dollar General may block requests without proxies
• Start Small: Test with maxResults: 10 first
• Filter Results: Use includePennyItemsOnly or includeClearanceOnly to find deals
• Category Pages: Use category URLs to focus on specific product types

⚠️ Limitations

• Dollar General may change their website structure
• Some products may require JavaScript rendering (not supported)
• Rate limiting may occur without proxies
• Product availability may vary by location

🛠️ Technical Details

• Runtime: Python 3.13
• Crawler: BeautifulSoupCrawler (Crawlee)
• Parser: lxml
• Proxy Support: Yes (Apify Proxy recommended)

📚 Resources

• Apify Documentation
• Crawlee for Python
• Actor Source Code

🐛 Issues & Support

Found a bug or need help? Please create an issue or contact the developer.

📄 License

This Actor is provided for educational and research purposes. Please respect Dollar General's Terms of Service and robots.txt when using this Actor.

---

Developer: janseling
Version: 0.0.4
Last Updated: 2026-03-09