Dollar General Product Store Scraper

This unofficial Actor returns Dollar General product listings, product details, filters, top-level categories, store details, and city store-directory results. It is not affiliated with or endorsed by Dollar General.

Quick start

{
  "operation": "search",
  "query": "milk",
  "store_id": 1,
  "page": 1,
  "limit": 5,
  "sort": "recommended"
}

Operations

Operation	Purpose	Key inputs
probe	One-item connectivity/product check	store_id
categories	Top-level /c/... links from the storefront	none
store	Exact public store-directory detail	store_id
stores	City directory or exact numeric lookup	query as City, ST or a store number; limit
search_filters	Search or category filters and public sort options	query, or a category identifier; store_id
search	Store-aware keyword product search	query, store_id, page, limit, sort
category	Store-aware category products	category_id, category_url, or category_slug; store_id, page, limit, sort
item	Store-aware product detail	upc or item_url; store_id

The defaults are operation=search, query=milk, store_id=1, page=1, limit=5, sort=recommended, and include_reviews=false. page is 1-based. limit accepts 1–100. Sort values are recommended, price_asc, price_desc, and highest_rated.

When a category URL or slug is supplied without an ID, the Actor opens that category page and uses the ID sent by the live frontend. It does not derive or guess an ID from the slug. Item review compatibility fields may be empty because reviews are not provided by the product-detail response.

Access and reliability

The current storefront requires a US Residential Chromium session. The Actor creates a fresh bounded proxy session for each bootstrap attempt and tries no more than five times because an individual Residential exit can still be blocked or fail to tunnel. Success on the first attempt is not guaranteed. Guest authorization is captured only from the live frontend, held in memory for the run, and never included in datasets, OUTPUT, or logs.

The runtime uses the official Python 3.11 Playwright Chrome Actor image; it does not download a browser at runtime. Product calls use the current dggo.dollargeneral.com provider endpoints. Store and category-directory operations parse public www.dollargeneral.com pages in the same browser session.

Exact store 1 is the known public Ada, Oklahoma directory page. Other numeric store IDs are resolved by a bounded scan of public directory links. If the exact location cannot be found within that scan, the Actor returns a fail-honest error rather than fabricated store data.

Output

Successful records are pushed individually to the default Dataset with their existing public field names and added operation and record_type context. A legitimate zero-match search leaves the Dataset empty.

Every run writes a JSON summary to the default key-value store under OUTPUT:

{
  "source": "dollargeneral_us",
  "operation": "search",
  "status": "success",
  "success": true,
  "total": 0,
  "errors": []
}

Expected failures are described only in OUTPUT; fake error records are never pushed to the Dataset.

Limitations

Prices, inventory, and fulfillment flags are store-aware and can change. Public page markup and frontend provider schemas can change. Proxy rotation reduces transient blocking but cannot eliminate it. Store-number discovery is deliberately bounded, so a valid store may fail honestly if its directory path is not encountered.