Coles Scraper

Scrapes products and prices from Coles Australia. You can search the same way as on the website, or download every product in a category. Prices and range come from the Coles store nearest to any suburb or postcode you choose.

Input

Set a search term or a category, not both: like on the website, Coles' search can't be limited to a category. Without a store, prices are the ones Coles shows a new visitor (delivery to Melbourne 3000).

Parameter	Options	Description
searchTerm	(optional) Text to search for	One search, like the search box on the website. Returns the same products, in the same order, as the website, sponsored ones included.
category	(optional) Category page URL, e.g. https://www.coles.com.au/browse/dairy-eggs-fridge/milk, or its path, e.g. dairy-eggs-fridge/milk	Every product in this category or subcategory, in the website's order.
store	(optional) Suburb or postcode, e.g. Cannington or 6107	Uses the prices and range of the nearest Coles store, as when you choose a Click & Collect store on the website. The run fails if Coles doesn't know the suburb or postcode.

Example using a search query:

{
  "searchTerm": "tim tam"
}

Example using a category, at a store in Perth (a category path works too):

{
  "category": "https://www.coles.com.au/browse/fruit-vegetables",
  "store": "Cannington"
}

Categories

The top-level categories and their paths. A subcategory's path is the part after /browse/ in its URL on the website, e.g. dairy-eggs-fridge/milk. Seasonal categories come and go.

Path	Category
meat-seafood	Meat & Seafood
fruit-vegetables	Fruit & Vegetables
dairy-eggs-fridge	Dairy, Eggs & Fridge
bakery	Bakery
deli	Deli
pantry	Pantry
chips-chocolates-snacks	Chips, Chocolates & Snacks
international-foods	International Foods
health-dietary	Health & Dietary
drinks	Drinks
liquorland	Liquorland
frozen	Frozen
cleaning-laundry	Cleaning & Laundry
health-beauty	Health & Beauty
baby	Baby
pet	Pet
home-garden	Home & Garden
tobacco	Tobacco
bonus-credit-products	Bonus Credit Products (seasonal)
back-to-school	Back to School (seasonal)
big-pack-value	Big Pack Value
deliver-more-range	Deliver More Range

Output

• Dataset: one item per product, as Coles returns it, plus the product page url and the main image. Search results also carry search_term. Products come in the order the website shows them, each once:
  • Sponsored products are kept where they appear, marked by adId. When a sponsored product is also in the listing further down, it is kept only at its first place.
  • Banners and "you might also like" products from other categories are left out.
  • Coles' default order sometimes reshuffles while a long listing is paged, which repeats some products and skips others. When that happens, the skipped products are collected from the listing sorted by price and added at the end.
• STORE (key-value store): what was requested and which store served the data, e.g. {"requested": "Cannington", "id": "322", "name": "Coles Carousel", "address": "Carousel S/C, 1382 Albany Hwy", "suburb": "Cannington", "postcode": "6107", "state": "WA"}.
• FAILED_REQUESTS (key-value store, only when some requests failed): the URL and error of each failed page. If any page failed, the run fails rather than returning an incomplete dataset as a success.

Product fields

Values come as Coles sends them. Prices are in AUD. Fields inside an object are written with a dot, e.g. pricing.now.

Field	Description
id	Product code, as at the end of the product page URL.
brand, name, size	Brand, name and pack size, e.g. Dairy Farmers, Protein Smoothie Chocolate, 400mL.
description	Full name in capitals.
url, image	Product page and main image.
availability	Available at the store. Unavailable products have no pricing.
pricing.now, pricing.was	Price now, and before a special (0 when there is none).
pricing.saveAmount, pricing.savePercent, pricing.saveStatement, pricing.priceDescription	The saving, e.g. save $1.25, 1/2 Price.
pricing.comparable	Unit price as shown, e.g. $8.25/ 1L. pricing.unit has it as numbers.
pricing.promotionType, pricing.specialType	Promotion, e.g. SPECIAL, EVERYDAY, NEW, and kind of special, e.g. MULTI_SAVE, PERCENT_OFF, REDUCED_TO_CLEAR.
pricing.offerDescription, pricing.multiBuyPromotion	Multi-buy offer, e.g. Pick any 2 for $8.
onlineHeirs	The product's places on the website: category, subcategory and aisle names and IDs.
restrictions	Purchase limits, liquor and tobacco age flags, delivery restrictions.
availableQuantity, locations	Units available and aisle information, when Coles has them.
imageUris, variations, minGuarantee	All images, number of other sizes or flavours, minimum shelf life.
adId, adSource	Set on a sponsored product: the ad ID and the ad network, e.g. citrus.
search_term	Search rows only: the search that returned the product.