Morrisons Grocery Scraper

Scrape Morrisons — one of the UK's largest supermarket chains — for products, prices and customer reviews. Pure HTTP: no browser, no login, no proxy.

What you get

One record per product in the run's default dataset:

Field	Example
retailerProductId	108444711
productId	UUID used by the review API
name, brand, packSizeDescription	Morrisons British Chicken Breast Fillets 1kg
price	{"amount": "6.99", "currency": "GBP"}
unitPrice	{"price": {"amount": "6.99", "currency": "GBP"}, "unit": "fop.price.per.kg", "unitName": "PER_1KG"}
promotions	active offers, e.g. Easy Chicken Dinner Bundle for £8
rating	{"overallRating": "2.2", "count": 101}
categoryPath, available, isNew, imageUrl, url	—
reviews, reviewsMeta	when reviews are enabled (see below)

With Include customer reviews on, every product also carries reviews[] — headline, star rating, nickname, date, full comment text, verified-buyer flag and helpful votes — plus reviewsMeta with the product's total review count and 5-star histogram.

How to use it

Fill any combination of the inputs; they run in this order:

• Search terms — keyword search, e.g. milk, chicken breast, gluten free pasta.
• Category IDs — department/category listings. Top-level IDs: 192077 New, 176738 Fruit & Veg, 179549 Meat & Fish, 176739 Fresh Foods, 102210 Bakery & Cakes, 180331 Frozen Food, 102705 Food Cupboard, 193648 Treats & Snacks, 192319 Dietary & Lifestyle Foods, 182137 World Foods, 103644 Drinks.
• Product URLs or IDs — full detail for specific products, e.g. https://groceries.morrisons.com/products/morrisons-british-chicken-breast-fillets-1kg/108444711 or just 108444711.
• Crawl the whole catalogue — walks all 22 top-level departments until Max products is reached (the catalogue holds ~30,000 products).

Max products caps the whole run. Leave every input empty to get a small default sample.

Review options

Reviews are fetched per product from Morrisons' own review API, up to Max reviews per product (100 per request under the hood), sorted by your chosen order: most recent, oldest, highest/lowest rated or most helpful.

Typical runs

• Price monitoring for a shopping list — a handful of search terms, Max products 100–500.
• Category research — one or two category IDs, reviews off.
• Full catalogue snapshot — Crawl the whole catalogue, Max products 30000+, reviews off.
• Review mining — search terms plus reviews on, filtered later on the reviews array.

Notes

• Prices are GBP as displayed on the site, including current promotions. Unit prices sometimes come back in pence (GBX) — that is how the site reports them.
• Runs are sequential with a small delay between requests; no proxy or browser is required, which keeps runs fast and cheap.
• Data reflects the Morrisons online catalogue at run time; delivery availability can vary by postcode but catalogue prices are national.

Local development

pip install -r requirements.txt
python3 -m src --search milk --max 20
python3 -m src --product 108444711 --reviews --max-reviews 5
python3 -m src --crawl --max 60