First public version. Collects product name, brand, price, currency, stock status and
seller from eight Gulf retailers across 36 storefronts, covering Saudi Arabia, the
UAE, Qatar, Kuwait, Oman and Bahrain.
- Country and language per store. The country sets the currency and the
catalogue, the language sets the product names, and both ride on every row, so two
rows for the same product in different markets stay distinguishable. Arabic
storefronts return Arabic titles rather than translations.
- Reads schema.org JSON-LD rather than CSS selectors, so a site redesign does not break
collection the way it breaks selector-based scrapers.
- Respects robots.txt on every request, including redirects, and fails closed when
robots.txt cannot be read.
- Drops rows missing a name or a price rather than shipping them half filled in.
- Reports blocked, delisted and unparseable pages separately in the run summary.
- Streams rows to the dataset as they are collected, so a run that stops early still
leaves everything it gathered.
- Optional proxy, under Advanced. Required for the storefronts marked "needs regional
proxy", which serve their own country only; the country is set from the storefront
you pick. Namshi, Extra and Nahdi need no proxy.
- Filter by keyword, matching the product URL.
- Centrepoint, Home Centre, Babyshop, Splash and Max Fashion are collected from
their category pages, which publish full product data. One request returns up to
48 priced products instead of one, so a run is roughly ten times faster.
