2.0 adds category scraping and reads everything from Farfetch's catalog service
rather than its website. Product records keep the 1.x fields and value types.
The one change to how an existing field is laid out is the Colour option, which
moves the size id from the first entry of each variant's options; the other
changes, listed below, are to what the fields contain.
- added category scraping: a storefront (
https://www.farfetch.com/,
https://www.farfetch.com/jp/) or department (/shopping/women/,
/uk/shopping/men/) start URL returns one category record per category —
1,216 in a whole storefront, titled in the language of the storefront's
product records. Each record has the category's path, a handle that is the
same in every storefront, its product count (sale included, where the
website's category page shows the full-price products) and its listing
filters: categories, designers and colours, each value with its product
count, plus price and discount ranges, full price or sale, and attributes
such as heel height. Size filters are not included. Category records share
the default dataset with products and have their own "Categories" view.
- added the
withCategoryProducts input: also scrape every product of those
categories, each product once however many categories list it. Off by
default.
- added set pages (
/sets/….aspx) as start URLs.
- added
source.language to product records, and each variant's sku: the
merchant's own stock keeping unit for the offer the variant is priced from,
whose format varies by merchant.
- changed: pay-per-event billing has two events,
product (per product saved,
as before) and category (per category record).
- changed: a product found through a listing gets a public page URL built from
its designer and name, such as
https://www.farfetch.com/shopping/women/jacquemus-les-doubles-sandals-item-28543291.aspx.
This is usually the website's own URL, but can differ for names with
punctuation (…-california-2-0-… for the website's …-california-20-…).
A product start URL keeps its own URL, without the query string, as
canonicalUrl; a short product link (/item-<id>.aspx) gets the product's
full page URL instead of the short link.
- changed: image URLs no longer end in
?ov=true. 1.x added it to the images
of some records and not others, the same image included.
- changed: each size is priced at the offer Farfetch shows for it (the
preferred merchant's), and its
stockCount adds up the stock of every
merchant selling that size. The product's price is its cheapest size in
stock. When several sizes share that price, 1.x took the one the website
listed first and 2.0 can take another, so price.stockCount can differ.
- changed:
variants list the sizes in stock first, as before, but in size
order, where 1.x followed the website's order. The other sizes follow, out
of stock at the product's price, as before.
- fixed: a product no merchant has on offer gets its full record (title,
details, images and currency), out of stock at 0; 1.x often returned an
empty record for it.
- changed:
options starts with a Colour option holding the product's colour:
the designer's (as extraInfo.color), or Farfetch's colour family when the
designer's is missing. Farfetch sells each colour as a product of its own,
so the option has one value. Each variant's options lists that colour's id
before the size id, so the size id is no longer the first entry.
- changed:
details.madeIn.origin names the country ("Made in China") where
the US storefront says "Imported", and is in English on every storefront,
also where the record is in another language ("Made in Italy" where the
Japanese storefront in Japanese, ?lang=ja-JP, says "イタリア製").
- changed: composition part names (
details.composition[].name) are in
English on every storefront ("Outer" where the German storefront says
"Außenmaterial").
- changed: with
withSizeFit, measurements that hold for every size, such as
a heel height, are given on every variant; 1.x left most variants'
measurements null.
- changed: each
recommendedProducts item's raw keeps the 1.x fields and
types, filled from Farfetch's catalog, and the recommended products
themselves differ from the website's. promotion, returnDetail and
rankingAlgorithm are always null. There are up to 12 per product, as
before, but often fewer: 1.x always gave 12, 2.0 gives 5 to 11 for about
half of all products.
- changed: listing order. Listings return the products the website lists, as
1.x did (full-price ones, except on sale pages), but the new way of
collecting them brings them in a different order, so a run capped by
maxRequestsPerCrawl gets a different selection of products than 1.x did.
A listing sorted by price (sort=4) keeps the same order from run to run.
- fixed: listings 1.x could not resolve work: search pages
(
/shopping/women/search/items.aspx?q=), and slugs its lookup missed
(/shopping/women/knitwear-1/, /shopping/men/stussy/). A slug the website
does not know is looked up by name within the URL's department
(/de/shopping/women/kleider-1/, /shopping/women/bags-1/); one that names
nothing fails the listing instead of scraping the whole department.
- fixed: the
attributes: listing filter (attributes:11006=19500036, a heel
height) narrows the listing; 1.x ignored it.
- fixed:
maxRequestsPerCrawl: 0 means no limit, as its description says; 1.x
processed no request at all.
- changed: URLs without a country folder (
https://www.farfetch.com/shopping/...)
are priced in the proxy's country when the run sets one
(apifyProxyCountry), as before; without one they are priced for the US in
USD, where 1.x followed whichever country its proxy happened to use.
- changed: the Chinese-language storefronts of Hong Kong, Macau and Taiwan
(
/zh-HK/, /zh-MO/, /zh-TW/) are priced in their country's currency
(HKD, TWD), as /hk/, /mo/ and /tw/ are; the website, and so 1.x,
prices those three in USD, which Farfetch's catalog has no prices in there.
- known limitation: the shipping-from, delivery-speed, sustainability-label and
size url filters are not applied; they are reported in the run log.
- improved listing scrape speed and reliability.
- fixed: scraping the same product from different locales returned only one
record; it now returns one record per locale, each with its own currency.
- fixed: a product pinned to a specific store no longer overwrites the same
product from a category or brand listing.
- restored support for listing filters in the url: colour, designer, price,
discount, size scale, sort order, attributes, exclusive, promotion and tags.
Filters that cannot be applied are reported in the run log instead of being
silently ignored.
- added support for sale listing pages.
- known limitation: the shipping-from and delivery-speed url filters are not
applied yet.
- improved product and listing scrape speed and stability.
- fixed per-variant prices, which were reported as 0 for non-default variants.
details.highlights is now a flat list of strings (previously nested),
matching the dataset schema.recommendedProducts: each item's raw now carries the full product field
set (price, attributes, delivery methods, tags, etc.).
- improve detail data retrieval stability.
- removed module check if product recommends is enabled.
- added support of fetch recommended products' detail.
- added support of fetch variant size/fit info.
- added support of fetch product recommends at most 90.
- added support of madein,label,condition info
- added support of fetch Highlights,Composition,Washing Instructions,Product IDs info
- fix to support new product list page
- update to supported new struct of product list page
- refactor scraper with newest crawlee
- fix
load product page html failed error
- fix bug of extract from product list page
- added support of new product data format
- added currentFormatted field for price
- removed useless color variant in the output if the origin not format it as seperate variant option
