Wayfair Scraper

Scrape Wayfair category listings: product name, current price, the strikethrough was-price with the discount percentage worked out, rating and review count. Point it at any category URL and it pages through.

Retries are normal, and raising them is the fix

A small share of requests come back empty on the first try. That is expected and it is not the site closing: asking again clears it, which is why the retries default is 5 rather than 1.

If you see fetch_failed in the run's errors, raise retries before assuming the page is unavailable. Everything else is handled for you.

Where the data actually is

Not in JSON-LD. Wayfair publishes only WebSite and BreadcrumbList there — nothing about products. The catalogue lives in an escaped GraphQL payload embedded in the HTML, which this Actor unescapes and reads.

Two quirks of that payload are worth knowing if you ever compare output against the raw page:

Prices carry a doubled currency symbol: "$$319.99". Stripping the first character, which is the obvious thing to do, turns $319.99 into $19.99. The numbers are extracted rather than trimmed.

Product blocks are anchored on leadPrice. That field occurs exactly once per priced product. displayListingId occurs 205 times on a 50-product page and listingUrl 225 times, because both also appear in tracking metadata. Anchoring on either over-counts fourfold, collapses the extraction windows, and shifts every field by one row — so you get product 3's price on product 2's name, with nothing to warn you. That is the single most dangerous failure this parser could have, and the tests pin it.

The name comes from the product URL slug, because that is the only place the page states a plain product name. Wayfair's marketingCopy is a description and is returned as description, under its real meaning.

Discounts are only reported when they are real

on_sale is set only when both conditions hold: Wayfair marks the price line SALE, and the strikethrough price is genuinely above the current one.

That caution is deliberate. A strikethrough that is not higher than the lead price is a formatting artefact, and on other retail sites a pair of prices that looks like was-and-now often turns out to be a low-to-high range across merchants. Reporting those as markdowns would invent discounts that do not exist. When the pair does not qualify, was_price comes back null rather than being filled in anyway.

discount_percent is computed from the two, so it only ever exists alongside a real markdown.

Fields

Field	What it is
listing_id	Wayfair's own id, e.g. W117455547. Stable, use it to join runs
variant_id	The priced variant
name	Product name from the URL slug
description	Wayfair's marketing copy
manufacturer	Brand, where published
price	Current price
was_price	Strikethrough price, only on a real markdown
on_sale	Whether a genuine discount is live
discount_percent	Percent off
rating, review_count	Average rating and how many reviews
url	Product page

Input reference

Field	Type	Default
category_url	Wayfair category page	sofas
limit	1-2000	96
retries	1-10	5

A non-Wayfair URL is rejected with a message rather than fetched.

Typical uses

• Competitive price tracking. Run a category on a schedule, join snapshots on listing_id, and you have a price history with the discount state attached.
• Discount depth analysis. discount_percent across a category shows how hard a range is being marked down, and on_sale tells you what share of the assortment is discounted at all.
• Assortment and review mining. Rating and review count per listing at category scale shows which products actually sell.

Notes on behaviour

Pages are paced with a short randomised gap. Requests already cost a retry now and then, and hammering the site would raise the block rate for everyone using the same pool.

A page that loads but yields no listing objects is reported as no_products rather than returned as an empty result, and a refused page is never parsed as an empty category. A successful-looking run with zero rows would be the worst outcome here, so both are made loud.

Prices are US dollars from the US site.