Walmart Products Scraper

Products from Walmart.com — keyword search, category shelves and individual item pages. HTTP-only, no API key, no login, no browser.

Modes

Mode	What you get
search	Walmart's own keyword search, walked page by page.
browse	A category shelf, e.g. electronics/3944.
products	The full item document for products you name — long description, full image set, UPC, model number, seller and review histogram.

Six upstream quirks it corrects

1. Every block is an HTTP 200

There is no 403 anywhere on this host. A blocked request returns:

HTTP 200, ~15,561 bytes, `px-captcha` in the body, no __NEXT_DATA__

PerimeterX under a 200 status. A scraper that classifies on the status code sees a clean success for every single block and concludes the shelf is simply empty — a run that "succeeds" with nothing in it, and a log that says nothing went wrong. This actor checks for the data layer and the block marker first, and only then looks at the status. perimeterxBlocksSeen is published on every run.

2. The block is per (TLS profile × surface), and one clean response is not a verdict

Full ladder, 14 profiles, same IP, minutes apart:

Surface	Profiles clean
browse page 1	all 14
browse ?page=2	9 of 14
/ip/{id} detail	3 of 14
/search?q=	1 of 14

Re-running the three detail survivors three times each separated the stable winners from the fluke: safari18_0 passed once and then failed 3/3, while chrome99_android and safari17_2_ios passed 3/3. Then the final matrix — 2 mobile profiles × 4 surfaces × 3 repetitions:

Profile	browse p1	browse pN	detail	search
chrome99_android	OK	OK	OK	blocked 3/3
safari17_2_ios	OK	OK	OK	OK

safari17_2_ios is the only profile that walks every surface, so it is not one option in a rotation — it is the actor. There is deliberately no rotation pool: every alternative is permanently blocked somewhere, so rotating after a failure would move the run onto a profile that can never succeed there. Retries re-try the same profile behind a fresh session and a longer backoff.

3. Search results are split across a dozen stacks, and most are not results

?q=laptop came back as 12 itemStacks:

stack0   LIST      'Results for "laptop"'        6 items
stack1   CAROUSEL  '4 stars and above'           6 items
stack3   CAROUSEL  'Shop trending items'        10 items
stack8   CAROUSEL  "Don't miss these savings"   10 items
stack11  LIST      'Results for "laptop"'       25 items
...

The CAROUSEL stacks are recommendation widgets. They hold real products, they are not results for your query, and concatenating everything inflates one page from 54 rows to 82 while a third of them answer a different question. Only layoutEnum == "LIST" is treated as a result; the rest are counted in carouselRowsExcluded so the discarding is auditable rather than silent.

4. Not every row in a results stack is a product

LIST stacks also carry AdPlaceholder and TileTakeOverProductPlaceholder entries — ad slots with no id, no name and no price, filled in client-side. Skipped and counted as placeholderRowsSkipped.

Sponsored products, by contrast, are kept and flagged. They sit inline with organic results and are indistinguishable by position; which of the two you want is your decision, so every row carries isSponsored.

5. Three disagreeing counts, and the obvious one is the worst

One ?q=laptop response carried, simultaneously:

Field	Value	What it actually is
aggregatedCount	13,208	matches for the query
count	82	rows across all stacks, carousels and ad slots included
gridItemsCount	56	grid rows only

count is the field that reads like a result count and is the least related to one. All three are published under their own names.

And aggregatedCount is not even stable across pages of the same query. Walking one browse shelf and reading it off each page:

p1 486,991 | p2 491,788 | p3 900,010 | p5 900,010 | p10 487,608
p15 900,010 | p20 900,008 | p21 900,009 | p22 0

It flips between roughly 487k and 900k — an 85% swing on an unchanged query. It is read once, from page 1, published as aggregatedCountFirstPage with aggregatedCountIsUnstable: true, and every per-page value ships alongside it in aggregatedCountPerPage so the instability is visible in your data instead of hidden behind one confident number.

6. maxPage over-claims, and disagrees between TLS profiles

The same browse URL told chrome124 its maxPage was 25 and safari17_2_ios that it was 50. Bisected with the latter, the walk really ends at page 21:

p20 -> 27 products | p21 -> 17 | p22 -> 0 | p23 -> 0 | p100 -> 0

Pages past the end return HTTP 200 with zero products — an honest empty page, with no wrap back to page 1 and no clamp to the last page. That empty page is the stop condition; maxPage is published for you as maxPageClaimedByUpstream and never used to decide when to stop.

Bonus: the same item has two incompatible price schemas

On a search or browse row:

itemPrice   "$278.00"   <- NOT what you pay, despite the name
linePrice   "$258.00"   <- what you pay
wasPrice    "$278.00"   <- a plain string

On that same item's own page, the same minute:

currentPrice  {"price": 258, "priceString": "$258.00", "currencyUnit": "USD"}
wasPrice      {"price": 278, ...}          <- an OBJECT this time

wasPrice is a string on one surface and an object on the other. A reader who takes itemPrice for the price overstates this item by $20; one who types wasPrice as a string breaks the moment a detail row arrives. Both shapes reduce to currentPrice / wasPrice / savingsAmount as plain floats, the raw object survives as priceInfo, and priceShape records which schema each row came from.

currency is reported only where upstream states it — detail rows carry USD, listing rows carry no currency code anywhere in the payload, and inventing one is a guess this actor does not make.

Output

One SEARCH_SUMMARY row per run, one PRODUCT row per product, one ERROR row per item that could not be fetched.

PRODUCT rows carry the upstream object verbatim, plus normalised twins: itemId, productUrl, productName, brandName, productType, modelNumber, upc, sellerName, availability, averageRating, reviewCount, currentPrice, wasPrice, savingsAmount, currency, isPriceReduced, isSponsored, resultRank, priceShape and metadataSource.

Limits

• There is no reachable total. Walmart's headline match count is unstable and its maxPage over-claims; a shelf claiming 487,000 products really ran out at page 21 (~600 rows). What you can reach is what the walk returns.
• Listing pages are sequential; maxConcurrency applies to detail fetches only.
• A detail miss degrades that one row back to its listing shape — it never drops the row and never fails the walk.
• Prices and availability are national defaults. This actor does not set a store or a ZIP, so nothing here is store-level inventory.