Barnes & Noble Intelligence Scraper

Scrape Barnes & Noble catalog data with pagination-aware discovery, book metadata, product URLs, ratings/prices when page access allows them, and commerce intelligence.

This actor is built for publishing, retail analytics, book-market research, catalog enrichment, and product URL discovery. It uses structured data when Barnes & Noble pages are accessible, falls back to visible product cards, follows pagination, and includes a public sitemap fallback so protected pages do not turn into empty runs.

What It Extracts

• Product title and URL
• Author / contributor
• ISBN-13, ISBN-10, UPC, SKU when available
• Format, publisher, and publication date
• Price, list price, currency, discount percent
• Availability / stock text
• Rating and review count
• Image URL
• Product description / overview
• Search rank and source query
• Buyer signals and an intelligence score

Intelligence Fields

Every row is enriched with:

• discountPercent: detected markdown from list price to current price
• intelligenceScore: 0-100 score based on metadata completeness, price, availability, rating, and reviews
• buyerSignals: tags such as high-rated, review-heavy, discounted, preorder, availability-risk, and book-format-detected
• opportunitySummary: compact readable summary for dashboards or alerts

The actor also writes SUMMARY and OUTPUT key-value-store records with product count, price stats, format breakdown, availability breakdown, top authors, best enriched rows, and warnings.

Pagination

The actor follows explicit next-page links when available and respects maxPages. The redesigned search currently returns its first page (usually up to 24 products). Its separate browser-verified Load More request is not supported yet. If more results exist and the requested limits require them, the actor writes a partial-results warning to the log and SUMMARY instead of silently claiming completeness.

Increasing maxItems or maxPages cannot currently fetch later search pages. Multiple targeted queries can still be supplied. Increase requestDelayMs for multiple requests if the site starts pushing back.

Input

• queries: Barnes & Noble search terms, for example atomic habits, romantasy paperback, manga box set
• startUrls: search, category, or product URLs from barnesandnoble.com
• maxItems: maximum unique products to save
• maxPages: maximum pages to visit across search/category/detail/pagination requests
• includeProductDetails: opens detail pages for richer metadata
• includePagination: follows next-page links
• minIntelligenceScore: optional post-filter
• requestDelayMs: delay between requests
• useSitemapFallback: returns matching products from the public Barnes & Noble product sitemap when search/product pages are blocked
• maxSitemapFiles: maximum product sitemap files to scan in fallback mode (default 2, capped at 50)
• proxyConfiguration: Apify proxy configuration

Example:

{ "queries": ["atomic habits", "manga box set"], "maxItems": 100, "maxPages": 10, "includeProductDetails": true, "includePagination": true, "useSitemapFallback": true }

Output Example

{ "title": "Atomic Habits", "author": "James Clear", "isbn13": "9780735211292", "price": 16.99, "currency": "USD", "availability": "InStock", "rating": 4.8, "reviewCount": 12345, "url": "https://www.barnesandnoble.com/w/atomic-habits-james-clear/1129201155", "discountPercent": 0, "intelligenceScore": 86, "buyerSignals": ["high-rated", "review-heavy"], "opportunitySummary": "by James Clear | $16.99 | 4.8 rating | high-rated, review-heavy" }

Reliability Notes

The actor parses the current site's streamed search data and structured product details, with Firefox browser and sitemap fallbacks. The site can deny requests, especially Load More continuation requests. Sitemap fallback rows are marked dataLevel: "discovery" and prioritize product discovery. They usually include title, inferred author, Barnes & Noble product URL, SKU, and sitemapLastModified; sitemap modification time is not a publication date. Live price/rating/availability require page access.

For best reliability:

• Keep includeProductDetails on for smaller, richer runs
• Increase requestDelayMs for large catalog runs
• Use Apify Residential Proxy
• Keep sitemap fallback enabled for smoke tests and QA runs
• Keep maxPages bounded