novica-scraper

Scrapes both products (handcrafted jewelry, home decor, clothing, and more, with price, discount, country-of-origin, and materials) and artisan profiles from Novica.com, a global fair-trade marketplace connecting artisans in over 50 countries directly with buyers — via the site's own category pages and embedded page data. No login required.

Contents

• Key features
• Output
• Input
• Input examples
• Incremental (delta) mode
• FAQ

🔑 Key features

• Two entity types, one Actor. Set entityType to artworks (products) or artists (artisans).
• Real country-of-origin data per product — every product carries the artisan's own region (India, Thailand, Africa, the Andes, and dozens more), a genuinely different angle from a typical art-marketplace listing.
• Choose which categories to crawl. Set categories to one or more slugs (e.g. "jewelry", "paintings", "home-decor") to scope the crawl, or leave it empty to walk the entire site — every top-level category, discovered fresh from Novica's own navigation each run.
• A real per-artisan rollup with no extra fetch cost per product. An artisan's own page reports a genuine total-item count, rating, and specialty in one fetch — entityType: "artists" doesn't need to examine every product on the site to build this, just one representative product per distinct artisan plus that artisan's own page.
• Discount tracking. price (current) and list_price (pre-discount) are read from two independent sources on the page and cross-checked — on_sale flags whenever they differ.
• Raw "Key Features" bullet list on every product (materials, clasp/finish, a "Made in {country}" line) — kept as free text rather than force-parsed into brittle structured fields, since the format varies by product type. made_in is pulled out separately for convenience.
• Delta mode built in. Every run classifies each item as new, changed, unchanged, or delisted against a persisted baseline — pay for what changed, not a full re-scrape every time.
• Confirmed-dead listings are never re-fetched. A removed product or artisan silently redirects to the homepage rather than 404ing — detected and tracked in a persisted registry so a future run skips it without a wasted request.
• maxItems defaults to 50 — a fast, cheap preview, and what keeps an unconfigured run within Apify's automated 5-minute QA check. Clear it (null) for a full crawl, or narrow categories instead.

📋 Output

One dataset item per product or artisan, depending on entityType — see .actor/dataset_schema.json for the full field list.

Example product record:

{
  "source": "novica",
  "entity_type": "artworks",
  "external_id": "175567",
  "url": "https://www.novica.com/p/lapis-lazuli-beaded-anklet-knowledge/175567/",
  "name": "Lapis lazuli beaded anklet, 'Knowledge'",
  "description": "Thought to encourage integrity in relationships, lapis lazuli was once used to access sacred knowledge...",
  "category": "jewelry",
  "subcategory": "anklets",
  "image_url": "https://images1.novica.net/pictures/15/p175567_2.jpg",
  "artist_name": "Narayannii",
  "artist_url": "https://www.novica.com/a/narayannii/6695/",
  "artist_specialty": "Sterling silver & gemstone jewelry",
  "region": "India",
  "currency": "USD",
  "price": 85.49,
  "list_price": 94.99,
  "on_sale": true,
  "availability": "for_sale",
  "sold": false,
  "rating_average": 4.79,
  "rating_count": 734,
  "features": ["Lapis lazuli, sterling silver", "Lobster claw clasp", "Made in India"],
  "made_in": "India",
  "change_type": "new"
}

Example artisan record:

{
  "source": "novica",
  "entity_type": "artists",
  "external_id": "6695",
  "url": "https://www.novica.com/a/narayannii/6695/",
  "name": "Narayannii",
  "specialty": "Sterling silver & gemstone jewelry",
  "region": "India",
  "rating_average": 4.8,
  "rating_count": 734,
  "item_count": 515,
  "quote": "As the saying goes, it's never too late to fulfill your passion...",
  "bio_preview": "Namastee, I'm Narayani from Delhi, where I grew up in a quiet and peaceful family environment...",
  "image_url": "https://images1.novica.net/pictures/15/a6695_3_size-200.jpg",
  "change_type": "new"
}

🔧 Input

Field	Type	Default	Description
entityType	string	artworks	artworks (products) or artists (artisans).
categories	array	—	Restrict the crawl to specific category slugs (e.g. "jewelry", "paintings"). Leave empty to crawl the entire site.
startUrls	array	—	Specific product/artisan URLs to scrape directly instead of discovering via categories.
maxItems	integer	50	Stop after pushing this many items. Set to null for a full crawl.
mode	string	auto	auto / full / incremental — see Incremental mode.
concurrency	integer	15	How many product/artisan detail pages to fetch in parallel.
impersonate	string	chrome	curl_cffi TLS-impersonation target.
proxyConfiguration	object	off	Apify Proxy config — not needed; no anti-bot friction found.

📥 Input examples

Default preview (50 products):

{ "entityType": "artworks" }

Every jewelry product:

{ "entityType": "artworks", "categories": ["jewelry"], "maxItems": null }

Full artisan directory:

{ "entityType": "artists", "maxItems": null }

Only paintings and home decor artisans:

{ "entityType": "artists", "categories": ["paintings", "home-decor"], "maxItems": null }

🔁 Incremental (delta) mode

auto mode does a full scan the first time it runs for a given entityType/scope, then only pushes new/changed items on later runs — including price/discount changes and sold-out transitions (price/list_price/availability are the watched fields for products; item_count/ rating_average for artisans). Only an uncapped run using the same categories selection each time (no maxItems limit reached, no startUrls, no item errors) can detect delistings or update the baseline.

❓ FAQ

What does "Categories to scrape" actually restrict? Use the last segment of a category's novica.com URL — "jewelry" for https://www.novica.com/jewelry/. A top-level category like "jewelry" already includes every narrower subcategory (anklets, bracelets, necklaces, ...) combined, so there's no need to list those separately.

Why does an artisan sometimes have no bio or rating? A real, honest gap on the site's own side — some artisan pages genuinely don't show a bio, rating, or specialty line at all (confirmed live), typically newer or less-established artisans. Nothing is guessed or filled in when the page itself has nothing there.

Why is bio_preview truncated? The site's own full-bio expansion needs an authenticated session this Actor doesn't establish — bio_preview is the real preview text shown on the page itself, up to the "read more" link.

Does this need a proxy? No — no anti-bot friction was found anywhere on this site while building this Actor (a real 30-request concurrent burst came back 100% clean).

Does this cover every Novica regional storefront? Not in this build — it's scoped to www.novica.com (USD pricing) only.