# Novica Artist & Artwork Scraper (`artsiom_k/novica-scraper`) Actor

Scrape Novica.com fair-trade artisan products (jewelry, home decor, clothing, and more, with price, discount, country-of-origin, and materials) and artisan profiles (bio, specialty, rating, real total-item count), with built-in delta mode. No login required.

- **URL**: https://apify.com/artsiom\_k/novica-scraper.md
- **Developed by:** [Artsiom Kunitsyn](https://apify.com/artsiom_k) (community)
- **Categories:** Other, E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## novica-scraper

Scrapes both **products** (handcrafted jewelry, home decor, clothing, and more, with price,
discount, country-of-origin, and materials) and **artisan profiles** from
[Novica.com](https://www.novica.com), a global fair-trade marketplace connecting artisans in over
50 countries directly with buyers — via the site's own category pages and embedded page data. No
login required.

### Contents

- [Key features](#key-features)
- [Output](#output)
- [Input](#input)
- [Input examples](#input-examples)
- [Incremental (delta) mode](#incremental-delta-mode)
- [FAQ](#faq)

### 🔑 Key features

- **Two entity types, one Actor.** Set `entityType` to `artworks` (products) or `artists`
  (artisans).
- **Real country-of-origin data per product** — every product carries the artisan's own region
  (India, Thailand, Africa, the Andes, and dozens more), a genuinely different angle from a typical
  art-marketplace listing.
- **Choose which categories to crawl.** Set `categories` to one or more slugs (e.g. `"jewelry"`,
  `"paintings"`, `"home-decor"`) to scope the crawl, or leave it empty to walk the entire site —
  every top-level category, discovered fresh from Novica's own navigation each run.
- **A real per-artisan rollup with no extra fetch cost per product.** An artisan's own page reports
  a genuine total-item count, rating, and specialty in one fetch — `entityType: "artists"` doesn't
  need to examine every product on the site to build this, just one representative product per
  distinct artisan plus that artisan's own page.
- **Discount tracking.** `price` (current) and `list_price` (pre-discount) are read from two
  independent sources on the page and cross-checked — `on_sale` flags whenever they differ.
- **Raw "Key Features" bullet list** on every product (materials, clasp/finish, a "Made in
  {country}" line) — kept as free text rather than force-parsed into brittle structured fields,
  since the format varies by product type. `made_in` is pulled out separately for convenience.
- **Delta mode built in.** Every run classifies each item as `new`, `changed`, `unchanged`, or
  `delisted` against a persisted baseline — pay for what changed, not a full re-scrape every time.
- **Confirmed-dead listings are never re-fetched.** A removed product or artisan silently redirects
  to the homepage rather than 404ing — detected and tracked in a persisted registry so a future run
  skips it without a wasted request.
- **`maxItems` defaults to 50** — a fast, cheap preview, and what keeps an unconfigured run within
  Apify's automated 5-minute QA check. Clear it (`null`) for a full crawl, or narrow `categories`
  instead.

### 📋 Output

One dataset item per product or artisan, depending on `entityType` — see
[`.actor/dataset_schema.json`](.actor/dataset_schema.json) for the full field list.

**Example product record:**

```json
{
  "source": "novica",
  "entity_type": "artworks",
  "external_id": "175567",
  "url": "https://www.novica.com/p/lapis-lazuli-beaded-anklet-knowledge/175567/",
  "name": "Lapis lazuli beaded anklet, 'Knowledge'",
  "description": "Thought to encourage integrity in relationships, lapis lazuli was once used to access sacred knowledge...",
  "category": "jewelry",
  "subcategory": "anklets",
  "image_url": "https://images1.novica.net/pictures/15/p175567_2.jpg",
  "artist_name": "Narayannii",
  "artist_url": "https://www.novica.com/a/narayannii/6695/",
  "artist_specialty": "Sterling silver & gemstone jewelry",
  "region": "India",
  "currency": "USD",
  "price": 85.49,
  "list_price": 94.99,
  "on_sale": true,
  "availability": "for_sale",
  "sold": false,
  "rating_average": 4.79,
  "rating_count": 734,
  "features": ["Lapis lazuli, sterling silver", "Lobster claw clasp", "Made in India"],
  "made_in": "India",
  "change_type": "new"
}
```

**Example artisan record:**

```json
{
  "source": "novica",
  "entity_type": "artists",
  "external_id": "6695",
  "url": "https://www.novica.com/a/narayannii/6695/",
  "name": "Narayannii",
  "specialty": "Sterling silver & gemstone jewelry",
  "region": "India",
  "rating_average": 4.8,
  "rating_count": 734,
  "item_count": 515,
  "quote": "As the saying goes, it's never too late to fulfill your passion...",
  "bio_preview": "Namastee, I'm Narayani from Delhi, where I grew up in a quiet and peaceful family environment...",
  "image_url": "https://images1.novica.net/pictures/15/a6695_3_size-200.jpg",
  "change_type": "new"
}
```

### 🔧 Input

| Field | Type | Default | Description |
|---|---|---|---|
| `entityType` | string | `artworks` | `artworks` (products) or `artists` (artisans). |
| `categories` | array | — | Restrict the crawl to specific category slugs (e.g. `"jewelry"`, `"paintings"`). Leave empty to crawl the entire site. |
| `startUrls` | array | — | Specific product/artisan URLs to scrape directly instead of discovering via categories. |
| `maxItems` | integer | `50` | Stop after pushing this many items. Set to `null` for a full crawl. |
| `mode` | string | `auto` | `auto` / `full` / `incremental` — see [Incremental mode](#incremental-delta-mode). |
| `concurrency` | integer | `15` | How many product/artisan detail pages to fetch in parallel. |
| `impersonate` | string | `chrome` | curl\_cffi TLS-impersonation target. |
| `proxyConfiguration` | object | off | Apify Proxy config — not needed; no anti-bot friction found. |

### 📥 Input examples

**Default preview (50 products):**

```json
{ "entityType": "artworks" }
```

**Every jewelry product:**

```json
{ "entityType": "artworks", "categories": ["jewelry"], "maxItems": null }
```

**Full artisan directory:**

```json
{ "entityType": "artists", "maxItems": null }
```

**Only paintings and home decor artisans:**

```json
{ "entityType": "artists", "categories": ["paintings", "home-decor"], "maxItems": null }
```

### 🔁 Incremental (delta) mode

`auto` mode does a full scan the first time it runs for a given `entityType`/scope, then only pushes
new/changed items on later runs — including price/discount changes and sold-out transitions
(`price`/`list_price`/`availability` are the watched fields for products; `item_count`/
`rating_average` for artisans). Only an uncapped run using the same `categories` selection each time
(no `maxItems` limit reached, no `startUrls`, no item errors) can detect delistings or update the
baseline.

### ❓ FAQ

**What does "Categories to scrape" actually restrict?**
Use the last segment of a category's novica.com URL — `"jewelry"` for
`https://www.novica.com/jewelry/`. A top-level category like `"jewelry"` already includes every
narrower subcategory (anklets, bracelets, necklaces, ...) combined, so there's no need to list
those separately.

**Why does an artisan sometimes have no bio or rating?**
A real, honest gap on the site's own side — some artisan pages genuinely don't show a bio, rating,
or specialty line at all (confirmed live), typically newer or less-established artisans. Nothing is
guessed or filled in when the page itself has nothing there.

**Why is `bio_preview` truncated?**
The site's own full-bio expansion needs an authenticated session this Actor doesn't establish —
`bio_preview` is the real preview text shown on the page itself, up to the "read more" link.

**Does this need a proxy?**
No — no anti-bot friction was found anywhere on this site while building this Actor (a real
30-request concurrent burst came back 100% clean).

**Does this cover every Novica regional storefront?**
Not in this build — it's scoped to `www.novica.com` (USD pricing) only.

# Actor input Schema

## `entityType` (type: `string`):

Scrape products (handcrafted jewelry, home decor, clothing, etc., with price/discount, country-of-origin, and materials) or artisan profiles (bio, specialty, rating, and a real total-item count — no extra fan-out cost). See dataset\_schema.json for the field set of each.

## `categories` (type: `array`):

Optional: restrict the crawl to specific categories. A top-level category like "Jewelry" already includes every narrower subcategory (anklets, bracelets, necklaces, ...) combined. Leave empty to crawl the entire site (every category listed here, discovered fresh from novica.com's own navigation at build time — see novica-scraper's docs if the site adds a new one later). Applies to both entity types.

## `startUrls` (type: `array`):

Optional list of specific Novica product (https://www.novica.com/p/{slug}/{id}/) or artisan (https://www.novica.com/a/{slug}/{id}/) URLs to scrape directly, instead of discovering via categories — use whichever kind matches entityType. Overrides "Categories to scrape". A startUrls-scoped run cannot detect delistings or update the incremental baseline.

## `maxItems` (type: `integer`):

Stop after pushing this many dataset items. Defaults to 50 — a fast, cheap preview, and what keeps an unconfigured run within Apify's automated 5-minute QA check. The full site has tens of thousands of products across ~2,000 artisans (Novica's own count). Raise this or clear it (set to null) for a full run, or narrow "Categories to scrape" instead.

## `mode` (type: `string`):

"auto" (recommended): full scan on the first run for a given entityType/scope, incremental (new/changed only) afterwards. "full": always push every item and refresh the baseline — schedule this periodically to catch delistings, price changes, and sold-out transitions. "incremental": always push only new/changed items. Only a plain, unscoped run (no startUrls, and the same "Categories to scrape" every time) can detect delistings or update the baseline.

## `concurrency` (type: `integer`):

How many product/artisan detail pages to fetch in parallel. Defaults to 15 — validated live against novica.com with zero blocking at concurrency 30 (no anti-bot layer found on this site at all). Lower it only if you start seeing errors.

## `impersonate` (type: `string`):

curl\_cffi browser TLS-impersonation target. Defaults to "chrome" internally, though no anti-bot friction was found anywhere on this site while building this actor.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration. Leave off unless you start seeing blocks from your own IP reputation — no anti-bot friction was found while building this actor.

## Actor input object example

```json
{
  "entityType": "artworks",
  "maxItems": 50,
  "mode": "auto",
  "concurrency": 15,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("artsiom_k/novica-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("artsiom_k/novica-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call artsiom_k/novica-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,artsiom_k/novica-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/PdivmAsh6ekHA9MZB/builds/HfQdATP6mhsWMweez/openapi.json