# Etsy Market Research Scraper (`valev-lab/etsy-scraper`) Actor

Collect Etsy market and category listings with prices, shops, SEO keywords, demand signals, and competition metrics.

- **URL**: https://apify.com/valev-lab/etsy-scraper.md
- **Developed by:** [Daniel Valev](https://apify.com/valev-lab) (community)
- **Categories:** E-commerce
- **Stats:** 2 total users, 1 monthly users, 50.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.98 / 1,000 listings

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Etsy Market Research Scraper do?

Etsy Market Research Scraper turns public Etsy market and category result pages into structured listing data. In addition to prices, ratings, shops, badges, and images, it calculates title keywords, price percentiles, price z-scores, a transparent demand signal, shop concentration, and result-set price statistics.

It is designed for product research, pricing analysis, Etsy SEO discovery, catalog benchmarking, and recurring niche snapshots. You can submit multiple keyword queries and category URLs in one run; duplicate listing IDs are stored and charged only once.

The Actor uses fast HTTP requests on Node.js 24. It starts without a proxy and escalates through Apify Shared and Residential proxies only when a response is classified as a challenge or rate limit.

### What can you use it for?

- compare price bands and discounts in a product niche;
- discover recurring words in successful listing titles;
- identify shops with a high share of visible results;
- filter out ads, digital products, low-rated listings, or unwanted keywords;
- export a clean dataset to JSON, CSV, Excel, or an Apify integration;
- schedule repeated market snapshots for downstream trend analysis.

### Input

Provide at least one `searchQueries` value or supported `startUrls` entry. Market and category URLs are supported. Etsy `/search`, listing-detail, shop, and review pages are intentionally excluded from this pure-HTTP MVP because current direct, Shared, and Residential probes receive DataDome challenges on those routes.

```json
{
  "searchQueries": ["handmade necklace", "silver ring"],
  "maxItemsPerQuery": 100,
  "sort": "most_relevant",
  "minPrice": 20,
  "maxPrice": 200,
  "excludeAds": true,
  "excludeKeywords": ["template", "printable"],
  "currency": "USD",
  "locale": "en-US"
}
```

Important controls:

- `maxItemsPerQuery` is an upper bound from 1 to 5,000. Etsy may expose fewer unique matching results.
- `maxItemsTotal` optionally caps the whole run.
- `sort` supports most relevant, lowest price, highest price, and recently listed.
- filters run in the documented schema order; filtered and incomplete records are not stored or charged.
- `aggressiveDepth` tries additional measured sort orders after a route stops yielding new listings.
- `proxyConfiguration` defaults to an Apify Residential fallback. No Etsy API key, cookie, or login is needed.

The no-input health check requests exactly five listings for `handmade necklace`.

### Output

Each complete listing is written to the default dataset with `recordType: "listing"`. One uncharged `runSummary` item is written last and reports delivered depth, duplicate suppression, filters, route yield, network retries, price statistics, top keywords, and shop concentration.

```json
{
  "recordType": "listing",
  "listingId": "984947628",
  "url": "https://www.etsy.com/listing/984947628/7th-anniversary-gift-for-wife-7-years",
  "title": "7th anniversary gift for wife ...",
  "price": 195.29,
  "originalPrice": 278.99,
  "currency": "USD",
  "discountPercent": 30,
  "shopName": "BestArtisanGifts",
  "titleKeywords": ["7th", "anniversary", "gift", "wife"],
  "pricePercentileInResultSet": 75,
  "priceZScore": 0.81,
  "estimatedDemandSignal": 4.62,
  "query": "handmade necklace",
  "position": 1,
  "page": 1,
  "sourceRoute": "market:most_relevant"
}
```

`estimatedDemandSignal` is not sales data. It is calculated as `ln(1 + reviews) + 2 × ln(1 + purchases shown in the last 24 hours) + 0.5 × ln(1 + carts shown)`. Missing visible signals contribute zero. `competitionScore` is the Herfindahl concentration index across shops, scaled from 0 to 10,000; a higher value means the visible results are concentrated among fewer shops.

Fields that require listing-detail pages—description, tags, materials, personalization, variants, and videos—remain null or empty in this MVP. The Actor never fabricates them.

#### Listing field reference

| Fields | Meaning |
| --- | --- |
| `recordType`, `listingId`, `url`, `title` | Record discriminator and stable public listing identity. Records missing any required identity field are rejected. |
| `query`, `position`, `page`, `sourceRoute`, `scrapedAt` | Search intent, one-based placement, source page, successful depth route, and extraction timestamp. |
| `price`, `originalPrice`, `currency`, `discountPercent`, `onSale` | Numeric card price data and calculated discount. Unknown values are null, never zero-filled. |
| `freeShipping`, `shippingText`, `availability`, `quantityAvailable` | Visible shipping and availability signals; quantity is null when the card does not expose it. |
| `imageUrl`, `imageUrls`, `videoUrl` | Primary and discovered card images; video remains null without detail enrichment. |
| `shopId`, `shopName`, `shopUrl`, `shopLocation`, `shopSales`, `shopReviewCount`, `isStarSeller` | Public shop identity and visible shop signals. Detail-only values remain null. |
| `rating`, `reviewCount` | Numeric rating and integer review count, or null when Etsy does not expose them. |
| `badges` | Booleans for `starSeller`, `freeShipping`, `bestseller`, `etsyChoice`, and `ad`. |
| `isDigitalDownload`, `hasVariations`, `priceMayBeMinimumVariant`, `variantDetectionStatus` | Product-type and variation diagnostics; unknown card-only values remain null. |
| `tags`, `materials`, `personalization` | Reserved detail fields; empty or null in this MVP. |
| `inCartCount`, `boughtInLast24Hours` | Visible demand hints, or null when absent. |
| `pricePercentileInResultSet`, `priceZScore`, `titleKeywords`, `estimatedDemandSignal`, `isLikelyAd` | In-process research metrics calculated without extra requests. |
| `extractionSources`, `detailEnrichmentStatus` | Per-field parser provenance and reserved enrichment status. |

#### Run summary field reference

| Fields | Meaning |
| --- | --- |
| `recordType`, `queries`, `requestedTotal`, `deliveredTotal`, `uniqueListings` | Summary identity, requested intents, requested ceiling, stored listing count, and complete unique candidates. |
| `duplicatesSuppressed`, `rejectedIncomplete`, `filteredOut`, `depthExhaustedReason` | Exact reasons why candidate rows did not become delivered listings. |
| `routesUsed`, `routeYield` | Attempted depth routes and the unique listings each contributed. |
| `resultSetTagFrequency`, `competitionScore`, `priceStats` | Top title-keyword counts, shop HHI, and `min`, `p25`, `median`, `p75`, `max`, and `mean` price. |
| `shopCount`, `topShops` | Distinct represented shops and their listing counts. |
| `blockedRequests`, `retries`, `proxyRungsUsed` | Network ladder diagnostics. |
| `runStartedAt`, `runFinishedAt`, `durationSeconds` | Run timing in ISO UTC timestamps and seconds. |

### Pricing

The recommended launch model is pay per event: Apify's $0.00005 synthetic run-start charge plus $0.00150 per stored listing ($1.50 per 1,000 listings). Duplicate, filtered, incomplete, blocked, and summary records are not charged.

| Listings delivered | Recommended event charge |
| ---: | ---: |
| 5 | $0.00755 |
| 100 | $0.15005 |
| 1,000 | $1.50005 |
| 5,000 | $7.50005 |

These values must match the Actor's Apify Console pricing configuration before publication. The recommended setup keeps **Pay per event + usage** disabled, so normal platform and proxy usage is included in the event price.

### Reliability and limits

The Actor retries each URL through a bounded network ladder with exponential backoff. It stops a route after two consecutive pages add no new listings and reports the reason in `depthExhaustedReason`. It also respects the run's maximum PPE charge budget and stops storing listings when the budget is reached.

Etsy can change its public HTML or anti-bot behavior without notice. A run can therefore return fewer items than requested. Use the `runSummary` record and the `COST_TELEMETRY` key-value-store record when diagnosing yield or proxy cost.

This Actor accesses public, signed-out result pages only. It does not log in, solve CAPTCHAs, or bypass private account boundaries. Use the data for a legitimate purpose and comply with applicable laws and Etsy's terms. This Actor is not affiliated with Etsy.

### Support

If a run produces an unexpected shape or too few results, open an issue with the Actor run ID and non-sensitive input. Do not include proxy credentials, cookies, or authentication tokens.

# Actor input Schema

## `searchQueries` (type: `array`):

Keyword searches to run. Matching is case-insensitive for deduplication, while the first spelling is preserved.

## `startUrls` (type: `array`):

Public Etsy market or category URLs. Listing, shop, and /search pages are not supported by the pure-HTTP MVP because current probes receive DataDome challenges.

## `maxItemsPerQuery` (type: `integer`):

Maximum unique, complete listing records delivered for each query.

## `maxItemsTotal` (type: `integer`):

Optional hard cap on unique listing records across all queries and start URLs.

## `sort` (type: `string`):

Requested Etsy ordering. The Actor maps these stable names to route-specific current Etsy parameters.

## `minPrice` (type: `number`):

Optional minimum item price in the selected currency.

## `maxPrice` (type: `number`):

Optional maximum item price in the selected currency. Must be at least the minimum price.

## `freeShipping` (type: `boolean`):

Request and retain only listings marked with free shipping.

## `onSale` (type: `boolean`):

Request and retain only listings currently marked as discounted.

## `currency` (type: `string`):

Three-letter ISO 4217 currency code used for prices, for example USD or EUR.

## `locale` (type: `string`):

BCP 47 locale used for Etsy content and country-matched residential proxy selection.

## `excludeDigitalDownloads` (type: `boolean`):

Filter step 3: remove listings identified as digital downloads.

## `excludeKeywords` (type: `array`):

Filter step 1: remove titles containing any value, case-insensitively.

## `includeKeywords` (type: `array`):

Filter step 2: keep only titles containing at least one value, case-insensitively. Leave empty to keep all remaining titles.

## `excludeAds` (type: `boolean`):

Filter step 4: remove sponsored or promoted listings.

## `minRating` (type: `number`):

Filter step 5: remove listings below this rating. Listings with unknown ratings are also removed when enabled.

## `minReviewCount` (type: `integer`):

Filter step 6: remove listings below this review count. Listings with unknown counts are also removed when enabled.

## `aggressiveDepth` (type: `boolean`):

Use the additional measured market/category sort orders to pursue the requested unique count.

## `maxConcurrency` (type: `integer`):

Maximum simultaneous HTML requests. Lower this if the target begins rate-limiting the run.

## `proxyConfiguration` (type: `object`):

Proxy settings for the residential fallback rung. Direct and Shared proxy probes are managed separately so the Actor can try cheaper paths first.

## Actor input object example

```json
{
  "searchQueries": [
    "handmade necklace"
  ],
  "startUrls": [
    {
      "url": "https://www.etsy.com/market/handmade_necklace"
    }
  ],
  "maxItemsPerQuery": 100,
  "sort": "most_relevant",
  "freeShipping": false,
  "onSale": false,
  "currency": "USD",
  "locale": "en-US",
  "excludeDigitalDownloads": false,
  "excludeKeywords": [],
  "includeKeywords": [],
  "excludeAds": false,
  "aggressiveDepth": true,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "handmade necklace"
    ],
    "startUrls": [
        {
            "url": "https://www.etsy.com/market/handmade_necklace"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("valev-lab/etsy-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["handmade necklace"],
    "startUrls": [{ "url": "https://www.etsy.com/market/handmade_necklace" }],
}

# Run the Actor and wait for it to finish
run = client.actor("valev-lab/etsy-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "handmade necklace"
  ],
  "startUrls": [
    {
      "url": "https://www.etsy.com/market/handmade_necklace"
    }
  ]
}' |
apify call valev-lab/etsy-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,valev-lab/etsy-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/rgBsj65mqQ9cGvbmm/builds/DwiyfhIQFohule2zU/openapi.json