# Etsy Ultimate Data Extractor (All-in-One) (`fanndev/etsy-ultimate-data-extractor`) Actor
Recon build.
- **URL**: https://apify.com/fanndev/etsy-ultimate-data-extractor.md
- **Developed by:** [Faisal Ahdan naufal](https://apify.com/fanndev) (community)
- **Categories:** E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
from $1.40 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.
Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event
## What's an Apify Actor?
An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.
For examples already wired to this Actor's own input schema, see the [API](#api) section below.
Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).
# README
## Etsy Ultimate Data Extractor (All-in-One)
Five Etsy scrapers in one actor: product discovery, shop analytics, deep product
detail, reviews with sentiment, and a chained crawl that runs the lot from a
single keyword. One input schema, one dataset, one relational record shape.
No login, no API key, no browser. Every surface it reads is one Etsy's
`robots.txt` allows.
***
### Modes
| Mode | What it does | Records it emits |
|---|---|---|
| **search** | Products from category pages; shops and their keyword-matched products from keywords | `search_result`, `shop_result` |
| **shop** | Seller metrics plus the whole active catalogue | `shop_profile`, `search_result`, optionally `review` |
| **product** | One listing in full: description, variants, stock, taxonomy | `product_detail`, optionally `review` |
| **reviews** | Reviews, ratings, buyer photos, and Etsy's AI sentiment tags | `review`, `review_sentiment` |
| **deepCrawl** | search → every product it finds → that product's reviews, in one run | all of the above |
#### Search
Two entry points, and they behave differently for a reason.
**A category URL** (`https://www.etsy.com/c/jewelry/rings`) walks the product
grid: 64 products per page, up to page 250 — about 16,000 listings per
category. Paste any `/c/...` URL from Etsy's own category navigation.
**A keyword** goes through shops. Etsy's `robots.txt` disallows `/search?q=`
product search for *every* crawler, so this actor does not request it. It uses
`/search/shops`, which Etsy explicitly allows with its own `Allow:` line, to
find the shops that match the keyword, then searches each of those shops'
catalogues for the same keyword. You get keyword-matched products plus the
sellers behind them. A nonsense keyword correctly returns nothing.
`shopsPerKeyword` controls how many matched shops get searched (default 12, one
full page of shop results). Shops and their products are interleaved, so a
small `maxItems` still returns products rather than a page of shops.
#### Deep crawl
```
mode: deepCrawl
searchKeywords: ["vintage ring"]
deepCrawlProducts: 10
maxItems: 200
```
Finds the shops, pulls their keyword-matched listings, then opens the first 10
products in full and collects their reviews and sentiment tags. Half the item
budget is reserved for the discovery pass so the detail stage always has room.
***
### Output
Every record has the same envelope, so products, shops, reviews and diagnostics
can share one dataset and still be joined:
```json
{
"item_type": "search_result",
"id": "4306368113",
"data": { "...": "depends on item_type" },
"metadata": {
"scrapedAt": "2026-09-21T09:14:03+00:00",
"sourceUrl": "https://www.etsy.com/uk/c/jewelry/rings",
"mode": "search",
"keyword": "vintage ring",
"rank": 5,
"page": 1,
"parentId": "PBJewelryGift",
"locale": "uk"
}
}
```
Join `metadata.parentId` to the `id` of the shop or product a row hangs off.
`metadata.rank` is the position across the whole target, continuing across
pages, so it survives pagination.
#### A sample of each type
product_detail
```json
{
"listingId": "4306368113",
"title": "Birth Flower Ring 925 Sterling Silver ...",
"description": "",
"shopName": "PBJewelryGift",
"shopTotalSales": 59500,
"shopTenure": "9 years",
"categoryPath": "Jewelry < Rings < Multi-Stone Rings",
"breadcrumbs": [{ "name": "Jewelry", "url": "https://www.etsy.com/c/jewelry" }],
"price": 27.58, "priceMin": 27.58, "priceMax": 34.81,
"originalPrice": 39.39, "currency": "GBP",
"availability": "InStock", "quantityAvailable": 431,
"lowStockRemaining": null, "peopleWithItemInCart": null,
"shipsFrom": "US",
"rating": 4.7, "reviewCount": 2282,
"images": [{ "url": "...", "thumbnail": "...", "alt": "May include: A delicate gold ring ..." }],
"variations": [{ "name": "Primary colour", "options": ["Silver", "Gold", "Rose gold"] }],
"highlights": ["Made by PBJewelryGift", "Materials: Sterling silver", "Adjustable"],
"reviewFeatureTags": ["Looks great", "Love it", "Perfect gift", "Fast shipping"],
"listingBadge": "etsys_pick"
}
```
review
```json
{
"reviewId": "5185546518",
"rating": 5,
"text": "This ring is so pretty, high quality.",
"date": "20 Sept, 2026",
"buyerName": "Etsy buyer",
"buyerProfileUrl": null,
"photos": ["https://i.etsystatic.com/iap/.../iap_300x300....jpg"],
"sellerResponse": null,
"purchasedListingId": "4509331836",
"purchasedListingTitle": "Sterling Silver Dangling Ball Ring ..."
}
```
shop_profile
```json
{
"shopName": "PBJewelryGift",
"location": "California, United States",
"totalSales": 59504,
"onEtsySince": 2017,
"rating": 4.5, "ratingAllTime": 4.5, "ratingRecencyWeighted": 4.7,
"reviewCount": 9023,
"announcement": "Welcome to my little handmade jewelry studio ...",
"members": [{ "name": "Gia Lur", "role": "owner, maker" }],
"faqs": [{ "question": "What carrier do you use?", "answer": "We currently use USPS only." }]
}
```
***
### Things worth knowing before you run it
These are measured facts about Etsy, not caveats about the actor. They change
what you should ask it for.
#### Grid stars describe the seller, not the item
A card in a category grid shows the **shop's** rating and review count. The card
for listing `4306368113` reports 4.72 and 9,023 — that listing's own page
reports 4.7 and **2,282**, and its shop reports exactly 9,023. Every card from
one shop in a grid carries the same pair.
The fields are therefore named `shopRating` and `shopReviewCount`, and
`minRating` / `minReviewCount` filter on them. For per-item ratings, run
`product` mode (or `deepCrawl`, which does it for you).
#### `bestsellerOnly` returns almost nothing right now
Etsy does not render the Bestseller or Star Seller badge into server-side HTML
on grid pages at all. The page's own listing-card config reports
`badges_disabled: true` and the `quality_signals.bestseller_logic_desktop`
experiment is `off` — confirmed across several locales and exit countries.
The option is still here because it starts working the moment Etsy re-enables
the badge, and the parser does read badges where Etsy *does* emit them (listing
pages carry `etsys_pick`). But for popularity filtering today, use
`minReviewCount` and `minRating`. The actor logs a warning at startup if you
turn `bestsellerOnly` on.
#### Seller SEO tags are gone
Etsy used to link a listing's seller tags at the bottom of its page as
`/market/` links. There is not one left on a current listing page.
`sellerTags` is always `null` rather than a guess. What you get instead, and
what most tag analysis was really after:
- `categoryPath` and `breadcrumbs` — Etsy's own taxonomy placement
- `highlights` — the structured attributes (materials, gemstone, made-to-order)
- `reviewFeatureTags` — Etsy's AI summary of what buyers say
- the full `title` and `description`
#### Product URLs need their exact slug
`https://www.etsy.com/listing/4306368113/birth-flower-ring-925-sterling-silver`
works. `https://www.etsy.com/listing/4306368113` does not, and neither does any
wrong slug: Etsy answers those with a redirect to the canonical URL, and its bot
protection blocks the redirect rather than following it. A bare listing ID
cannot be resolved into a URL from outside.
This is why the modes chain. Grids and shop pages emit the full href, so
`search` → `product` works; typing an ID into `productUrls` does not, and the
actor tells you so in an `unresolvable_listing` row instead of failing quietly.
#### Currency follows the exit IP unless you pin it
The same ring came back priced in IDR, EUR, GBP, AUD and CAD purely by changing
the proxy's exit country. Set `locale` to pin it: `uk` → GBP, `de-en` → EUR,
`au` → AUD, and so on. Leave it empty only if you are running through a US exit
and want USD.
`search_result` and `product_detail` carry both `currency` (the ISO code, from
Etsy's JSON-LD) and `currencySymbol` (as rendered).
#### Use residential proxies
Etsy is behind DataDome. In testing, datacentre addresses were challenged on
every surface, including the category grid that residential addresses cleared.
Without any proxy, one address degrades after roughly 30 requests. The default
proxy configuration is `RESIDENTIAL`; the actor warns if you change it.
When a page is challenged the actor rotates TLS fingerprint and retries; a URL
that every fingerprint fails on produces a `challenge_wall` diagnostic row
rather than a silent gap.
#### Ceilings
| Surface | Per page | Ceiling |
|---|---|---|
| Category grid | 64 products | page 250 (~16,000 per category) |
| Shop keyword search | 12 shops | until results run out |
| Shop catalogue | ~36–46 products | until results run out |
| Shop reviews | 14 reviews | until results run out |
| Listing page | 4 reviews inline | the rest come from the shop's reviews pages |
To go deeper than a category's 16,000, split it into its subcategories — each
gets its own 250 pages.
***
### Cost control
`maxItems` is one ceiling for the whole run, shared across every stage of a
chained crawl. The settings that actually drive request count:
- `maxPagesPerTarget` — pages per category, shop or keyword
- `shopsPerKeyword` — one request per shop searched
- `deepCrawlProducts` — one request per product opened in full
- `maxReviewPages` — one request per 14 reviews
A category page is about 1 MB, so a long run is bandwidth-heavy on residential
proxy. `requestDelaySecs` defaults to 2 and lowering it raises the block rate.
For scale: a verification run on the Apify platform — one category page, three
products opened in full, ten reviews, 28 records — took 110 s and 0.015 compute
units at 512 MB, peaking at 71 MB. The default memory is 512 MB for that reason;
raising it costs more without making anything faster.
***
### Compliance
`robots.txt` was re-read on 2026-09-21. It names no AI crawler (no `Claude`,
`anthropic-ai`, `CCBot` or `GPTBot` rule). This actor requests none of the paths
disallowed for `User-agent: *`:
- `/search?q=` product search, in any of its four disallowed spellings
- `/api/` — every internal AJAX endpoint, which is why deep reviews come from
the shop's own paginated reviews pages instead
- `*/shop/*/sold*`, `*/shop/*/favoriters*`, `*/listing/*/favoriters*`
- `*/shop/*/rss_shop.php`
- `/*?order=*` — so results come back in Etsy's default relevancy order and no
sort parameter is ever sent
- `/c/*attr_*=*`, `/c/*price_bucket=*`, `/c/*ship_to=*`, `/c/*search_type=*`
Only public pages are read. Nothing requires a login, and no personal data
beyond the reviewer display names Etsy itself publishes on a public page is
collected.
Two surfaces that look useful are deliberately unused because they are decoys:
`/c?q=` answers 200 with a full grid that is byte-for-byte identical
for every keyword including nonsense ones (the `q` is parsed and discarded), and
`/market/` carries only an eight-card promotional strip rather than a
result grid.
***
### Development
```bash
python test_local.py # 70+ parser checks against saved pages, offline
python test_local.py --live # plus a few real requests, one per mode
```
The fixtures in `tests/fixtures/` are real Etsy pages captured on 2026-09-21.
They are what keeps a markup change visible as a failing assertion instead of a
dataset full of nulls.
`CRAWLING_METHOD.md` documents how each gate was found and why each surface was
chosen.
# Actor input Schema
## `mode` (type: `string`):
Which extraction to run. 'search' discovers products and shops, 'shop' profiles a seller and its catalogue, 'product' pulls one listing in full, 'reviews' pulls reviews and sentiment, and 'deepCrawl' chains search into product and reviews in one pass.
## `searchKeywords` (type: `array`):
Keywords for search and deepCrawl. IMPORTANT: Etsy's robots.txt disallows /search?q= product search for every crawler, so this actor uses the surface Etsy explicitly allows instead - /search/shops - and then searches each matched shop's own catalogue for the keyword. You get keyword-matched products and the shops behind them, without a disallowed request. Nonsense keywords correctly return nothing.
## `startUrls` (type: `array`):
Etsy URLs to process. Category pages (https://www.etsy.com/c/jewelry/rings), shop pages (https://www.etsy.com/shop/PBJewelryGift), product pages (https://www.etsy.com/listing/4306368113/birth-flower-ring-925-sterling-silver) and shop searches (https://www.etsy.com/search/shops?q=candle) all work; the actor routes each by shape. A product URL must include its exact slug - see productUrls for why.
## `productUrls` (type: `array`):
Full Etsy listing URLs for product and reviews modes, as an alternative to startUrls. The exact slug is required: Etsy answers /listing/ and any wrong slug with a redirect, and DataDome blocks the redirect, so a bare listing ID cannot be resolved. Paste the url field of any search\_result record.
## `shopNames` (type: `array`):
Shop names (e.g. PBJewelryGift) or shop URLs for shop and reviews modes. Bare names are fine - the actor builds the URL itself, including the locale prefix that Etsy's bot protection requires on shop pages.
## `maxItems` (type: `integer`):
Hard ceiling on dataset records for the whole run, shared across every stage of a chained crawl. 0 means no limit. This is the main control over compute cost.
## `maxPagesPerTarget` (type: `integer`):
How many result pages to walk per category, shop or keyword. A category page holds 64 products, a shop search page 12 shops, a shop reviews page 14 reviews. Category grids stop at page 250 no matter what you set here.
## `proxyConfiguration` (type: `object`):
Etsy is behind DataDome. Residential proxies are strongly recommended: during recon, datacentre addresses were challenged on every surface tested, including the category grid that residential cleared. Without a proxy, request rate from one address degrades results after roughly 30 requests.
## `locale` (type: `string`):
Etsy's locale path prefix. Prices otherwise follow the proxy's exit country - the same ring came back in IDR, EUR, GBP, AUD and CAD purely by changing the exit - so set this if you need a stable currency. Leave empty for Etsy's default US site (USD from a US exit). Shop pages always get a prefix regardless, because the unprefixed shop path is blocked.
## `extractReviews` (type: `boolean`):
In shop and product modes, also emit review records. A product page carries only the handful of reviews Etsy renders inline; the deep set comes from the shop's reviews pages, 14 per page.
## `deepReviews` (type: `boolean`):
In reviews mode, after reading a product's inline reviews, also walk its shop's reviews pages and keep every review left for that product. Costs one request per 14 reviews.
## `maxReviewPages` (type: `integer`):
How many shop reviews pages to walk (14 reviews each). Defaults to maxPagesPerTarget when unset.
## `deepCrawlProducts` (type: `integer`):
In deepCrawl, how many of the discovered products to open in full. Each one is a separate page request, so this is the setting that decides the run's cost.
## `shopsPerKeyword` (type: `integer`):
How many of the shops matched by a keyword to search for products. Each shop costs one request and returns up to 36 keyword-matched listings.
## `bestsellerOnly` (type: `boolean`):
Keep only products carrying a Bestseller or Star Seller badge. READ THIS FIRST: Etsy currently does not render these badges into server-side HTML on grid pages at all - the page's own config reports badges\_disabled and the bestseller experiment is off - so this filter returns almost nothing today. It is kept because it starts working the moment Etsy re-enables the badge. For popularity filtering that works now, use minReviewCount and minRating.
## `minReviewCount` (type: `integer`):
Keep only products whose seller has at least this many reviews. Grid cards publish the SHOP's review count, not the item's (Etsy renders the seller's stars on every card), so this is a seller-quality filter - and it is the popularity signal that actually works here, unlike bestsellerOnly. For per-item review counts, run product mode.
## `minRating` (type: `integer`):
Keep only products whose seller is rated at least this many stars (1-5). Like minReviewCount this reads the shop's rating, because that is what a grid card carries.
## `includeAds` (type: `boolean`):
Category and shop grids mix in promoted listings, which Etsy renders without a shop name. They are dropped by default; turn this on to keep them, flagged with isAd.
## `requestDelaySecs` (type: `integer`):
Floor between requests. DataDome scores request rate per address, so lowering this raises the block rate on long runs.
## `maxRetries` (type: `integer`):
Attempts per URL before giving up. Each retry rotates to a different TLS fingerprint, which is what actually clears a DataDome challenge, so this is at least the number of profiles in the ladder.
## `requestTimeoutSecs` (type: `integer`):
Per-request timeout. Etsy pages run from 200 KB to 1 MB, so keep this generous on residential proxies.
## Actor input object example
```json
{
"mode": "search",
"searchKeywords": [
"vintage ring"
],
"startUrls": [
{
"url": "https://www.etsy.com/c/jewelry/rings"
}
],
"maxItems": 200,
"maxPagesPerTarget": 5,
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
]
},
"locale": "uk",
"extractReviews": false,
"deepReviews": false,
"deepCrawlProducts": 10,
"shopsPerKeyword": 12,
"bestsellerOnly": false,
"includeAds": false,
"requestDelaySecs": 2,
"maxRetries": 6,
"requestTimeoutSecs": 60
}
```
# Actor output Schema
## `results` (type: `string`):
Every product, shop, review and diagnostic record from this run.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"searchKeywords": [
"vintage ring"
],
"startUrls": [
{
"url": "https://www.etsy.com/c/jewelry/rings"
}
]
};
// Run the Actor and wait for it to finish
const run = await client.actor("fanndev/etsy-ultimate-data-extractor").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"searchKeywords": ["vintage ring"],
"startUrls": [{ "url": "https://www.etsy.com/c/jewelry/rings" }],
}
# Run the Actor and wait for it to finish
run = client.actor("fanndev/etsy-ultimate-data-extractor").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"searchKeywords": [
"vintage ring"
],
"startUrls": [
{
"url": "https://www.etsy.com/c/jewelry/rings"
}
]
}' |
apify call fanndev/etsy-ultimate-data-extractor --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,fanndev/etsy-ultimate-data-extractor"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/cKvTwNGbeEsgek2Qi/builds/Puta1sgjXJLLhaFBY/openapi.json