Etsy Product & Shop Analyzer

An Apify Actor for extracting product listings, shop profiles, sales estimates, reviews, and keyword data from Etsy.

Features

• Shop scraping — Extract full shop profiles including sales count, reviews, rating, member since, location, announcement, and social links.
• Listing scraping — Extract product details: title, description, price, currency, stock, views, favourites, materials, tags, categories, images, and variations.
• Review extraction — Pull customer reviews with author, rating, date, and text.
• Listing discovery — Automatically discover and scrape all listings from a shop page.
• Multi-method parsing — Uses JSON-LD structured data, meta tags, embedded app-state JSON, and HTML parsing for maximum data coverage.
• Rate limiting — Configurable delay between requests to avoid detection.
• Apify proxy support — Uses the Apify proxy pool (residential recommended) to bypass Etsy's anti-bot protection.
• Graceful abort — Handles platform abort signals cleanly.

Input

Field	Type	Default	Description
shopUrls	string[]	[]	List of Etsy shop page URLs (e.g. https://www.etsy.com/shop/CeramicMugShop)
listingUrls	string[]	[]	List of individual listing URLs (e.g. https://www.etsy.com/listing/1234567890/...)
maxListings	integer	50	Max listings to scrape per shop. 0 = unlimited.
includeReviews	boolean	true	Whether to extract customer reviews from listing pages.
maxReviews	integer	10	Max reviews to extract per listing. 0 = all available.
proxy	object	Apify Residential	Proxy configuration for bypassing Etsy's bot protection.
requestDelayMs	integer	2000	Delay between requests in milliseconds.

Output

Each record in the dataset is either a listing or shop type.

Listing fields

Field	Description
listingId	Etsy numeric listing ID
title	Product title
description	Product description
price	Current price
originalPrice	Original price (if on sale)
currency	Currency code (USD, EUR, GBP, etc.)
quantity	Available stock
views	View count
favorites	Heart / favourite count
materials	Array of materials
tags	Array of search tags
categories	Category hierarchy
images	Array of image URLs
variations	Product variations (size, colour, etc.)
shopName	Shop name
shopUrl	Shop page URL
shopLocation	Shop location
shopAverageRating	Shop rating out of 5
shopTotalReviews	Total shop reviews
reviews	Array of review objects
url	Full listing URL
scrapedAt	ISO 8601 scrape timestamp

Shop fields

Field	Description
shopName	Shop name
shopUrl	Shop page URL
shopLocation	Location string
shopTotalSales	Estimated total sales
shopTotalReviews	Total review count
shopAverageRating	Average rating (0–5)
shopMemberSince	Member join date
numberOfListings	Number of active listings
shopAnnouncement	Shop announcement text
socialLinks	Social media profile URLs
url	Full shop URL
scrapedAt	ISO 8601 scrape timestamp

Review object

{
  "author": "CustomerName",
  "rating": 5,
  "review": "Great product!",
  "date": "2024-03-15"
}

Usage

Local development

# Install dependencies
pip install -r requirements.txt

# Run with Apify CLI
apify run --purge

Configuration

Create or modify .actor/input_schema.json to set input defaults. For local runs, Apify reads input from the INPUT.json file in the storage/ directory.

Apify platform

1. Push the actor: apify push
2. Configure input in the Apify Console web UI
3. Run the actor

How it works

The actor uses a multi-strategy parsing pipeline:

1. JSON-LD (<script type="application/ld+json">) — Primary source for structured product and shop data from Etsy's Schema.org markup.
2. Embedded JSON — Scans <script> tags with data-app-state, data-init-state, application/json, and window.__INITIAL_STATE__ assignments for additional data.
3. Meta tags — Open Graph (og:title, og:price:amount, etc.), Twitter Card, and standard meta tags for fallback data.
4. HTML parsing — Visible DOM elements for favourites, stock, materials, tags, reviews, and shop stats.

Requests are sent via httpx.AsyncClient with Chrome user-agent headers, automatic retry on 403/429/503 responses, and exponential backoff.

Notes

• Etsy uses DataDome anti-bot protection. For production use, residential proxies (available through Apify proxy) are strongly recommended.
• Shop listing discovery depends on the shop page HTML structure. Some listings may not be found if the shop uses pagination (only the first page is scraped).
• Reviews are often loaded dynamically. The actor extracts whatever is in the initial HTML — for full review sets, additional API calls may be needed.
• Sales counts are estimates based on what Etsy displays on the shop page, which may be rounded or approximated.

Limitations

• Does not handle shop listing pagination (only the first shop page)
• Review extraction may be incomplete due to Etsy's dynamic loading
• Etsy rate-limits aggressively — keep requestDelayMs at 2000+
• Some fields (views, exact stock) may not be available from all pages

License

MIT