All notable changes to the Tokopedia Product Scraper will be documented in this file.
shopId field - unique seller identifier
shopUrl field - direct link to seller shop
isOfficialStore field - boolean for verified sellers
searchKeyword field - echo of search term used
- User-agent rotation (4 different UAs)
- 429 rate limit handling with exponential backoff
- 403 forbidden detection with graceful skip
- Request headers to better mimic real browser traffic
- Error handling and logging
- Initial GraphQL API version
- Replaced Playwright DOM scraping with HTTP requests
- Uses Tokopedia's internal
SearchProductV5Query endpoint
sellerTier - Seller tier level (1-4)
productId - Unique product identifier
category - Product category name
categoryBreadcrumb - Full category path
hasFreeShipping - Free shipping indicator
- Playwright/Crawlee dependencies (no longer needed)
- DOM-based selectors (replaced with API parsing)
- Initial release with Playwright-based scraping
- Search by keyword or category URL
- Basic product data extraction
- Anti-bot detection caused 0 products returned
- Replaced by v2.0.3 GraphQL approach
Maintained by Strive