# Shopee Api Scraper (`chartedsea/shopee-api-scraper`) Actor Query Shopee's unofficial API for product searches by keyword, category, or shop. Access detailed information including prices, orders, stock levels, and ratings. Also retrieve related entities like the category tree, shop listings, and keyword suggestions. - **URL**: https://apify.com/chartedsea/shopee-api-scraper.md - **Developed by:** [Charted Sea](https://apify.com/chartedsea) (community) - **Categories:** E-commerce - **Stats:** 940 total users, 4 monthly users, 94.1% runs succeeded, 21 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing $50.00/month + usage To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Shopee API Scraper ### What is Shopee API Scraper and how does it work? Shopee API Scraper allows you to directly invoke the unofficial API of [Shopee](https://shopee.com/), the leading ecommerce marketplace in South East Asia and also present in Central and South America. This unofficial API is used by Shopee for its frontend, that's why it provides all the data displayed on the website. In fact, it actually provides more information (e.g. sales and stock levels per product variant) and is more flexible (e.g. configurable page size). The Shopee API Scraper takes care of all the "low-level" details such as setting the right HTTP headers, to allow you to easily and efficiently requests data. ### Quick Start Click on *Try for free*, copy the following JSON input and click *Start*: ```json { "requests": [ { "url": "https://shopee.sg/Pants-cat.11012963.11027757" }, { "url": "https://shopee.sg/search?keyword=tshirt" }, { "url": "https://shopee.sg/Japanese-Samurai-T-Shirt-New-Samurai-T-Shirt-Distro-T-Shirts-i.414243778.23600133176?sp_atk=43c5e72a-9613-4ee1-9cd7-e29ebcdbd4b9&xptdk=43c5e72a-9613-4ee1-9cd7-e29ebcdbd4b9" }, { "url": "https://shopee.sg/lorealparissg" }, ], "proxy": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "SG" }, } ```` This performs four distinct requests: | Request | URL | |------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Search products in the category "Men's Wear > Pants" | https://shopee.sg/Pants-cat.11012963.11027757 | | Search for products with the keyword "tshirt" | https://shopee.sg/search?keyword=tshirt | | Retrieve detailed product information | https://shopee.sg/Japanese-Samurai-T-Shirt-New-Samurai-T-Shirt-Distro-T-Shirts-i.414243778.23600133176?sp\_atk=43c5e72a-9613-4ee1-9cd7-e29ebcdbd4b9\&xptdk=43c5e72a-9613-4ee1-9cd7-e29ebcdbd4b9 | | Retrieve detailed information about a shop | https://shopee.sg/lorealparissg | After few seconds the Shopee API Scraper returns the following results: - [Product listing for the category "Men's Wear > Pants"](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/f70bdc44b3f5a61b60f43cce414fc9e24636ff29/example-product-search-by-category-page-url.json) - [Product listing for the "tshirt" keyword](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/f70bdc44b3f5a61b60f43cce414fc9e24636ff29/example-product-search-by-keyword-page-url.json) - [Detailed information for the product "Japanese Samurai T-Shirt | New Samurai T-Shirt | Distro T-Shirts"](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/f70bdc44b3f5a61b60f43cce414fc9e24636ff29/example-product-detail-page-url.json) - [Detailed information for the shop "L'Oréal Paris Official Store"](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0fab3e6f12871e8f12d0181bbbcf4bd586a2605a/example-shop-detail-page-url.json) While the example above demonstrates the ease of using page URLs directly from your browser, advanced users might find greater value in tapping into the Shopee unofficial API for enhanced control and access to extended features not available through the website interface. For example, leveraging the API allows for: - Retrieving up to 50 product ratings in a single request, bypassing the standard limit of 6 results per page. - Accessing a broader range of up to 440 popular keyword suggestions, far exceeding the usual cap of 11. For a comprehensive understanding of how to utilize the Shopee API Scraper effectively, including more examples and detailed explanations, please consult the [How to Scrape Shopee?](#how-to-scrape-shopee) section below. ### What data can you extract? Shopee API Scraper seamlessly interacts with Shopee's unofficial API to extract a wide range of data, including: - Product searches by keyword and category. - Detailed product information (description, price, stock levels, sales data, and variants). - Buyer ratings, complete with comments and media. - Hierarchical category trees and detailed category information. - Lists of official shops, filtered by category if needed. - Searches for shops by keyword. - Comprehensive shop details (description, opening date, product counts, and performance metrics). - Shop products. - Keyword suggestions and autocompletion hints. > Note: please visit the [API reference section](#api-reference) below for more details about each API. Advanced users can execute other APIs as well, you can find them by using the [DevTools](https://crawlee.dev/docs/introduction/real-world-project#devtools---the-scrapers-toolbox) "Network" tab. Please [contact me](#your-feedback) if you encounter issues or need additional features. ### Why scrape Shopee data? Discover what you can achieve with the Shopee API Scraper through these example use cases: - **Market and Competitive Analysis**: Track category performance and identify emerging trends by analyzing sales data, product reviews, and competitor strategies. This can reveal insights into market demands and areas for strategic advantage. - **Opportunity Discovery**: Use aggregated product search results to uncover popular yet underserved keywords and niches in the market. This can highlight potential areas for new product development or market entry. - **Product Review Analysis**: Employ AI, such as [ChatGPT](https://openai.com/product), to deeply analyze product reviews. Summarize buyer feedback to gain insights into consumer satisfaction, preferences, and areas for product improvement. - **Competitor Watch**: Regularly scrape competitor data to monitor for critical changes like products going out of stock, pricing adjustments, or new product launches. This enables real-time alerts and swift response strategies. ### How much will it cost you? Here’s a breakdown to help you estimate your expenses: - **Execution Cost**: This includes computing units (CPU and memory), data transfer, and storage operations. On Apify, running the Shopee API Scraper costs approximately **$0.50 for every 1,000 requests**. - **Residential Proxy Cost**: Typically charged per gigabyte, the cost of crawling products with their details is around **0.07GB per 1,000 requests**. In total, scraping 1,000 requests on Apify with the [Starter plan](https://apify.com/pricing), which includes residential proxies, will be around **$1.40 ($0.50 for execution and $0.90 for the proxy)**. As you can see, residential proxy is the more expensive component, and the cost varies based on type and volume of requests. For alternative, potentially more cost-effective proxy providers, consider exploring resources like the [Apify](https://discord.com/invite/jyEM2PRvMU) and [The Web Scraping Club](https://substack.thewebscraping.club/) Discord servers. For personalized advice, feel free to [start a discussion](#your-feedback). > **Tip for Cost Optimization**: To minimize costs, try to execute multiple requests in a single execution. This > approach helps to reduce resource wastage during the scraper's start-up phase. For a comprehensive understanding of Apify’s pricing, refer to their [usage documentation](https://docs.apify.com/platform/actors/running/usage-and-resources#usage). ### How to scrape Shopee? #### Getting started Get started with Shopee API Scraper by following these simple steps: 1. **Start**: Click on *Try for free*. 2. **Prepare Requests**: Formulate the API request URLs you wish to scrape. Refer to the [API reference section](#api-reference) for guidance. Example URLs include: - Search for popular products by keyword: `https://shopee.sg/api/v4/search/search_items?keyword=tshirt` - Retrieve top categories: `https://shopee.sg/api/v4/pages/get_category_tree` - List products by category: `https://shopee.sg/api/v4/search/search_items?match_id=11027757` - List all official shops: `https://shopee.sg/api/v4/official_shop/get_shops_by_category` - List products by shop: `https://shopee.sg/api/v4/shop/rcmd_items?shop_id=178877065` - Access product detail: `https://shopee.sg/api/v4/pdp/get_pc?shop_id=414243778&item_id=23600133176` - Access product ratings: `https://shopee.sg/api/v2/item/get_ratings?shopid=414243778&itemid=23600133176` 3. **Set Up Proxy**: Input the prepared URLs and configure the proxy to 'Residential' with your target country. 4. **Default Parameters**: Maintain other input parameters at their default settings for simplicity. 5. **Execution**: Click *Start* and wait for the results. The status will switch from *Running* to *Succeeded* upon completion. 6. **Results**: Access your data in the *Storage* tab, available in various formats like JSON, CSV, and more. Example input: ```json { // Search products with the "tshirt" keyword "requests": [ { "url": "https://shopee.sg/api/v4/search/search_items?keyword=tshirt" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, } ``` Example JSON output from a scraping session: ```json { "url": "https://shopee.sg/api/v4/search/search_items?keyword=tshirt", "enrichedUrl": "https://shopee.sg/api/v4/search/search_items?by=relevancy&extra_params=%7B%22global_search_session_id%22%3A%22gs-c44e90e2-6f01-4ed4-9fe3-fd38fe17f6e9%22%2C%22search_session_id%22%3A%22ss-59a709b9-72fe-4fdf-8a46-23edd0ff3255%22%7D&keyword=tshirt&limit=5&newest=0&order=desc&page_type=search&scenario=PAGE_GLOBAL_SEARCH&version=2&view_session_id=53f46a2f-80b1-45b4-afbb-17cca078a722", "referrer": "https://shopee.sg/search?keyword=tshirt", "knownApi": "PRODUCT_SEARCH", "responseStatus": 200, "responseBody": { // ... "total_count": 3000, "items": [ { "item_basic": { "itemid": 22541793907, "shopid": 391955424, "name": "Casual T-shirts Short-sleeved Tshirt Korean Round Neck Youth T-shirt Plain Color Tshirt Fashion Popular T-shirt", "image": "sg-11134201-7qvez-lgj50bviqvph8b", "currency": "SGD", "stock": 242, "ctime": 1683532055, "sold": 826, "historical_sold": 3044, "catid": 100011, "price": 850000, "price_before_discount": 1990000, "discount": "57%", "item_rating": { "rating_star": 4.702664796633941, "rating_count": [ 713, 11, 9, 33, 75, 585 ], "rcount_with_context": 254, "rcount_with_image": 164 }, // ... }, "itemid": 22541793907, "shopid": 391955424, // ... }, // ... ], // ... } } ``` This output includes fields like the original URL (`url`), the enriched URL sent to Shopee (`enrichedUrl`), and the HTTP response (`responseStatus` and `responseBody`), among others. > Tip: Enable 'Enrich URLs' and 'Generate referrers' for enhanced scraping. Check the > [input parameters](#input-parameters) for more details. #### Input parameters The Shopee API Scraper offers a range of input parameters, allowing you to tailor your scraping process according to your needs. These parameters can be broadly categorized into common parameters applicable to all APIs and specific parameters unique to certain APIs (detailed in the [API reference](#api-reference) section). Key common parameters include: 1. **API Requests**: This is where you specify the list of API URLs you intend to scrape. Additionally, you have the option to include a [Referer header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer) with each request. If 'Generate Referrers' is enabled, the scraper automatically generates a referrer for each request. However, if you manually set a referrer, it will override the automatically generated one, even if 'Generate Referrers' is active. An example configuration is as follows: ```json { "requests": [ { "url": "https://shopee.sg/api/v4/shop/rcmd_items?shop_id=178877065", "userData": { "referrer": "https://shopee.sg/xiaomiofficialstore.sg" } } ] } ``` 2. **Proxy Configuration**: Use a residential proxy set to the target country for effective scraping. 3. **Concurrency Settings** (Optional): Define the minimum and maximum number of parallel requests. Align this with the number of CPU cores allocated. See [Crawlee documentation](https://crawlee.dev/docs/guides/scaling-crawlers#minconcurrency-and-maxconcurrency) for more. 4. **Maximum Requests per Crawl** (Optional): Set the maximum number of requests the crawler will execute. The process stops once this limit is reached. 5. **Enrich URLs**: Enabled by default, this option automatically adds missing parameters to URLs, such as result limits and session IDs. 6. **Generate Referrers**: Also enabled by default, this generates the referrer HTTP header for each API request to enhance authenticity. For any queries or assistance, feel free to [reach out](#your-feedback). #### Output fields The data returned from the Shopee API Scraper includes several important fields. Below is a description of each: - **url**: This is the original URL you submitted for scraping. - **enrichedUrl**: Represents the actual URL sent to Shopee. If you enable `Enrich URLs`, this URL is automatically enhanced with default values. For more details, refer to the [input parameters](#input-parameters) section. - **referrer**: The [Referer HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer) sent to Shopee. It's generated automatically if `Generate referrers` is enabled. You have the option to set this manually as well (see [input parameters](#input-parameters) for more). - **knownApi**: Enumerated value corresponding to the requested API. This helps you identify which API endpoint was used: | API | KnownApi | |---------------------------------------------|----------------------------| | /api/v4/search/search\_items | PRODUCT\_SEARCH | | /api/v4/pdp/get\_pc | PRODUCT\_DETAIL | | /api/v2/item/get\_ratings | PRODUCT\_RATINGS | | /api/v4/pages/get\_category\_tree | FE\_CATEGORY\_TREE | | /api/v4/search/get\_fe\_category\_detail | FE\_CATEGORY\_DETAIL | | /api/v4/official\_shop/get\_shops\_by\_category | OFFICIAL\_SHOPS\_BY\_CATEGORY | | /api/v4/search/search\_user | SHOP\_SEARCH | | /api/v4/shop/get\_shop\_base | SHOP\_DETAIL | | /api/v4/shop/rcmd\_items | SHOP\_PRODUCTS | | /api/v4/search/search\_suggestion | KEYWORD\_SUGGESTIONS | | /api/v4/search/search\_hint | KEYWORD\_AUTOCOMPLETION | | -- Other API -- | -- not set -- | - **responseStatus**: Indicates the [HTTP response status](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) received from Shopee. - **responseBody**: Contains the HTTP response body from Shopee in JSON format. The specific content varies depending on the requested API. ### API reference #### Product Search by Keyword or Category - **URL Path**: - `/api/v4/search/search_items` - **URL Parameters**: - `keyword`: Search keyword, e.g., "tshirt". - `match_id`: Category ID from `/api/v4/pages/get_category_tree`. > Note: `keyword` and `match_id` are mutually exclusive. - `by` (default = "relevancy"): Sort by "relevancy", "ctime", "sales", or "price". - `order` (default = "desc"): Sort order, "asc" or "desc". - `limit` (default = set by scraper): Number of results per page. - `newest` (default = 0): Offset for results, typically a multiple of `limit`. - `page_type`, `scenario`, `version`: Additional parameters with default settings. - `extra_params`, `view_session_id`: Automatically generated, unless overridden. - **Scraper Input Parameters** - `productSearch_enrichUrlQuery_pageSize`: Number of results per request (default: 60). - `productSearch_crawlNextPages`: Set to true to crawl subsequent pages. - `productSearch_crawlNextPages_maxPages`: Limit on number of pages to crawl (default: 50). - `productSearch_crawlNextPages_maxResults`: Maximum products to crawl (default: 3000). - `productSearch_crawlProductDetails`: Set to true to crawl details of each product. - **Example Input**: ```json { // Search products with the "tshirt" keyword "requests": [ { "url": "https://shopee.sg/api/v4/search/search_items?keyword=tshirt" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Set the "limit" URL parameter to 60 "productSearch_enrichUrlQuery_pageSize": 60, // Automatically crawl the next pages "productSearch_crawlNextPages": true, // Crawl until the 3rd page "productSearch_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of products to crawl "productSearch_crawlNextPages_maxResults": 180, // Automatically crawl the detail of each listed product "productSearch_crawlProductDetails": true } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-product-search-by-keyword.json) - **Notable Response Fields**: - `items`: Array of product summaries. - `items/itemid` and `items/shopid`: Essential for scraping product details and ratings. - `items/item_basic/name`: Product name. - `items/item_basic/image`: Image key. Convert to URL with `https://down-${country}.img.susercontent.com/file/${imageKey}`, for example the URL to download the image "sg-11134201-7qvez-lgj50bviqvph8b" is "https://down-sg.img.susercontent.com/file/sg-11134201-7qvez-lgj50bviqvph8b". - `items/item_basic/ctime`: Creation timestamp. - `items/item_basic/sold`: Monthly sales. - `items/item_basic/historical_sold`: Total sales. - `items/item_basic/price`: Divide by 100,000 to get actual price. #### Product Details - **URL Path**: - `/api/v4/pdp/get_pc` - **URL Parameters**: - `shop_id`: The unique identifier of the shop. - `item_id`: The unique identifier of the product. - **Scraper Input Parameters** - `productDetail_crawlProductRatings`: Define the types of product ratings to crawl. Options include "ALL", "ONE\_STAR", "TWO\_STARS" to "FIVE\_STARS", "WITH\_COMMENTS", "WITH\_MEDIA", "LOCAL\_REVIEWS". Combine multiple values for a broader range, e.g., `["ONE_STAR", "TWO_STARS"]`. - **Example Input**: ```json { // Get the details of a product "requests": [ { "url": "https://shopee.sg/api/v4/pdp/get_pc?shop_id=414243778&item_id=23600133176" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Automatically crawl ratings of any stars "productDetail_crawlProductRatings": ["ALL"], // Set the rating "limit" URL parameter to 50 "productRatings_enrichUrlQuery_pageSize": 50, // Automatically crawl the next rating pages "productRatings_crawlNextPages": true, // Crawl until the 3rd rating page "productRatings_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of ratings to crawl "productRatings_crawlNextPages_maxResults": 150, } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-product-detail.json) - **Notable Response Fields**: - `data/item/item_id` and `data/item/shop_id`: Essential identifiers for the product and shop. - `data/item/title`: Product name. - `data/item/description`: Product description. - `data/item/image`: Image key. Convert to URL using the provided format: `https://down-${country}.img.susercontent.com/file/${imageKey}`. - `data/item/ctime`: Product creation timestamp. - `data/item/fe_categories`: Category information including parent categories. - `data/item/models`: Detailed SKU information. - `data/product_review`: Aggregated rating statistics. - `data/product_shipping`: Shipping details. - `data/shop_detailed`: Comprehensive shop details including name, username (slug), and image. #### Product Ratings - **URL Path**: - `/api/v2/item/get_ratings` - **URL Parameters**: - `shopid` & `itemid`: Unique identifiers for the shop and product. - `limit` (default = set by scraper): Number of results per page. - `offset` (default = 0): Offset for results, typically a multiple of `limit`. - `type` (default = 0, max = 5): Filter by number of stars (e.g. 2 = 2-stars ratings only). 0 = all ratings. - `filter` (default = 0): Filter by content: 1 = with comments, 3 = with media, 9 = local ratings, 0 = all ratings. - `exclude_filter`, `filter_size`, `flag`, `fold_filter`, `relevant_reviews`, `request_source`, `tag_filter`, `variation_filters`: Additional parameters with default settings. - **Scraper Input Parameters** - `productRatings_enrichUrlQuery_pageSize`: Number of results per request (default: 6). - `productRatings_crawlNextPages`: Set to true to crawl subsequent pages. - `productRatings_crawlNextPages_maxPages`: Limit on number of pages to crawl (default: 500). - `productRatings_crawlNextPages_maxResults`: Maximum products to crawl (default: 3000). - `productRatings_autofixError10002`: Automatically corrects error 10002 by splitting requests (default: true). - **Example Input**: ```json { // Get the ratings of a product "requests": [ { "url": "https://shopee.sg/api/v2/item/get_ratings?shopid=414243778&itemid=23600133176" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Set the rating "limit" URL parameter to 50 "productRatings_enrichUrlQuery_pageSize": 50, // Automatically crawl the next rating pages "productRatings_crawlNextPages": true, // Crawl until the 3rd rating page "productRatings_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of ratings to crawl "productRatings_crawlNextPages_maxResults": 150, // Automatically fix the error 10002 "productRatings_autofixError10002": true, } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-product-ratings.json) - **Notable Response Fields**: - `data/ratings/cmtid`: Unique identifier for each rating. - `data/ratings/ctime`: Timestamp of the rating. - `data/ratings/rating_star`: Star rating from 1 to 5. - `data/ratings/comment`: Buyer's comment. - `data/ratings/author_username`: Buyer's username (sometimes anonymized). - `data/ratings/author_portrait`: Buyer's profile image. - `data/ratings/images` & `videos`: Media associated with the rating. - `data/ratings/product_items`: SKU details of the rated product. > Data Privacy Note: Always be mindful of buyers' personal information and comply with local data protection laws. > Avoid storing personal data unless necessary. #### Shopee Category Tree - **URL Path**: - `/api/v4/pages/get_category_tree` - **Example Input**: ```json { // Get the category tree "requests": [ { "url": "https://shopee.sg/api/v4/pages/get_category_tree" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" } } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-category-tree.json) - **Notable Response Fields**: - `data/category_list/catid`: Unique identifier for each category. - `data/category_list/name`: Category name. - `data/category_list/children`: Sub-categories. > Note: this API only provides the two top category levels. #### Category Details - **URL Path**: - `/api/v4/search/get_fe_category_detail` - **URL Parameters**: - `catids`: Unique identifiers for the category. - **Example Input**: ```json { // Retrieve details for a specific category "requests": [ { "url": "https://shopee.sg/api/v4/search/get_fe_category_detail?catids=11027710" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" } } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-category-detail.json) - **Notable Response Fields**: - `data/categories/catid`: The unique identifier for the category. - `data/categories/parent_cat_id`: Identifier for the parent category, helping to understand category hierarchy. - `data/categories/display_name`: The category name as displayed in different languages across the platform. #### Official Shops - **URL Path**: - `/api/v4/official_shop/get_shops_by_category` - **URL Parameters**: - `category_id` (default = -1): Specify a category ID to retrieve official shops within that category. Use -1 to list all official shops. - `need_zhuyin`: An additional parameter, typically set to default. > Note: Only shops associated with registered brands are considered "official" on Shopee. - **Example Input**: ```json { // Retrieve a list of all official shops "requests": [ { "url": "https://shopee.sg/api/v4/official_shop/get_shops_by_category" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" } } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-official-shops-by-category.json) - **Notable Response Fields**: - `data/brands/brand_ids/shopid`: The unique identifier for each official shop. - `data/brands/brand_ids/brand_name`: The name of the brand associated with the shop. - `data/brands/brand_ids/username`: The shop's username or slug, useful for direct queries or referencing. #### Shop Search By Keyword - **URL Path**: - `/api/v4/search/search_user` - **URL Parameters**: - `keyword` (minimum: 3 characters): Enter a search term, such as "adi", to find relevant shops. - `limit` (default = set by scraper): Number of results per page. - `offset` (default = 0): Offset for results, typically a multiple of `limit`. - `page`, `with_search_cover`: Additional parameters with default settings. > Note 1: This API covers all shops on Shopee, both official and unofficial. > Note 2: For a comprehensive shop list, consider searching with various [trigrams](https://en.wikipedia.org/wiki/Trigram). - **Scraper Input Parameters** - `shopSearch_enrichUrlQuery_pageSize`: Number of results per request (default: 6). - `shopSearch_crawlNextPages`: Set to true to crawl subsequent pages. - `shopSearch_crawlNextPages_maxPages`: Limit on number of pages to crawl (default: 500). - `shopSearch_crawlNextPages_maxResults`: Maximum shops to crawl (default: 30000). - `shopSearch_crawlShopDetails`: Set to true to crawl details of each shop. - **Example Input**: ```json { // Search shop with the "adi" keyword "requests": [ { "url": "https://shopee.sg/api/v4/search/search_user?keyword=adi" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Set the "limit" URL parameter to 50 "shopSearch_enrichUrlQuery_pageSize": 50, // Automatically crawl the next pages "shopSearch_crawlNextPages": true, // Crawl until the 3rd page "shopSearch_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of shops to crawl "shopSearch_crawlNextPages_maxResults": 150, // Automatically crawl the detail of each listed shop "shopSearch_crawlShopDetails": true } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-shop-search.json) - **Notable Response Fields**: - `data/users/shopid`: Unique identifier for the shop. - `data/users/shopname`: Name of the shop. - `data/users/username`: The shop's username or slug, useful for direct queries or referencing. - `data/users/portrait`: Shop image key. Convert to URL with `https://down-${country}.img.susercontent.com/file/${imageKey}`, for example the URL to download the image "6d4ff02a5978ee10a7b4527c25396b0b" is "https://down-sg.img.susercontent.com/file/6d4ff02a5978ee10a7b4527c25396b0b". #### Shop Detail - **URL Path**: - `/api/v4/shop/get_shop_base` - **URL Parameters**: - `username`: Specify the shop's username (not the shop ID). - `entry_point`, `need_cancel_rate`, `request_source`, `version`: Additional parameters with default settings. - **Scraper Input Parameters** - `shopDetail_crawlShopProducts`: Set to true to enable automatic crawling of the shop's products. - **Example Input**: ```json { // Retrieve detailed information about a specific shop "requests": [ { "url": "https://shopee.sg/api/v4/shop/get_shop_base?username=xiaomiofficialstore.sg" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Automatically crawl shop products "shopSearch_crawlShopDetails": true, // Set the product "limit" URL parameter to 30 "shopProducts_enrichUrlQuery_pageSize": 30, // Automatically crawl the next product pages "shopProducts_crawlNextPages": true, // Crawl until the 3rd product page "shopProducts_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of products to crawl "shopProducts_crawlNextPages_maxResults": 90, } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-shop-detail.json) - **Notable Response Fields**: - `data/shopid`: The unique identifier for the shop. - `data/name`: Official name of the shop. - `data/description`: Brief description of the shop. - `data/cover`: Image key for the shop's profile picture. Convert to URL using the provided format: `https://down-${country}.img.susercontent.com/file/${imageKey}`. - `data/account/username`: The shop's unique username or slug. - `data/ctime`: Timestamp marking the shop's creation. - `data/shop_rating`: Aggregated rating details of the shop. - `data/seller_metrics`: Various metrics providing insights into the shop's performance. - `data/item_count`: Total number of products offered by the shop. #### Shop Products - **URL Path**: - `api/v4/shop/rcmd_items` - **URL Parameters**: - `shop_id`: Unique identifier for the shop. - `limit` (default = set by scraper): Number of results per page. - `offset` (default = 0): Offset for results, typically a multiple of `limit`. - `sort_type` (default = 1): Sorting option for products. 1 = popular first, 2 = latest first, 13 = top sales first, 8 = price low to high, 4 = price high to low - `cat_id` (optional): Filter products by a category. - `bundle`, `upstream`: Additional parameters with default settings. - **Scraper Input Parameters** - `shopProducts_enrichUrlQuery_pageSize`: Number of results per request (default: 30). - `shopProducts_crawlNextPages`: Set to true to crawl subsequent pages. - `shopProducts_crawlNextPages_maxPages`: Limit on number of pages to crawl (default: 100). - `shopProducts_crawlNextPages_maxResults`: Maximum products to crawl (default: 3000). - `shopProducts_crawlNextPages_minSales`: Minimum sales threshold for page crawling (optional). Crawling stops when a product with sales equal to or less than this value appears, assuming products are sorted by sales in descending order. - `shopProducts_crawlProductDetails`: Set to true to automatically crawl product details (default: false). - **Example Input**: ```json { // Get the products of a shop "requests": [ { "url": "https://shopee.sg/api/v4/shop/rcmd_items?shop_id=178877065" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Set the product "limit" URL parameter to 30 "shopProducts_enrichUrlQuery_pageSize": 30, // Automatically crawl the next product pages "shopProducts_crawlNextPages": true, // Crawl until the 3rd product page "shopProducts_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of products to crawl "shopProducts_crawlNextPages_maxResults": 150, // Automatically crawl product details "shopProducts_crawlProductDetails": true } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/d4b2ced90b29f6e8ad76e70d26becb8a37f6e133/example-shop-products.json) > Note: this API is scraped by using the [POST method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST). - **Notable Response Fields**: - `data/items`: Array of product summaries. - `data/items/itemid` and `items/shopid`: Essential for scraping product details and ratings. - `data/items/name`: Product name. - `data/items/image`: Image key. Convert to URL with `https://down-${country}.img.susercontent.com/file/${imageKey}`. - `data/items/ctime`: Creation timestamp. - `data/items/stock`: Stock level. - `data/items/sold`: Monthly sales. - `data/items/historical_sold`: Total sales. - `data/items/price`: Divide by 100,000 to get actual price. #### Keyword Suggestion - **URL Path**: - `/api/v4/search/search_suggestion` > Note: The keyword suggestions are normally displayed under the top search bar. - **URL Parameters**: - `limit` (default = set by scraper): Number of results per page. - `offset` (default = 0): Offset for results, typically a multiple of `limit`. - `bundle`: An additional parameter, usually set to default. - **Scraper Input Parameters** - `keywordSuggestions_enrichUrlQuery_pageSize`: Number of keyword suggestions per request (default: 8). - `keywordSuggestions_crawlNextPages`: Set to true to crawl subsequent pages. - `keywordSuggestions_crawlNextPages_maxPages`: Limit on number of pages to crawl (default: 55). - `keywordSuggestions_crawlNextPages_maxResults`: Maximum keyword suggestions to crawl (default: 440). - **Example Input**: ```json { // Retrieve the 50 top suggested keywords "requests": [ { "url": "https://shopee.sg/api/v4/search/search_suggestion?limit=50" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" }, // Set the "limit" URL parameter to 50 "keywordSuggestions_enrichUrlQuery_pageSize": 50, // Automatically crawl the next pages "keywordSuggestions_crawlNextPages": true, // Crawl until the 3rd page "keywordSuggestions_crawlNextPages_maxPages": 3, // Alternatively, indicate the maximum number of keyword suggestions to crawl "keywordSuggestions_crawlNextPages_maxResults": 150 } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-keyword-suggestion.json) - **Notable Response Fields**: - `data/queries/text`: Suggested keyword. #### Keyword Autocompletion - **URL Path**: - `/api/v4/search/search_hint` - **URL Parameters**: - `keyword`: The initial characters of a potential search term. - `extra_params`, `search_type`, `version`: Additional parameters with default settings. - **Example Input**: ```json { // Retrieve autocomplete suggestions for the keyword prefix "n" "requests": [ { "url": "https://shopee.sg/api/v4/search/search_hint?keyword=n" } ], // Use a residential proxy matching the country of the API endpoint (e.g. "SG" for "shopee.sg", "MY" for "shopee.com.my", ...etc.) "proxy": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "SG" } } ``` - **Example Response**: - [View Full Example Response](https://gist.githubusercontent.com/marcplouhinec/6cc2c3e648fed6893ef71d01778e6377/raw/0b5ba3d6950c1cc50458b4d24cb3458466180eed/example-keyword-autocompletion.json) - **Notable Response Fields**: - `keywords/keyword`: Suggested keyword based on the input characters. ### Is it legal to scrape Shopee? It is legal to scrape publicly available data such as product descriptions, prices, or ratings. Read Apify's blog post on [the legality of web scraping](https://blog.apify.com/is-web-scraping-legal/) to learn more. ### Need to find product pairs between Shopee and another online shop? Use the [AI Product Matcher](https://apify.com/equidem/ai-product-matcher). This AI model allows you to compare items from different web stores, identifying exact matches and comparing real-time data obtained via web scraping. With the AI Product Matcher, you can use scraped product data to monitor product matches across the industry, implement dynamic pricing for your website, replace or complement manual mapping, and obtain realistic estimates against your competition for upcoming promo campaigns. Most importantly, it is relatively easy to get started with (just follow [this guide](https://blog.apify.com/product-matching-ai-pricing-intelligence-web-scraping/)), and it can check thousands of pairs of products. ### Integrations and Shopee API Scraper Shopee API Scraper can be connected with almost any cloud service or web app thanks to [integrations on the Apify platform](https://apify.com/integrations). You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, [Google Drive](https://docs.apify.com/integrations), and more. Or you can use [webhooks](https://docs.apify.com/integrations/webhooks) to carry out an action whenever an event occurs, e.g. get a notification whenever Shopee API Scraper successfully finishes a run. ### Your feedback Please don't hesitate to share your feedback (improvement ideas, bug, ...etc.). You can create an issue on the Actor’s issues tab in Apify Console. In addition, feel free to chat with me [on Discord (username "marcplouhinec")](https://discord.com/users/marcplouhinec). ### Thanks Shopee API Scraper is built with [Crawlee](https://crawlee.dev/), a great JavaScript framework to accelerate scraper development. It also uses [ungoogled-chromium](https://github.com/ungoogled-software/ungoogled-chromium) via the [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). These powerful technologies have been instrumental in "opening" the website despite anti-bot protections. This README file was inspired from other ecommerce scrapers such as the [Amazon Product Scraper](https://apify.com/junglee/amazon-crawler) and [Temu Scraper](https://apify.com/mscraper/temu-scraper). Finally, a lot of knowledge that enabled the development of Shopee API Scraper comes from [The Web Scraping Club](https://substack.thewebscraping.club/) and its Discord Community. # Actor input Schema ## `requests` (type: `array`): One or more URLs of the Shopee APIs to invoke. Look at the README document for detailed explanations and examples. ## `proxy` (type: `object`): Select proxies to be used to scrape data. Use RESIDENTIAL proxies in the same country as your target. ## `enrichUrlQuery` (type: `boolean`): Automatically generate missing URL parameters. For example add result limits and session IDs. ## `generateReferrer` (type: `boolean`): Automatically generate the referrer HTTP header before sending API requests, to make them look more "authentic". ## `verboseLogs` (type: `boolean`): Enable this option to activate detailed logging within the scraper's operation. This feature is particularly useful for troubleshooting errors. ## `minConcurrency` (type: `integer`): Minimum concurrency (parallelism) for the crawl. ## `maxConcurrency` (type: `integer`): Maximum concurrency (parallelism) for the crawl. ## `maxRequestsPerCrawl` (type: `integer`): Maximum number of requests that the crawler will execute. The crawl will stop when this limit is reached. ## `maxConsecutiveBlockagesInTotal` (type: `integer`): Stop scraping when too many requests get blocked consecutively. This can happen with poor quality proxies or if this scraper is not compatible anymore with an upgraded anti-bot technology. ## `securitySdkLoadTimeoutMs` (type: `integer`): Sets the maximum time, in milliseconds, to wait for the Security SDK report from Shopee's anti-bot measures during website load. ## `maxPageReloadsPerProxy` (type: `integer`): Defines the limit for reloading a page using the same proxy when faced with blockages. Exceeding this limit triggers the use of a new proxy for the next attempt. ## `websiteLoadFailureScreenshot` (type: `boolean`): Enables capturing and saving a screenshot in the key-value store if the website fails to load, aiding in troubleshooting issues. ## `productSearch_enrichUrlQuery_pageSize` (type: `integer`): Maximum number of results per request to crawl when searching products. ## `productSearch_crawlNextPages` (type: `boolean`): Automatically crawl next pages when scraping search results. ## `productSearch_crawlNextPages_maxPages` (type: `integer`): Maximum number of pages to crawl when searching products. ## `productSearch_crawlNextPages_maxResults` (type: `integer`): Maximum number of results to crawl when searching products. ## `productSearch_crawlNextPages_minSales` (type: `integer`): Minimum sales threshold to halt automatic page navigation in shop product listings. Crawling stops when a product with sales equal to or less than this value appears, assuming products are sorted by sales in descending order. Useful for scraping all selling products of a shop or a category and avoiding scraping products with no sales. ## `productSearch_crawlProductDetails` (type: `boolean`): Automatically crawl product details from the search results. ## `productDetail_crawlProductRatings` (type: `array`): Automatically crawl product ratings after querying a product detail. Also indicate what filter to apply. If multiple filters are provided, requests for each filter are executed. ## `productRatings_enrichUrlQuery_pageSize` (type: `integer`): Maximum number of results per request when crawling product ratings. ## `productRatings_crawlNextPages` (type: `boolean`): Automatically crawl next pages when scraping product ratings. ## `productRatings_crawlNextPages_maxPages` (type: `integer`): Maximum number of pages to crawl when visiting product ratings. ## `productRatings_crawlNextPages_maxResults` (type: `integer`): Maximum number of results to crawl when visiting product ratings. ## `productRatings_autofixError10002` (type: `boolean`): Sometimes the product ratings API returns the error 10002. Enabling this option allows this scraper to fix this issue by splitting problematic requests into multiple smaller ones. ## `shopSearch_enrichUrlQuery_pageSize` (type: `integer`): Maximum number of results per request to crawl when searching shops. ## `shopSearch_crawlNextPages` (type: `boolean`): Automatically crawl next pages when scraping search results. ## `shopSearch_crawlNextPages_maxPages` (type: `integer`): Maximum number of pages to crawl when searching shops. ## `shopSearch_crawlNextPages_maxResults` (type: `integer`): Maximum number of results to crawl when searching shops. ## `shopSearch_crawlShopDetails` (type: `boolean`): Automatically crawl shop details from the search results. ## `shopDetail_crawlShopProducts` (type: `boolean`): Automatically crawl products after querying a shop detail. ## `shopProducts_enrichUrlQuery_pageSize` (type: `integer`): Maximum number of results per request when crawling shop products. ## `shopProducts_crawlNextPages` (type: `boolean`): Automatically crawl next pages when scraping shop products. ## `shopProducts_crawlNextPages_maxPages` (type: `integer`): Maximum number of pages to crawl when visiting shop products. ## `shopProducts_crawlNextPages_maxResults` (type: `integer`): Maximum number of results to crawl when visiting shop products. ## `shopProducts_crawlNextPages_minSales` (type: `integer`): Minimum sales threshold to halt automatic page navigation in shop product listings. Crawling stops when a product with sales equal to or less than this value appears, assuming products are sorted by sales in descending order. Useful for scraping all selling products of a shop and avoiding scraping products with no sales. ## `shopProducts_crawlProductDetails` (type: `boolean`): Automatically crawl product details from the shop products listing. ## `keywordSuggestions_enrichUrlQuery_pageSize` (type: `integer`): Maximum number of results per request when crawling keyword suggestions. ## `keywordSuggestions_crawlNextPages` (type: `boolean`): Automatically crawl next pages when scraping keyword suggestions. ## `keywordSuggestions_crawlNextPages_maxPages` (type: `integer`): Maximum number of pages to crawl when visiting keyword suggestions. ## `keywordSuggestions_crawlNextPages_maxResults` (type: `integer`): Maximum number of results to crawl when visiting keyword suggestions. ## Actor input object example ```json { "requests": [ { "url": "https://shopee.com.my/api/v4/search/search_items?keyword=hawaii%20shirt%20men" } ], "proxy": { "useApifyProxy": true, "apifyProxyCountry": "MY", "apifyProxyGroups": [ "RESIDENTIAL" ] }, "enrichUrlQuery": true, "generateReferrer": true, "verboseLogs": true, "minConcurrency": 1, "maxConcurrency": 1, "maxRequestsPerCrawl": 999999999, "maxConsecutiveBlockagesInTotal": 200, "securitySdkLoadTimeoutMs": 20000, "maxPageReloadsPerProxy": 2, "websiteLoadFailureScreenshot": false, "productSearch_enrichUrlQuery_pageSize": 60, "productSearch_crawlNextPages": false, "productSearch_crawlNextPages_maxPages": 50, "productSearch_crawlNextPages_maxResults": 3000, "productSearch_crawlProductDetails": false, "productRatings_enrichUrlQuery_pageSize": 6, "productRatings_crawlNextPages": false, "productRatings_crawlNextPages_maxPages": 500, "productRatings_crawlNextPages_maxResults": 3000, "productRatings_autofixError10002": true, "shopSearch_enrichUrlQuery_pageSize": 6, "shopSearch_crawlNextPages": false, "shopSearch_crawlNextPages_maxPages": 500, "shopSearch_crawlNextPages_maxResults": 3000, "shopSearch_crawlShopDetails": false, "shopDetail_crawlShopProducts": false, "shopProducts_enrichUrlQuery_pageSize": 30, "shopProducts_crawlNextPages": false, "shopProducts_crawlNextPages_maxPages": 100, "shopProducts_crawlNextPages_maxResults": 3000, "shopProducts_crawlProductDetails": false, "keywordSuggestions_enrichUrlQuery_pageSize": 8, "keywordSuggestions_crawlNextPages": false, "keywordSuggestions_crawlNextPages_maxPages": 55, "keywordSuggestions_crawlNextPages_maxResults": 440 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "requests": [ { "url": "https://shopee.com.my/api/v4/search/search_items?keyword=hawaii%20shirt%20men" } ], "proxy": { "useApifyProxy": true, "apifyProxyCountry": "MY", "apifyProxyGroups": [ "RESIDENTIAL" ] } }; // Run the Actor and wait for it to finish const run = await client.actor("chartedsea/shopee-api-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "requests": [{ "url": "https://shopee.com.my/api/v4/search/search_items?keyword=hawaii%20shirt%20men" }], "proxy": { "useApifyProxy": True, "apifyProxyCountry": "MY", "apifyProxyGroups": ["RESIDENTIAL"], }, } # Run the Actor and wait for it to finish run = client.actor("chartedsea/shopee-api-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "requests": [ { "url": "https://shopee.com.my/api/v4/search/search_items?keyword=hawaii%20shirt%20men" } ], "proxy": { "useApifyProxy": true, "apifyProxyCountry": "MY", "apifyProxyGroups": [ "RESIDENTIAL" ] } }' | apify call chartedsea/shopee-api-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=chartedsea/shopee-api-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Shopee Api Scraper", "description": "Query Shopee's unofficial API for product searches by keyword, category, or shop. Access detailed information including prices, orders, stock levels, and ratings. Also retrieve related entities like the category tree, shop listings, and keyword suggestions.", "version": "1.7", "x-build-id": "YY0JrwCtjL44ycTXI" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/chartedsea~shopee-api-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-chartedsea-shopee-api-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/chartedsea~shopee-api-scraper/runs": { "post": { "operationId": "runs-sync-chartedsea-shopee-api-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/chartedsea~shopee-api-scraper/run-sync": { "post": { "operationId": "run-sync-chartedsea-shopee-api-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "requests", "proxy" ], "properties": { "requests": { "title": "API requests", "type": "array", "description": "One or more URLs of the Shopee APIs to invoke. Look at the README document for detailed explanations and examples.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "proxy": { "title": "Proxy configuration", "type": "object", "description": "Select proxies to be used to scrape data. Use RESIDENTIAL proxies in the same country as your target." }, "enrichUrlQuery": { "title": "Enrich URLs", "type": "boolean", "description": "Automatically generate missing URL parameters. For example add result limits and session IDs.", "default": true }, "generateReferrer": { "title": "Generate referrers", "type": "boolean", "description": "Automatically generate the referrer HTTP header before sending API requests, to make them look more \"authentic\".", "default": true }, "verboseLogs": { "title": "Verbose logs", "type": "boolean", "description": "Enable this option to activate detailed logging within the scraper's operation. This feature is particularly useful for troubleshooting errors.", "default": true }, "minConcurrency": { "title": "Minimum concurrency", "minimum": 1, "type": "integer", "description": "Minimum concurrency (parallelism) for the crawl.", "default": 1 }, "maxConcurrency": { "title": "Maximum concurrency", "minimum": 1, "type": "integer", "description": "Maximum concurrency (parallelism) for the crawl.", "default": 1 }, "maxRequestsPerCrawl": { "title": "Maximum requests per crawl", "minimum": 1, "type": "integer", "description": "Maximum number of requests that the crawler will execute. The crawl will stop when this limit is reached.", "default": 999999999 }, "maxConsecutiveBlockagesInTotal": { "title": "Abort after too many consecutive blockages", "minimum": 1, "type": "integer", "description": "Stop scraping when too many requests get blocked consecutively. This can happen with poor quality proxies or if this scraper is not compatible anymore with an upgraded anti-bot technology.", "default": 200 }, "securitySdkLoadTimeoutMs": { "title": "Security SDK Load Timeout", "minimum": 1000, "maximum": 60000, "type": "integer", "description": "Sets the maximum time, in milliseconds, to wait for the Security SDK report from Shopee's anti-bot measures during website load.", "default": 20000 }, "maxPageReloadsPerProxy": { "title": "Max Page Reloads Per Proxy", "minimum": 1, "maximum": 30, "type": "integer", "description": "Defines the limit for reloading a page using the same proxy when faced with blockages. Exceeding this limit triggers the use of a new proxy for the next attempt.", "default": 2 }, "websiteLoadFailureScreenshot": { "title": "Website load failure screenshot", "type": "boolean", "description": "Enables capturing and saving a screenshot in the key-value store if the website fails to load, aiding in troubleshooting issues.", "default": false }, "productSearch_enrichUrlQuery_pageSize": { "title": "Page size (product search)", "type": "integer", "description": "Maximum number of results per request to crawl when searching products.", "default": 60 }, "productSearch_crawlNextPages": { "title": "Crawl next pages (product search)", "type": "boolean", "description": "Automatically crawl next pages when scraping search results.", "default": false }, "productSearch_crawlNextPages_maxPages": { "title": "Maximum number of pages (product search)", "minimum": 1, "type": "integer", "description": "Maximum number of pages to crawl when searching products.", "default": 50 }, "productSearch_crawlNextPages_maxResults": { "title": "Maximum number of results (product search)", "type": "integer", "description": "Maximum number of results to crawl when searching products.", "default": 3000 }, "productSearch_crawlNextPages_minSales": { "title": "Minimum sales threshold for page crawling (product search)", "minimum": 0, "type": "integer", "description": "Minimum sales threshold to halt automatic page navigation in shop product listings. Crawling stops when a product with sales equal to or less than this value appears, assuming products are sorted by sales in descending order. Useful for scraping all selling products of a shop or a category and avoiding scraping products with no sales." }, "productSearch_crawlProductDetails": { "title": "Crawl product details (product search)", "type": "boolean", "description": "Automatically crawl product details from the search results.", "default": false }, "productDetail_crawlProductRatings": { "title": "Crawl product ratings (product detail)", "type": "array", "description": "Automatically crawl product ratings after querying a product detail. Also indicate what filter to apply. If multiple filters are provided, requests for each filter are executed.", "items": { "type": "string", "enum": [ "ALL", "ONE_STAR", "TWO_STARS", "THREE_STARS", "FOUR_STARS", "FIVE_STARS", "WITH_COMMENTS", "WITH_MEDIA", "LOCAL_REVIEWS" ], "enumTitles": [ "All ratings", "1 star ratings", "2 stars ratings", "3 stars ratings", "4 stars ratings", "5 stars ratings", "Ratings with comments", "Ratings with media", "Local ratings" ] } }, "productRatings_enrichUrlQuery_pageSize": { "title": "Page size (product ratings)", "maximum": 59, "type": "integer", "description": "Maximum number of results per request when crawling product ratings.", "default": 6 }, "productRatings_crawlNextPages": { "title": "Crawl next pages (product ratings)", "type": "boolean", "description": "Automatically crawl next pages when scraping product ratings.", "default": false }, "productRatings_crawlNextPages_maxPages": { "title": "Maximum number of pages (product ratings)", "minimum": 1, "type": "integer", "description": "Maximum number of pages to crawl when visiting product ratings.", "default": 500 }, "productRatings_crawlNextPages_maxResults": { "title": "Maximum number of results (product ratings)", "type": "integer", "description": "Maximum number of results to crawl when visiting product ratings.", "default": 3000 }, "productRatings_autofixError10002": { "title": "Automatically fix the error 10002 (product ratings)", "type": "boolean", "description": "Sometimes the product ratings API returns the error 10002. Enabling this option allows this scraper to fix this issue by splitting problematic requests into multiple smaller ones.", "default": true }, "shopSearch_enrichUrlQuery_pageSize": { "title": "Page size (shop search)", "type": "integer", "description": "Maximum number of results per request to crawl when searching shops.", "default": 6 }, "shopSearch_crawlNextPages": { "title": "Crawl next pages (shop search)", "type": "boolean", "description": "Automatically crawl next pages when scraping search results.", "default": false }, "shopSearch_crawlNextPages_maxPages": { "title": "Maximum number of pages (shop search)", "minimum": 1, "type": "integer", "description": "Maximum number of pages to crawl when searching shops.", "default": 500 }, "shopSearch_crawlNextPages_maxResults": { "title": "Maximum number of results (shop search)", "type": "integer", "description": "Maximum number of results to crawl when searching shops.", "default": 3000 }, "shopSearch_crawlShopDetails": { "title": "Crawl shop details (shop search)", "type": "boolean", "description": "Automatically crawl shop details from the search results.", "default": false }, "shopDetail_crawlShopProducts": { "title": "Crawl shop products (shop detail)", "type": "boolean", "description": "Automatically crawl products after querying a shop detail.", "default": false }, "shopProducts_enrichUrlQuery_pageSize": { "title": "Page size (shop products)", "type": "integer", "description": "Maximum number of results per request when crawling shop products.", "default": 30 }, "shopProducts_crawlNextPages": { "title": "Crawl next pages (shop products)", "type": "boolean", "description": "Automatically crawl next pages when scraping shop products.", "default": false }, "shopProducts_crawlNextPages_maxPages": { "title": "Maximum number of pages (shop products)", "minimum": 1, "type": "integer", "description": "Maximum number of pages to crawl when visiting shop products.", "default": 100 }, "shopProducts_crawlNextPages_maxResults": { "title": "Maximum number of results (shop products)", "type": "integer", "description": "Maximum number of results to crawl when visiting shop products.", "default": 3000 }, "shopProducts_crawlNextPages_minSales": { "title": "Minimum sales threshold for page crawling (shop products)", "minimum": 0, "type": "integer", "description": "Minimum sales threshold to halt automatic page navigation in shop product listings. Crawling stops when a product with sales equal to or less than this value appears, assuming products are sorted by sales in descending order. Useful for scraping all selling products of a shop and avoiding scraping products with no sales." }, "shopProducts_crawlProductDetails": { "title": "Crawl product details (shop products)", "type": "boolean", "description": "Automatically crawl product details from the shop products listing.", "default": false }, "keywordSuggestions_enrichUrlQuery_pageSize": { "title": "Page size (keyword suggestions)", "type": "integer", "description": "Maximum number of results per request when crawling keyword suggestions.", "default": 8 }, "keywordSuggestions_crawlNextPages": { "title": "Crawl next pages (keyword suggestions)", "type": "boolean", "description": "Automatically crawl next pages when scraping keyword suggestions.", "default": false }, "keywordSuggestions_crawlNextPages_maxPages": { "title": "Maximum number of pages (keyword suggestions)", "minimum": 1, "type": "integer", "description": "Maximum number of pages to crawl when visiting keyword suggestions.", "default": 55 }, "keywordSuggestions_crawlNextPages_maxResults": { "title": "Maximum number of results (keyword suggestions)", "maximum": 440, "type": "integer", "description": "Maximum number of results to crawl when visiting keyword suggestions.", "default": 440 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```