# Tokopedia Product Scraper - Deduplication Ability (`confidential_gnat/tokopedia-products-scraper`) Actor

Scrape Tokopedia product data - title, shop, price in IDR, stock, category and images - from any /find/ keyword listing or product URL. Export to JSON, CSV or Excel, and never re-scrape the same product twice across runs via built-in cross-run deduplication.

- **URL**: https://apify.com/confidential\_gnat/tokopedia-products-scraper.md
- **Developed by:** [ActorFlow](https://apify.com/confidential_gnat) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Tokopedia Products Scraper

**Scrape product data from Tokopedia (tokopedia.com)**, Indonesia's largest online marketplace, without writing a line of code. This **Tokopedia scraper** extracts product title, shop, price in rupiah, stock level, category breadcrumbs, description and images from keyword listing pages or individual product URLs — just paste a URL and press Start. Results export to **JSON, CSV or Excel**, or can be pulled straight from the **Apify API**. Unlike most Tokopedia scrapers, it also offers **cross-run deduplication** — name a project once and every later run returns only products you have never scraped before.

**Target website:** [tokopedia.com](https://www.tokopedia.com)

### ✨ Features of this Tokopedia product scraper

- **Product data extraction** — title, shop, price, currency, stock level, category breadcrumbs, full description and image URLs.
- **Pagination support** — walks keyword listing pages automatically until the per-URL item limit is reached.
- **Proxy support** — full Apify proxy configuration, defaulting to Indonesian residential for reliable access.
- **Automatic URL detection** — listing and product URLs are told apart automatically, so you can mix both in Start URLs.
- **Whole-site mode** — one toggle crawls a set of main keyword listing pages instead of a hand-picked list.
- **Cross-run deduplication (unique feature)** — set a `cacheProjectName` once and the actor remembers every product URL it has ever scraped under that name, then skips them on all future runs. Most scrapers dedupe only within a single run; this one persists across runs, schedules and months.
- **No browser required** — runs on plain HTTP requests instead of a headless browser, which makes runs fast and cheap.

### 🚀 How to scrape Tokopedia products in 5 steps

1. [Sign up](https://apify.com/sign-up) for a free Apify account — includes **$5 monthly credit**.
2. Open the actor page and click **Try for free**.
3. Fill in the **Input** fields (at least one start URL is required).
4. Click **Start** and wait for the run to complete.
5. Download results from the **Output** tab in JSON, CSV, or Excel format.

You can also run this actor via the [Apify API](https://docs.apify.com/api/v2) or integrate it directly into your workflows using [Zapier](https://zapier.com/apps/apify), [Make](https://www.make.com/), or [n8n](https://n8n.io/).

### 💰 Pricing

This actor uses **pay-per-result** billing based on the compute units a run consumes.

- New Apify accounts include **$5 of free monthly credit**.
- Because the scraper uses plain HTTP requests rather than a headless browser, it costs significantly less to run than browser-based Tokopedia scrapers.
- Proxy usage is billed separately. Tokopedia does respond without a proxy at low volume, so switching `proxyConfiguration` off is the cheapest way to run small jobs.

### 🔧 Input configuration

| Field                | Type    | Required | Default                                                                                   | Description                                                                                                                                                                                              |
| -------------------- | ------- | -------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `startUrls`          | array   | ✅       | `[{"url": "https://www.tokopedia.com/find/laptop"}]`                                      | Tokopedia listing URLs (`/find/{keyword}`) or direct product URLs. The type of each URL is detected automatically.                                                                                       |
| `maxItems`           | integer | —        | `5`                                                                                       | Maximum number of products to scrape for each start URL.                                                                                                                                                 |
| `scrapeWholeSite`    | boolean | —        | `false`                                                                                   | Ignore Start URLs and crawl the main Tokopedia keyword listing pages instead.                                                                                                                            |
| `cacheProjectName`   | string  | —        | —                                                                                         | Optional. When set, products already scraped under this project name in a previous run are skipped instead of being scraped again.                                                                       |
| `proxyConfiguration` | object  | —        | `{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "ID"}` | Tokopedia responds without a proxy in testing but blocks bursts of traffic from datacenter ranges. Indonesian residential is the default; datacenter or no proxy is cheaper if it works for your volume. |

**Supported URL types:**

- **Listing pages** — keyword listing pages such as `https://www.tokopedia.com/find/laptop` (this is the listing path Tokopedia's `robots.txt` allows for crawling).
- **Product pages** — `{shop}/{product}` URLs, e.g. `https://www.tokopedia.com/mage-goods/mage-genus-anti-bara-backpack-tas-gendong-pria-wanita-premium-ransel-laptop-14-inch-1730838809683789809`.

### 🔁 Cross-run deduplication — scrape only new Tokopedia products

Most Tokopedia scrapers deduplicate only *within* one run: start them again tomorrow on the same keyword and you pay to
re-scrape and re-store every product you already have. This actor deduplicates **across runs**, which makes it well suited
to recurring price monitoring and incremental catalogue building.

**How it works:**

1. Set the **Cache project name** (`cacheProjectName`) input to any label you choose, e.g. `laptop-price-monitor`.
2. The actor opens a named Apify key-value store under that name and loads the set of product URLs it has scraped before,
   stored under the `SCRAPED_URLS` key.
3. While crawling, any product URL already in that set is skipped before the page is parsed or pushed — it never reaches
   your dataset and never counts against `maxItems`.
4. At the end of the run the set is written back, so the next run picks up exactly where this one left off.

**What this gives you:**

- **Incremental scraping** — a daily or weekly [scheduled run](https://docs.apify.com/platform/schedules) returns only the
  products that are genuinely new, instead of the same catalogue over and over.
- **Lower cost** — skipped products are not fetched, parsed or stored, so repeat runs consume fewer compute units and
  fewer proxy requests than a full re-scrape.
- **Clean datasets** — no duplicate rows to filter out downstream in your database, dashboard or repricing tool.
- **Separate projects, separate memory** — use a different project name per keyword, client or pipeline and each keeps its
  own independent history. Leave the field empty and the actor behaves like a normal scraper, scraping everything every time.

> **Tip:** deduplication is keyed on the product URL, so a product whose price changes is *not* re-scraped under the same
> project name. For price tracking over time, leave `cacheProjectName` empty; for discovering newly listed products, set it.

### 📦 Tokopedia scraper output data

Each run produces a structured JSON dataset (exportable to CSV or Excel) with one record per product, containing `url`, `title`, `shop`, `price`, `currency`, `priceText`, `description`, `stock`, `stockText`, `breadcrumbs`, `category` and `images`. The dataset has one view: **Overview**, a compact table of each product's title, shop, price, currency, rating, review count and URL.

**Sample output:**

```json
{
    "url": "https://www.tokopedia.com/mage-goods/mage-genus-anti-bara-backpack-tas-gendong-pria-wanita-premium-ransel-laptop-14-inch-1730838809683789809",
    "title": "MAGE Genus Anti Bara Backpack | Tas Gendong Pria Wanita Premium | Ransel Laptop 14 Inch - Black",
    "shop": "mage-goods",
    "price": 172000,
    "currency": "IDR",
    "priceText": "Rp172.000",
    "description": "GENUS ANTI BARA adalah ransel laptop premium yang dirancang khusus untuk pria. Dengan desain modern dan fungsional, ransel ini cocok untuk penggunaan sehari-hari maupun perjalanan. Kapasitas Besar: Dapat menampung laptop hingga 14 inci dan memiliki ruang tambahan untuk barang lainnya.",
    "stock": 10,
    "stockText": "Stok: Sisa 10",
    "breadcrumbs": ["Fashion Pria", "Tas Pria", "Tas Ransel Pria"],
    "category": "Tas Ransel Pria",
    "images": [
        "https://images.tokopedia.net/img/cache/700/aphluv/1997/1/1/b5283de0d30e4bb5ad4642f847c65830~.jpeg.webp",
        "https://images.tokopedia.net/img/cache/150/aphluv/1997/1/1/b5283de0d30e4bb5ad4642f847c65830~.jpeg"
    ]
}
```

### 🐍 How to scrape Tokopedia with Python, JavaScript or the API

Run the actor programmatically with the official Apify clients. Replace `<YOUR_API_TOKEN>` with the token from your [Apify Console](https://console.apify.com/account/integrations).

**Python** (`pip install apify-client`):

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_API_TOKEN>")

run = client.actor("<username>/tokopedia-products-scraper").call(run_input={
    "startUrls": [{"url": "https://www.tokopedia.com/find/laptop"}],
    "maxItems": 5,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)
```

**JavaScript** (`npm install apify-client`):

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_API_TOKEN>' });

const run = await client.actor('<username>/tokopedia-products-scraper').call({
    startUrls: [{ url: 'https://www.tokopedia.com/find/laptop' }],
    maxItems: 5,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

**cURL** — start a run and wait for the dataset:

```bash
curl -X POST "https://api.apify.com/v2/acts/<username>~tokopedia-products-scraper/run-sync-get-dataset-items?token=<YOUR_API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://www.tokopedia.com/find/laptop"}],"maxItems":5}'
```

### 💡 What you can use Tokopedia product data for

- Monitoring competitor pricing across Indonesian marketplace sellers
- Tracking price changes and stock levels over time
- Building product and assortment catalogues for market analysis
- Sourcing and supplier research on Indonesia's largest marketplace
- Feeding pricing dashboards and automated repricing tools
- Incremental catalogue building — detecting newly listed products without re-scraping the ones you already have

This data is typically used by e-commerce teams, price analysts, brand protection specialists and market researchers working in retail, consumer goods, price intelligence and market research.

### ⚠️ Limitations & known issues

- **Rate limiting** — Tokopedia drops bursts of traffic from datacenter ranges. Runs succeed without a proxy at modest volume; for larger runs keep the residential proxy default and concurrency low.
- **Ratings and review counts** — these are loaded by the site after the initial page response, so they are not part of every product record even though the Overview view includes the columns.
- **Search paths** — Tokopedia's `robots.txt` disallows `/search`-style URLs, so the actor uses the permitted `/find/{keyword}` listing pages instead.
- **Region-dependent data** — prices and availability can vary by location, so results may differ between proxy regions.

### ❓ Frequently asked questions

#### Can I scrape Tokopedia legally?

This actor only collects data that is already publicly visible on Tokopedia product pages — it does not log in, add items to a cart, or access seller-only or buyer-only areas. Scraping publicly available data is generally considered lawful (see *hiQ Labs v. LinkedIn* as precedent). You remain responsible for complying with Tokopedia's Terms of Service and with any applicable laws, including GDPR or CCPA if you process personal data.

#### Does this Tokopedia scraper get product ratings and review counts?

Not reliably. Tokopedia loads ratings and review counts after the initial page response, so those fields are absent from most records. Price, stock, description, category and images come from the initial response and are captured consistently.

#### How many products can I scrape from one Tokopedia URL?

As many as you want — set `maxItems` to control it. The limit applies **per start URL**, not across the whole run, and the scraper pages through keyword listings automatically until it hits that number.

#### Do I need a proxy to scrape tokopedia.com?

Not for small runs — Tokopedia responds without a proxy at low volume. For anything larger the actor defaults to Indonesian residential proxies, because Tokopedia blocks bursts of traffic from datacenter ranges. Turning proxies off is the cheapest way to run small jobs.

#### How do I scrape Tokopedia with Python?

Install the official client with `pip install apify-client`, then call the actor with your start URLs and iterate the resulting dataset. See the [Python example above](#-how-to-scrape-tokopedia-with-python-javascript-or-the-api) for a complete snippet.

#### How do I scrape only new Tokopedia products and skip ones I already have?

Set the **Cache project name** (`cacheProjectName`) input. The actor stores every product URL it scrapes under that project
name and skips those URLs on every future run, so each run returns only products that are new to you. This cross-run
deduplication is what makes the actor practical for incremental, scheduled price and assortment monitoring — see
[Cross-run deduplication](#-cross-run-deduplication--scrape-only-new-tokopedia-products) above.

#### Can I run this Tokopedia scraper on a schedule?

Yes. Use [Apify Schedules](https://docs.apify.com/platform/schedules) to run the actor hourly, daily or weekly. Set `cacheProjectName` to a fixed value and each scheduled run will skip products already scraped under that name, so you only collect what's new — this pairing of schedules with cross-run deduplication is the intended way to run the actor long-term.

#### What output formats are supported?

JSON, CSV, Excel, XML, HTML table and RSS — downloadable from the run's Output tab or retrievable through the Apify API.

***

### 🇮🇩 Panduan Singkat Scraper Produk Tokopedia (Bahasa Indonesia)

Actor ini mengambil data produk dari **Tokopedia** secara otomatis — nama produk, toko, harga, stok, kategori, deskripsi, dan gambar.

**Cara pakai:**

1. Isi **Start URLs** dengan halaman daftar produk (misalnya `https://www.tokopedia.com/find/laptop`) atau langsung URL produk. Jenis URL dideteksi otomatis.
2. Atur **Max items per start URL** untuk membatasi jumlah produk per URL.
3. Aktifkan **Scrape the whole site** jika ingin menelusuri beberapa kategori utama sekaligus.
4. Isi **Cache project name** (opsional) untuk **deduplikasi antar-run**: actor mengingat semua URL produk yang pernah diambil dengan nama proyek tersebut, lalu melewatinya pada run berikutnya — jadi Anda hanya mendapat produk baru dan biaya run jauh lebih hemat.
5. Klik **Start**, lalu unduh hasilnya dalam format JSON, CSV, atau Excel.

**Catatan:** actor ini hanya mengambil data yang tampil untuk publik di Tokopedia — tanpa login. Untuk run berskala besar, sebaiknya aktifkan proxy agar permintaan tidak diblokir.

### 💬 Support & contact

If you encounter any issues or have questions, please [open an issue](https://apify.com/confidential_gnat/tokopedia-products-scraper/issues/open).

You can also find more of our actors on the [Apify Store](https://apify.com/confidential_gnat).

# Actor input Schema

## `startUrls` (type: `array`):

Tokopedia listing URLs (/find/{keyword}) or direct product URLs. The type of each URL is detected automatically.

## `maxItems` (type: `integer`):

Maximum number of products to scrape for each start URL.

## `scrapeWholeSite` (type: `boolean`):

Ignore Start URLs and crawl the main Tokopedia keyword listing pages instead.

## `cacheProjectName` (type: `string`):

Optional. When set, products already scraped under this project name in a previous run are skipped instead of being scraped again - so each run returns only new products.

## `proxyConfiguration` (type: `object`):

Tokopedia responds without a proxy in testing, but blocks bursts of traffic from datacenter ranges. Enable datacenter proxies first if you start seeing dropped requests, and Indonesian residential only if datacenter still fails.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.tokopedia.com/find/laptop"
    }
  ],
  "maxItems": 5,
  "scrapeWholeSite": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "ID"
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.tokopedia.com/find/laptop"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("confidential_gnat/tokopedia-products-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.tokopedia.com/find/laptop" }] }

# Run the Actor and wait for it to finish
run = client.actor("confidential_gnat/tokopedia-products-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.tokopedia.com/find/laptop"
    }
  ]
}' |
apify call confidential_gnat/tokopedia-products-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,confidential_gnat/tokopedia-products-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/b8Og94Lrza0yhfNEg/builds/Ch3WrVVJLVuTrpcEy/openapi.json