# Tokopedia Scraper — Products & Suppliers, One for All (`scrapenode/tokopedia-scraper-products-suppliers-one-for-all`) Actor

Search Tokopedia products over pure HTTP — no browser, no cookie, no login — and turn the results into a supplier list you can compare. Built for price research and supplier discovery in Indonesia.

- **URL**: https://apify.com/scrapenode/tokopedia-scraper-products-suppliers-one-for-all.md
- **Developed by:** [Muhammad Sajid Izzulhaq](https://apify.com/scrapenode) (community)
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Tokopedia Product Search

Search products on Tokopedia over pure HTTP and group their sellers into a comparable supplier list. The whole pipeline is **pure HTTP** — no headless browser.

From a residential IP (e.g. your home machine when running locally) no proxy is needed at all. From a datacenter IP — including the **Apify runtime** — Tokopedia silently drops the request at the Akamai edge (status 0 on every transport, not a 403), so a **residential** proxy is required. See **Proxy** below.

The endpoint used is `SearchProductV5` in Tokopedia's internal GraphQL. The query document was taken from the frontend JS bundle, then cleaned of `@client`-marked fields the server never receives. Only three headers are required: `content-type`, `user-agent`, `accept`. Without `accept`, the server hangs until timeout — it does not reject — so no error can be caught.

### Input

| Input | Type | Default | Description |
|---|---|---|---|
| `searchStrings` | list of string | `["laptop"]` | One or more product keywords. All are combined in a single run so the **Toko** sheet immediately sees every seller that appears under any keyword. |
| `maxItems` | integer | `100` | Limit of unique products **for the whole run**, split evenly across keywords. The server accepts at most 200 rows per request, so a value above 200 means more than one request. Maximum `600`. |
| `orderBy` | enum | `"23"` | `"23"` = relevance (Tokopedia default), `"5"` = highest rating. |
| `proxy` | proxy | — | Proxy Configuration component. Pick a residential proxy from your own account, or paste a custom proxy URL. If left empty, the actor falls back to the runtime's residential proxy. See **Proxy** below. |
| `minPrice` | integer | — | Drop products priced below this number (Rupiah). Products without a price are dropped too. Empty = no filtering. |
| `maxPrice` | integer | — | Drop products priced above this number (Rupiah). Products without a price are dropped too. Empty = no filtering. |
| `minRating` | number | — | Drop products rated below this number (0-5). Products without reviews are dropped too. Empty = no filtering. |
| `debugDump` | boolean | `false` | Save the raw GraphQL payload to storage. Turn on only when results look wrong. |

The hard `maxItems` cap lives in `tokopedia_core.MAX_PLAUSIBLE_LIMIT`, shared by the input schema and the code, so the number in the UI and in Python can never drift apart.

The `minPrice`, `maxPrice`, and `minRating` filters are **client-side**: applied after results are fetched, not sent to the server (Tokopedia's server is not confirmed to accept `pmin`/`pmax`/`rt`). That is why the number of products returned can be **lower** than `maxItems` — raise `maxItems` if you want more results to survive the filter. How many were dropped is logged and recorded in `SUMMARY` (`filtered`).

### Proxy

Three paths, tried in order:

1. **The `proxy` input component** — highest priority. The user picks a residential proxy from their own account, or pastes a custom proxy URL. Credentials belong to the account that **runs** the run, so the quota consumed is the runner's, not the actor owner's.
2. **`SCRAPER_PROXY`** — an environment variable holding your own rented proxy in `http://user:pass@host:port` format. Used when the Proxy component is empty.
3. **Apify residential proxy** — used automatically when both are empty and the runtime provides `APIFY_PROXY_PASSWORD`. Built with group `RESIDENTIAL` and `country-ID` so the egress is Indonesian residential.

If none of the three exist, the actor still runs without a proxy and will fail from a datacenter IP. That is not a transport configuration bug — use a residential proxy.

> Apify's default proxy group is **datacenter** and won't get through: it has been tested and gives the exact same status 0 as the direct path. It must be `RESIDENTIAL`.

### Output

One dataset row = one product, 19 columns:

`no`, `query`, `page`, `oldID`, `name`, `url`, `price`, `priceOriginal`, `discountPercent`, `countSold`, `rating`, `shopId`, `shopName`, `shopUrl`, `shopCity`, `shopTier`, `categoryName`, `categoryBreadcrumb`, `imageUrl`

Two views in the Output tab:

- **Produk (Product)** — the natural reading order when comparing suppliers: what is sold, its price, sales and satisfaction proof, then who the seller is.
- **All columns** — every field, raw.

Three artifacts also land in the key-value store:

- **`products.xlsx`** — two sheets: `Produk` (15 columns, reading order) and `Toko` (13 columns, aggregated) plus a `Ringkasan` sheet. Download it through the **Storage** button. The spreadsheet shows a subset of the 19 dataset columns; the rest (`oldID`, long `url`, `imageUrl`, breadcrumb) are intentionally left out so the table stays readable.
- **`SUMMARY`** — run statistics: number of requests, duplicates, empty-response retries, and whether the server ran out of results.

#### What to know when reading the columns

- **`price` has no divisor.** `price.number` is identical to the number shown on the page (`98254` = Rp98,254). There is no /100 or /1000 anywhere in the codebase.
- **`discountPercent` is computed from the promo label**, not from `discountPercentage` in the response — that field measured 0 on 100/100 products. When the `ri_ribbon` label is absent, the value is `null`, not `0`.
- **`countSold` can be `null`.** `"250+ terjual"` means *at least* 250, and `"7rb+ terjual"` means *at least* 7,000. `null` means the seller shows no label, not zero.
- **`shopTier` is the only official-status marker**: 1 = Tokopedia Mall, 2 = Power Merchant, 3 = regular seller. The `badge.url` filename is **not** a marker — `official_store/badge_os.png` also shows up for tier 3 sellers, so an "Official" column is deliberately absent.
- **`rating` can be `null`** for products with no reviews, and it can be a tiered string (`5.0`, `4.9`, `4.8`) — that is the real value, not rounding.
- **`imageUrl` is signed and carries `x-expires`.** Don't store it as an asset.
- **`oldID` is the dedup key**, not `url`, because the URL carries campaign parameters that change every session.

### Pagination and server limits

`maxItems` is a **total**, not per page. The server silently truncates to **200 rows per request**; asking for 300 or 500 still returns 200. Offset moves by `start = (page - 1) * rows` — not by `page`. Duplicates across windows are real (the search index shifts between two requests), so dedup keeps running.

Responses are sometimes empty without an error (HTTP 200, `products: []`), so every page is retried before it's considered finished. If the next page adds no new products, the scraper stops on its own — further requests would just waste quota.

The pause between requests is 1.5–3 seconds with jitter, and exponential backoff for 429/503. All of that lives in `tokopedia_core.TokopediaSession`, not in the actor.

### Running locally

```bash
mkdir -p .tmp-actor-run/storage/key_value_stores/default
## write input to .tmp-actor-run/storage/key_value_stores/default/INPUT.json
CRAWLEE_STORAGE_DIR=.tmp-actor-run/storage \
PYTHONPATH="actors/tokopedia-search;packages/tokopedia-core/src" \
  .venv/Scripts/python -m src
```

The actor's build context is the **repo root** (`dockerContextDir: "../../.."`), which is why `packages/tokopedia-core` must be copied along. Don't change it to `COPY . .`.

### Tests

```bash
pytest packages/tokopedia-core/tests -q
```

Parsing regressions use raw payloads in `data/scraped/_debug/tokopedia-*.json` as fixtures, so no internet connection is needed.

# Actor input Schema

## `searchStrings` (type: `array`):

Satu atau beberapa kata kunci produk, contoh 'laptop', 'sepatu lari', 'kopi specialty'. Seluruh kata kunci digabung dalam satu run supaya sheet Toko langsung melihat semua penjual yang muncul.

## `maxItems` (type: `integer`):

Batas jumlah produk unik untuk seluruh run, dibagi rata ke tiap kata kunci. Server Tokopedia menerima maksimal 200 baris per request, jadi nilai di atas 200 berarti lebih dari satu request.

## `orderBy` (type: `string`):

Cara server mengurutkan hasil. Relevansi adalah urutan bawaan Tokopedia; rating tertinggi dipakai kalau yang dicari adalah toko yang sudah terbukti.

## `proxy` (type: `object`):

Pilih proxy residensial dari akunmu sendiri, atau masukkan URL proxy custom milikmu. Kalau dibiarkan kosong, actor memakai proxy residensial dari environment APIFY_PROXY_PASSWORD milik akun yang menjalankan run. Kredensial tidak pernah ditulis di kode.

## `minPrice` (type: `integer`):

Buang produk dengan harga di bawah angka ini (Rupiah, tanpa titik). Contoh 50000 untuk menyaring produk di bawah Rp50.000. Produk tanpa harga ikut dibuang.

## `maxPrice` (type: `integer`):

Buang produk dengan harga di atas angka ini (Rupiah, tanpa titik). Contoh 500000 untuk menyaring produk di atas Rp500.000. Produk tanpa harga ikut dibuang.

## `minRating` (type: `number`):

Buang produk dengan rating di bawah angka ini (skala 0-5). Contoh 4.5. Produk yang belum punya ulasan (rating kosong) ikut dibuang.

## `debugDump` (type: `boolean`):

Simpan payload GraphQL apa adanya ke storage. Nyalakan hanya kalau hasil terlihat aneh atau parsing bermasalah.

## Actor input object example

```json
{
  "searchStrings": [
    "laptop"
  ],
  "maxItems": 100,
  "orderBy": "23",
  "minPrice": 50000,
  "maxPrice": 500000,
  "minRating": 4.5,
  "debugDump": false
}
```

# Actor output Schema

## `products` (type: `string`):

Semua produk hasil pencarian Tokopedia, termasuk harga, jumlah terjual, rating, dan data toko penjualnya.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchStrings": [
        "laptop"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapenode/tokopedia-scraper-products-suppliers-one-for-all").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchStrings": ["laptop"] }

# Run the Actor and wait for it to finish
run = client.actor("scrapenode/tokopedia-scraper-products-suppliers-one-for-all").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchStrings": [
    "laptop"
  ]
}' |
apify call scrapenode/tokopedia-scraper-products-suppliers-one-for-all --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrapenode/tokopedia-scraper-products-suppliers-one-for-all"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/szFvfb3pPFsN9VlZR/builds/Y8WiIwZNLfv3gbYbv/openapi.json