Indonesia Stock Market Scraper

English

Apify actor for scraping IDX/LQ45 stock quote data from Yahoo Finance.

Features

• Supports IDX tickers such as BBCA or BBCA.JK.
• Automatically normalizes tickers to the .JK suffix.
• Uses the full default LQ45 list defined in the code when no symbols are provided.
• Can limit the number of processed symbols through maxItems.
• Can filter saved dataset fields through outputFields.
• Uses Playwright as the primary path and falls back to price data recovery when the main Yahoo Finance DOM is incomplete.
• Generates an HTML output report in Apify so users can review results in a cleaner UI than raw JSON alone.

Output

Available fields:

• symbol
• name
• price
• marketCap
• peRatio
• forwardPe
• priceToBook
• previousClose
• open
• volume
• avgVolume
• dayRange
• fiftyTwoWeekRange
• timestamp

Full 5-symbol sample dataset:

• Input example: examples/full-output-5-symbols-input.json
• Output example: examples/full-output-5-symbols.json

Actor Input

Example input:

{
  "maxItems": 45,
  "outputFields": ["symbol", "name", "price", "marketCap", "volume", "timestamp"],
  "useFallbackOnly": false
}

Details:

• symbols: list of IDX tickers. You can provide BBCA or BBCA.JK. If omitted, the actor uses the default LQ45 list.
• maxItems: limits how many symbols are processed after normalization and deduplication.
• outputFields: if provided, only the selected fields are stored in the dataset.
• useFallbackOnly: uses only the Yahoo chart API without DOM scraping.

Sample Output

[
  {
    "symbol": "BBCA.JK",
    "name": "PT Bank Central Asia Tbk (BBCA.JK)",
    "price": "6,725.00",
    "marketCap": "808.81T",
    "peRatio": "14.08",
    "forwardPe": "13.21",
    "priceToBook": "2.87",
    "previousClose": "6,575.00",
    "open": "6,675.00",
    "volume": "64,289,000",
    "avgVolume": "213,106,377",
    "dayRange": "6,675.00 - 6,800.00",
    "fiftyTwoWeekRange": "6,350.00 - 9,800.00",
    "timestamp": "2026-04-14T05:02:10.938Z"
  }
]

Important Notes

• The recommended default mode is useFallbackOnly: false.
• useFallbackOnly: true is available, but Yahoo may return 429 Too Many Requests for HTTP-only requests. Use it only when needed.
• The actor is more stable with low concurrency. The current configuration uses maxConcurrency: 1.
• The currently validated working versions are apify 3.7.0, crawlee 3.16.0, and playwright 1.59.1.

Run Locally

Install dependencies:

$npm install

Run the actor:

$npm start

The local sample input is stored in storage/key_value_stores/default/INPUT.json.

Push To Apify

Make sure the Apify CLI is logged in:

$apify login

Then push the actor:

$apify push

The actor input schema is defined in .actor/input_schema.json, so the input form renders properly in Apify Console.

Output Tab

The actor also generates an HTML report and a JSON summary in the default key-value store. In Apify Console, these are exposed in the Output tab for a cleaner post-run experience.

Actor Schemas

• .actor/input_schema.json: controls the run input form in Apify Console.
• .actor/output_schema.json: defines what appears in the Output tab after a run.
• .actor/dataset_schema.json: documents dataset item fields and configures Overview, Compact Quotes, Valuation Focus, Liquidity Watch, and Full Metrics views.
• .actor/key_value_store_schema.json: groups HTML report, summary JSON, error JSON, and SDK diagnostics in the Storage tab.

Bahasa Indonesia

Actor Apify untuk mengambil data quote emiten IDX/LQ45 dari Yahoo Finance.

Fitur

• Mendukung input ticker IDX seperti BBCA atau BBCA.JK.
• Otomatis menormalkan ticker ke suffix .JK.
• Default list memakai seluruh daftar LQ45 yang ada di kode.
• Bisa membatasi jumlah simbol lewat maxItems.
• Bisa memfilter field output lewat outputFields.
• Menggunakan Playwright untuk jalur utama dan fallback data harga jika DOM utama tidak lengkap.
• Menghasilkan laporan HTML di Apify Output tab agar hasil run lebih mudah dibaca dibanding JSON mentah saja.

Output

Field yang tersedia:

• symbol
• name
• price
• marketCap
• peRatio
• forwardPe
• priceToBook
• previousClose
• open
• volume
• avgVolume
• dayRange
• fiftyTwoWeekRange
• timestamp

Contoh dataset penuh untuk 5 emiten:

• Input contoh: examples/full-output-5-symbols-input.json
• Output contoh: examples/full-output-5-symbols.json

Input Actor

Contoh input:

{
  "maxItems": 45,
  "outputFields": ["symbol", "name", "price", "marketCap", "volume", "timestamp"],
  "useFallbackOnly": false
}

Keterangan:

• symbols: daftar ticker IDX. Boleh BBCA atau BBCA.JK. Jika dikosongkan, actor memakai daftar default LQ45.
• maxItems: membatasi jumlah simbol yang diproses setelah normalisasi dan deduplikasi.
• outputFields: jika diisi, dataset hanya menyimpan field yang dipilih.
• useFallbackOnly: hanya memakai chart API Yahoo tanpa scraping DOM.

Contoh Output

[
  {
    "symbol": "BBCA.JK",
    "name": "PT Bank Central Asia Tbk (BBCA.JK)",
    "price": "6,725.00",
    "marketCap": "808.81T",
    "peRatio": "14.08",
    "forwardPe": "13.21",
    "priceToBook": "2.87",
    "previousClose": "6,575.00",
    "open": "6,675.00",
    "volume": "64,289,000",
    "avgVolume": "213,106,377",
    "dayRange": "6,675.00 - 6,800.00",
    "fiftyTwoWeekRange": "6,350.00 - 9,800.00",
    "timestamp": "2026-04-14T05:02:10.938Z"
  }
]

Catatan Penting

• Mode default yang direkomendasikan adalah useFallbackOnly: false.
• useFallbackOnly: true sudah tersedia, tetapi Yahoo kadang memberi 429 Too Many Requests untuk jalur HTTP-only. Gunakan hanya jika memang perlu.
• Actor ini lebih stabil bila dijalankan dengan concurrency rendah. Konfigurasi saat ini memakai maxConcurrency: 1.
• Versi yang tervalidasi berjalan normal: apify 3.7.0, crawlee 3.16.0, dan playwright 1.59.1.

Jalankan Lokal

Install dependency:

$npm install

Jalankan actor:

$npm start

Contoh input lokal ada di storage/key_value_stores/default/INPUT.json.

Push Ke Apify

Pastikan Apify CLI sudah login:

$apify login

Lalu push actor:

$apify push

Schema input actor ada di .actor/input_schema.json, sehingga field input akan muncul rapi di Apify Console.

Output Tab

Actor ini juga membuat laporan HTML dan ringkasan JSON di key-value store default. Di Apify Console, hasil tersebut bisa ditampilkan di Output tab agar pengalaman pengguna lebih nyaman.

Schema Actor

• .actor/input_schema.json: mengatur form input di Apify Console.
• .actor/output_schema.json: menentukan apa yang muncul di Output tab setelah run selesai.
• .actor/dataset_schema.json: mendeskripsikan field dataset dan mengatur view Overview, Compact Quotes, Valuation Focus, Liquidity Watch, dan Full Metrics.
• .actor/key_value_store_schema.json: mengelompokkan laporan HTML, summary JSON, error JSON, dan file diagnostik SDK di Storage tab.