# Indonesian News Scraper (`automation-lab/indonesian-news-aggregation-feed`) Actor

Search Kompas and Detik by topic and export normalized public Indonesian news for recurring media monitoring.

- **URL**: https://apify.com/automation-lab/indonesian-news-aggregation-feed.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Indonesian News Scraper

Collect public **Indonesian news** from Kompas and Detik in one normalized feed. Search several topics, select either publisher or both, and export headlines, summaries, article links, publication times, sections, images, and collection provenance for media monitoring.

The Actor uses the publishers' public topic-search pages. It does not require a publisher account, browser automation, or a residential proxy.

### What Indonesian News Scraper does

Indonesian News Scraper turns two different publisher result formats into one stable dataset.

For every topic, it:

1. searches the selected Kompas and Detik public surfaces;
2. paginates until the requested limit is reached or results end;
3. alternates records across selected publishers for balanced coverage;
4. removes duplicate canonical article URLs;
5. normalizes the remaining records into one output schema;
6. stores the results in the run's default Apify dataset.

Results are shared fairly across multiple topics, so a large first topic does not consume the entire result limit before later topics run.

### Who is it for

- **PR and communications teams** tracking coverage of a company, executive, campaign, or issue.
- **Media-monitoring analysts** comparing Kompas and Detik reporting.
- **Market researchers** collecting Indonesian economic, technology, policy, or consumer-news signals.
- **Newsrooms and academics** building a repeatable article-discovery dataset.
- **Data teams** sending current public news metadata to a spreadsheet, warehouse, or alerting pipeline.

This Actor is for article discovery and metadata monitoring. It does not download full article bodies or generate sentiment, emotion, or factuality scores.

### Why use this Indonesian news feed

- One schema across Kompas and Detik.
- Topic and publisher controls instead of an unfocused site crawl.
- Balanced multi-source and multi-topic collection.
- Stable canonical URLs for deduplication between scheduled runs.
- Public metadata only; no login or account handling.
- Lightweight direct HTTP extraction with 256 MB of memory.
- Per-result charging: rejected, duplicate, and empty records are not item-charged.

### Data you can extract

| Field | Meaning |
| --- | --- |
| `title` | Article headline shown by the publisher |
| `publisher` | Normalized `Kompas` or `Detik` name |
| `section` | Publisher section or channel, when available |
| `articleUrl` | Canonical public article URL |
| `publishedAt` | Best-effort ISO 8601 publication date/time |
| `publishedText` | Date/time text exactly as exposed by the source |
| `summary` | Public search-result excerpt, when available |
| `imageUrl` | Search-result thumbnail URL, when available |
| `query` | Input topic that produced the result |
| `rank` | One-based accepted-result rank for that topic |
| `page` | One-based publisher result page |
| `sourceUrl` | Search page used to collect the record |
| `fetchedAt` | ISO 8601 collection timestamp |

Nullable fields remain `null` when a publisher does not expose them. The Actor does not invent missing summaries or exact publication times.

### Get started

1. Open the Actor in Apify Console.
2. Add one or more Indonesian topics under **News topics**.
3. Keep both publishers selected, or choose only Kompas or Detik.
4. Set **Maximum articles** for the whole run.
5. Click **Start**.
6. Open the **News feed** dataset view to inspect or export the records.

A useful first run is:

```json
{
  "queries": ["kecerdasan buatan"],
  "publishers": ["kompas", "detik"],
  "maxItems": 20
}
```

### Input parameters

#### `queries`

Required array of 1–20 non-empty topic strings. Each topic can contain up to 150 characters.

Use concrete Indonesian terms such as:

- `kecerdasan buatan`
- `ekonomi Indonesia`
- `energi terbarukan`
- `kebijakan pemerintah`

Multiple topics receive a fair share of `maxItems`. If an earlier topic has too few results, later topics can use the remaining capacity.

#### `publishers`

Required source selection with one or both values:

- `kompas`
- `detik`

The default includes both. Selecting one publisher is useful for source-specific editorial monitoring.

#### `maxItems`

Maximum number of unique records saved across all topics and publishers.

- minimum: `1`
- default: `100`
- maximum: `1000`

A lower value makes smoke tests faster and cheaper. A higher value may require more publisher result pages.

### Output example

This record shape comes from the current public Kompas search behavior; live headlines change over time:

```json
{
  "title": "Mengurai Kemacetan Rantai Pasok Sumatera lewat Kecerdasan Buatan",
  "publisher": "Kompas",
  "section": "Properti",
  "articleUrl": "https://www.kompas.com/properti/read/2026/07/31/082820121/mengurai-kemacetan-rantai-pasok-sumatera-lewat-kecerdasan-buatan",
  "publishedAt": "2026-07-31T00:00:00.000Z",
  "publishedText": "31 Juli 2026",
  "summary": "KIM mengelola dua kawasan industri utama yang menampung ratusan aktivitas manufaktur dan logistik skala regional maupun internasional.",
  "imageUrl": "https://asset.kompas.com/example-image.jpg",
  "query": "kecerdasan buatan",
  "rank": 1,
  "page": 1,
  "sourceUrl": "https://search.kompas.com/search/?q=kecerdasan+buatan&page=1",
  "fetchedAt": "2026-08-14T14:50:00.000Z"
}
```

The example image URL is shortened for documentation. Dataset records retain the source image URL.

### How much does it cost to collect Indonesian news?

Pricing combines one small Actor-start charge with one `item` event for each accepted dataset record. Duplicate URLs, malformed cards, and valid no-result searches do not produce item charges.

The one-time start price is `$0.0005`. Item prices decrease across the six Apify tiers:

| Tier | Price per accepted article |
| --- | ---: |
| FREE | $0.0007636 |
| BRONZE | $0.000664 |
| SILVER | $0.00051792 |
| GOLD | $0.0003984 |
| PLATINUM | $0.0002656 |
| DIAMOND | $0.00018592 |

For planning, multiply your tier's item price by the requested record ceiling, then add the single start charge. At the BRONZE price:

| Accepted records | Example total |
| ---: | ---: |
| 10 | $0.00714 |
| 100 | $0.06690 |
| 500 | $0.33250 |

A run may cost less than that ceiling if the selected topics have fewer available results. These examples do not promise that every query returns the requested maximum.

### Media-monitoring workflows

#### Monitor a brand or organization

Schedule a daily run with the organization name and both publishers. Use `articleUrl` as the stable key when comparing today's dataset with yesterday's.

#### Compare publisher coverage

Search one policy or market topic across both publishers. Group the output by `publisher`, then compare headline language, section placement, and publication timing.

#### Build a multi-topic policy feed

Submit several related policy terms in one run. The Actor allocates result capacity across topics rather than allowing the first topic to consume all output.

#### Send results to a spreadsheet or warehouse

Connect the default dataset to Make, Zapier, Google Sheets, or an Apify webhook. Store `query`, `publisher`, `publishedAt`, and `fetchedAt` alongside each canonical URL for auditable monitoring.

### Scheduling and change detection

Apify schedules can run this Actor hourly, daily, or weekly. For change detection:

1. schedule the same input;
2. export each run's default dataset;
3. compare records by `articleUrl`;
4. treat unseen URLs as newly discovered coverage;
5. use `fetchedAt` as collection provenance, not as publication time.

The Actor itself does not persist a cross-run alert state or send notifications. Use an integration, webhook, or downstream dataset comparison for alerts.

### API usage with cURL

Start a run and wait for its dataset:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~indonesian-news-aggregation-feed/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["ekonomi Indonesia"],
    "publishers": ["kompas", "detik"],
    "maxItems": 20
  }'
```

Keep your Apify token in an environment variable. Do not commit it to source control.

### API usage with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/indonesian-news-aggregation-feed').call({
  queries: ['energi terbarukan'],
  publishers: ['kompas', 'detik'],
  maxItems: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### API usage with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/indonesian-news-aggregation-feed').call(run_input={
    'queries': ['kebijakan pemerintah'],
    'publishers': ['kompas', 'detik'],
    'maxItems': 50,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use with Apify MCP

Add this Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/indonesian-news-aggregation-feed"
```

#### Claude Desktop, Cursor, and VS Code setup

Use the equivalent HTTP MCP configuration in Claude Desktop, Cursor, or VS Code:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/indonesian-news-aggregation-feed"
    }
  }
}
```

Example prompts:

- “Run Indonesian News Scraper for `kecerdasan buatan` across Kompas and Detik, with at most 30 articles.”
- “Collect 15 Detik records about `ekonomi Indonesia` and summarize the recurring themes.”
- “Build a two-topic feed for `kebijakan pemerintah` and `energi terbarukan`, then group links by publisher.”

### Limits and freshness

- Coverage is limited to public Kompas and Detik topic-search results.
- Search ranking, retention, and available pagination are controlled by each publisher.
- Kompas may expose a date without an exact time; such dates are normalized to midnight UTC and the original text is preserved.
- Detik usually exposes a machine-readable timestamp, but fields can still be absent.
- `summary` is a search excerpt, not the full article body.
- The Actor stops at 1000 accepted records per run.
- Public result pages can change. An unrecognized page shape fails visibly instead of silently returning a misleading empty dataset.
- The Actor retries transient network, HTTP 429, and server failures up to a bounded limit. It does not blindly retry deterministic client errors.

### Responsible use and legality

This Actor collects publicly displayed news metadata. Users remain responsible for complying with publisher terms, copyright rules, robots guidance, privacy law, and the requirements of their jurisdiction.

Do not republish copyrighted article text or images without permission. Prefer linking to the canonical article, retaining publisher attribution, and using excerpts only for legitimate monitoring, research, or analysis. Avoid collecting or using personal data in ways that create harm.

### Troubleshooting

#### The dataset is empty

Verify the spelling and specificity of the topic. A valid topic can naturally have no current indexed results. Try a broader Indonesian phrase and confirm at least one publisher is selected.

#### The run reports an unrecognized result page

A publisher may have changed its HTML or returned a challenge page. Retry later once. If the same error persists, keep the run ID and logs when reporting the problem; repeated identical retries will not repair a changed parser.

#### Fewer records were returned than `maxItems`

`maxItems` is a ceiling, not padding. The selected publishers may expose fewer unique matching URLs, and duplicates are removed.

#### Why are some publication times midnight UTC?

Kompas search results sometimes expose only an Indonesian calendar date. `publishedAt` uses midnight UTC as a machine-readable date while `publishedText` preserves the exact displayed value.

#### Does the Actor use a proxy?

No proxy is enabled by default because both selected public search routes work over direct HTTP. This keeps runs lightweight and avoids unmeasured proxy costs.

### Related Automation Lab actors

- [Naver News Search Scraper](https://apify.com/automation-lab/naver-news-search-scraper) for Korean news discovery by keyword.
- [Brave News Search Scraper](https://apify.com/automation-lab/brave-news-search-scraper) for broader web news-search workflows.
- [Seeking Alpha Analysis Feed Scraper](https://apify.com/automation-lab/seeking-alpha-analysis-feed-scraper) for market-analysis feed collection.

Use those Actors when the buyer job or source differs. They are not silent fallbacks for Kompas or Detik.

### FAQ

#### Does it scrape full articles?

No. It returns public search metadata and available excerpts, not complete article bodies.

#### Can I select only Kompas or Detik?

Yes. Set `publishers` to `['kompas']` or `['detik']`.

#### Can I monitor more than one topic?

Yes. Supply up to 20 topics. The Actor distributes available output capacity across them.

#### Are results translated into English?

No. Headlines and summaries remain in the language exposed by the publisher.

#### Does it perform sentiment analysis?

No. The normalized records are suitable input for your own sentiment, clustering, or language-model workflow, but this Actor does not claim those enrichments.

#### How should I deduplicate scheduled runs?

Use `articleUrl` as the primary key. Keep `query` and `fetchedAt` to retain matching and collection provenance.

# Actor input Schema

## `queries` (type: `array`):

Indonesian keywords, brands, people, or issues to monitor across the selected publishers.

## `publishers` (type: `array`):

Choose Kompas, Detik, or both. Results use the same normalized output fields.

## `maxItems` (type: `integer`):

Maximum unique article records saved across all topics and publishers.

## Actor input object example

```json
{
  "queries": [
    "kecerdasan buatan",
    "ekonomi Indonesia"
  ],
  "publishers": [
    "kompas",
    "detik"
  ],
  "maxItems": 20
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing all matched article records.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "kecerdasan buatan",
        "ekonomi Indonesia"
    ],
    "publishers": [
        "kompas",
        "detik"
    ],
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/indonesian-news-aggregation-feed").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "kecerdasan buatan",
        "ekonomi Indonesia",
    ],
    "publishers": [
        "kompas",
        "detik",
    ],
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/indonesian-news-aggregation-feed").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "kecerdasan buatan",
    "ekonomi Indonesia"
  ],
  "publishers": [
    "kompas",
    "detik"
  ],
  "maxItems": 20
}' |
apify call automation-lab/indonesian-news-aggregation-feed --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/indonesian-news-aggregation-feed"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/w2XIyh4aKW1NrhLqB/builds/1G5fcmvGM9bMvpoYK/openapi.json