# Kompas News Scraper (`confidential_gnat/kompas-news-scraper`) Actor Scrapes news articles from Kompas.com, including headline, author, editor, publish date, category, images and full article text. - **URL**: https://apify.com/confidential_gnat/kompas-news-scraper.md - **Developed by:** [ActorFlow](https://apify.com/confidential_gnat) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Kompas.com News Scraper - Indonesian News Articles & Headlines **Scrape news articles from Kompas.com (kompas.com)**, one of Indonesia's leading news portals: headline, summary, author, editor, publish and update time, category, keywords, images and the **full article text**. Export the results as **JSON, CSV or Excel**, or call the actor as an **API**. Paste a Kompas listing or article URL, press Start, and get a clean **Indonesian news dataset**. **Target website:** [kompas.com](https://www.kompas.com) ### ✨ Features of this Kompas news scraper - **Article extraction** — headline, summary, authors, editor, publish and update timestamps, category, section, keywords, images, full body text and word count. - **Full multi-page articles** — long stories split across several pages are fetched complete, in one record. - **Pagination support** — walks the listing pages automatically until Max items per start URL is reached. - **Automatic URL detection** — listing pages and article URLs are told apart automatically, so you can mix both in Start URLs. - **Whole-site mode** — one toggle crawls the index of every major Kompas channel instead of a hand-picked list. - **Optional sentiment analysis** — switch on **Analyze sentiment** to label every article positive, negative, neutral or mixed, with a confidence score. Off by default. - **Incremental runs** — with a cache project name, articles you already scraped are skipped on later runs. - **Proxy support** — optional Apify Proxy; Kompas normally works without one. - **No browser required** — plain HTTP requests keep runs fast and cheap. ### 🚀 How to scrape Kompas news in 5 steps 1. [Sign up](https://apify.com/sign-up) for a free Apify account — includes **$5 monthly credit**. 2. Open the actor page and click **Try for free**. 3. Fill in the **Input** fields (at least one Kompas start URL is required). 4. Click **Start** and wait for the run to complete. 5. Download results from the **Output** tab in JSON, CSV, or Excel format. You can also run this actor via the [Apify API](https://docs.apify.com/api/v2) or integrate it directly into your workflows using [Zapier](https://zapier.com/apps/apify), [Make](https://www.make.com/), or [n8n](https://n8n.io/). ### 💰 Kompas scraper pricing This actor uses **pay-per-result** billing: you pay for each article saved to the dataset. - New Apify accounts include **$5 of free monthly credit**. - The actor uses plain HTTP requests, so it costs significantly less to run than browser-based scrapers. - Articles skipped by the cache are not saved and not charged. - Kompas works without a proxy, so switching Apify Proxy off keeps runs cheapest. ### 🔧 Kompas scraper input configuration | Field | Type | Required | Default | Description | | ----- | ---- | -------- | ------- | ----------- | | Start URLs (`startUrls`) | array | ✅ | `kolom.kompas.com` (prefill) | Kompas listing URLs (`indeks.kompas.com` or channel front pages) or direct article URLs. The type of each URL is detected automatically. | | Max items per start URL (`maxItems`) | integer | — | `5` | Maximum number of articles to scrape for each start URL. | | Cache project name (`cacheProjectName`) | string | — | — | Use the same name on each run to skip articles already scraped under it. | | Scrape the whole site (`scrapeWholeSite`) | boolean | — | `false` | Ignore Start URLs and crawl the index pages of every major Kompas channel. | | Analyze sentiment (`analyzeSentiment`) | boolean | — | `false` | Classify each article as positive, negative, neutral or mixed. Adds `sentiment`, `sentimentConfidence` and `sentimentScores`. | | Proxy configuration (`proxyConfiguration`) | object | — | `{"useApifyProxy": true}` | Kompas responds without a proxy. Try datacenter proxies first if you get blocked, residential only if that still fails. | ![Kompas news scraper input form with the Start URLs, Max items per start URL, Cache project name and Scrape the whole site fields marked 1 to 4](https://i.postimg.cc/YCj0Kddy/image.png) The numbered markers in the screenshot show the four fields most runs use: 1. **Start URLs** — a Kompas listing page (for example a channel front page) or a direct article URL. 2. **Max items per start URL** — how many articles to save for each URL. 3. **Cache project name** — optional; reuse the same name on every scheduled run to get only new articles. 4. **Scrape the whole site** — tick it to crawl every major channel and ignore Start URLs. **Supported URL types:** - **Listing pages** — the index at `https://indeks.kompas.com/` and channel front pages such as `https://tekno.kompas.com/`. - **Article pages** — any URL under `/read/{yyyy}/{mm}/{dd}/`, for example `https://edukasi.kompas.com/read/2026/09/10/091950471/cerita-mendikti`. ### 📦 Kompas scraper output data Each article is one dataset item with the fields `url`, `title`, `description`, `authors`, `editor`, `publisher`, `publishedAt`, `modifiedAt`, `category`, `section`, `keywords`, `images`, `body` and `wordCount`, plus `sentiment`, `sentimentConfidence` and `sentimentScores` when **Analyze sentiment** is on. The dataset has **1 view**, **Overview**, a table of title, category, authors, publish date, sentiment, word count and article link. The sentiment column is empty unless **Analyze sentiment** is on. With **Analyze sentiment** on, each article gets a `sentiment` (`positive`, `negative`, `neutral` or `mixed`), a `sentimentConfidence` between 0 and 1, and `sentimentScores` with the probability of every label, based on the headline, summary and the start of the article text: ```json { "sentiment": "negative", "sentimentConfidence": 0.92, "sentimentScores": { "positive": 0, "negative": 0.94, "neutral": 0.06, "mixed": 0 } } ``` **Output at a glance:** | Title | Category | Authors | Published | Words | | ----- | -------- | ------- | --------- | ----- | | [Demokrasi Tanpa "Delivery"](http://nasional.kompas.com/read/2026/09/10/15350031/demokrasi-tanpa-delivery) | Nasional | Raisa Ayu Rininta | 2026-09-10 | 869 | | [Mengenang Airlangga Pribadi, Intelektual Penjaga Nalar](http://nasional.kompas.com/read/2026/09/10/16330061/mengenang-airlangga-pribadi-intelektual-penjaga-nalar) | Nasional | Aji Cahyono | 2026-09-10 | 1,567 | | [Bangun Bursa Mineral dan Komoditi Strategis yang Kredibel](http://money.kompas.com/read/2026/09/10/160600726/bangun-bursa-mineral-dan-komoditi-strategis-yang-kredibel) | Ekbis | Gusmardi Bustami | 2026-09-10 | 1,143 | **Sample output (raw JSON):** ```json { "url": "https://otomotif.kompas.com/read/2026/09/10/100200315/tren-modifikasi-motor-sport--fokus-suara-knalpot-dan-baut-titanium?page=all", "title": "Tren Modifikasi Motor Sport: Fokus Suara Knalpot dan Baut Titanium", "description": "Tren modifikasi motor sport masih didominasi penggantian knalpot full system demi suara maksimal, disusul penggunaan baut titanium", "authors": [ "Muhammad Fathan Radityasani" ], "publisher": "Kompas.com", "publishedAt": "2026-09-10T10:02:00+07:00", "modifiedAt": "2026-09-10T10:02:00+07:00", "category": "Motor", "section": "Kompas.com", "keywords": [ "modifikasi", "motor sport", "Modifikasi motor", "tren modifikasi", "modifikasi motor sport" ], "editor": "Agung Kurniawan", "images": [ "https://asset.kompas.com/crops/Ss-MNDULXxicH4_TnQrrne7Pyj8=/0x21:1518x1033/1200x675/filters:watermark(data/photo/2026/01/30/697c81750a668.png,0,-0,1)/data/photo/2022/12/20/63a0ee275c869.jpg", "https://asset.kompas.com/crops/XVemya23I-AMe_pLdzT99fcbkqA=/0x21:1518x1033/1200x900/data/photo/2022/12/20/63a0ee275c869.jpg" ], "body": "JAKARTA, KOMPAS.com - Populasi motor sport saat ini memang tidak sebanyak skutik di jalanan. Meski demikian, performa mumpuni serta karakter berkendara yang agresif membuat motor jenis ini tetap memiliki tempat tersendiri, khususnya sebagai kendaraan penyalur hobi.\n\nBerbeda dengan skutik yang opsi modifikasinya sangat beragam mulai dari peningkatan performa hingga pemanis tampilan, pemilik motor s …", "wordCount": 403 } ``` ### 🐍 How to scrape Kompas with Python, JavaScript or the API Run the actor programmatically with the official Apify clients. Replace `` with the token from your [Apify Console](https://console.apify.com/account/integrations). **Python** (`pip install apify-client`): ```python from apify_client import ApifyClient client = ApifyClient("") run = client.actor("confidential_gnat/kompas-news-scraper").call(run_input={ "startUrls": [{"url": "https://indeks.kompas.com/"}], "maxItems": 20, }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ``` **JavaScript** (`npm install apify-client`): ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: '' }); const run = await client.actor('confidential_gnat/kompas-news-scraper').call({ startUrls: [{ url: 'https://indeks.kompas.com/' }], maxItems: 20, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` **cURL** — start a run and wait for the dataset: ```bash curl -X POST "https://api.apify.com/v2/acts/confidential_gnat~kompas-news-scraper/run-sync-get-dataset-items?token=" \ -H "Content-Type: application/json" \ -d '{"startUrls": [{"url": "https://indeks.kompas.com/"}], "maxItems": 20}' ``` ### 💡 What you can use Kompas news data for - Tracking how a topic, company or public figure is covered in Indonesian media over time - Building a searchable archive of Indonesian news articles - Training or evaluating Indonesian-language NLP and sentiment models - Competitive and brand monitoring across Kompas channels - Feeding dashboards, alerts and newsletters with fresh headlines Media monitoring teams, researchers, data scientists, PR and marketing agencies, and analysts covering Indonesia typically use this data. ### ⚠️ Kompas scraper limitations and known issues - **Live front pages change constantly** — running the same listing URL twice may return a different set of articles as new stories are published. ### ❓ Kompas scraper FAQ #### Can I scrape Kompas.com legally? The actor only collects data that is already publicly visible on Kompas.com: it does not log in, bypass a paywall or access private content. Scraping publicly available data is generally considered lawful (see *hiQ Labs v. LinkedIn*), but you are responsible for complying with Kompas's Terms of Service and applicable laws such as GDPR if article text or bylines contain personal data. #### What data does the Kompas news scraper return? Every article has its URL, headline, summary, authors, editor, publisher, publish and update time, category, section, keywords, images, full body text and word count. #### How many Kompas articles can I scrape from one URL? `maxItems` limits the number of articles **per start URL**, not for the whole run. On listing pages the actor keeps moving to the next page until that limit is reached. #### Do I need a proxy to scrape kompas.com? No. Kompas responds without a proxy, which keeps runs cheapest. If you start seeing blocks, enable datacenter proxies first and use residential proxies only if datacenter still fails. #### How do I scrape Kompas news with Python? Use the Python example above: call the actor with `ApifyClient`, then iterate over the dataset items of the finished run. #### Can I run this Kompas scraper on a schedule? Yes. Use Apify Schedules to run it daily or hourly. Set a **Cache project name** and every scheduled run saves only articles that earlier runs under that name have not already scraped. #### What formats can I export Kompas news in? JSON, CSV, Excel, XML and RSS, either from the **Output** tab or through the Apify API. #### Can I get the sentiment of Kompas articles? Yes. Switch on **Analyze sentiment** and every article is classified as positive, negative, neutral or mixed, with a confidence score and per-label probabilities. It is off by default and the four labels are fixed. #### Does the scraper get full articles that are split across several pages? Yes. Multi-page stories are requested in full, so each long article comes back as a single record with the complete text. ### 🔗 Other actors you may find useful | Scraper | Description | | ------- | ----------- | | [📰 Detik News AI Scraper](https://apify.com/confidential_gnat/detik-news-scraper) | Indonesian news from Detik.com: headline, author, publish date, category, images and full article text. | | [🗞️ Sydney Morning Herald (SMH) News Scraper](https://apify.com/confidential_gnat/smh-news-scraper) | News articles from smh.com.au: headline, author, publish date, section, keywords and full text. | | [🔎 Google News AI Scraper](https://apify.com/confidential_gnat/google-news-ai-scraper) | Google News by keyword, with optional full article text and AI-generated summaries. | | [💬 Sentiment Analysis Using Jev](https://apify.com/confidential_gnat/sentiment-analysis) | Classifies article text as positive, negative, neutral or mixed, or into your own custom labels. | | [✍️ Substack Scraper](https://apify.com/confidential_gnat/substack-newsletter-scraper) | Substack posts and comments by keyword, publication or post URL: title, author, date and full text. | ### 💬 Support & contact If you encounter any issues or have questions, please [open an issue](https://apify.com/confidential_gnat/kompas-news-scraper/issues/open) or reach out via the [Apify Community Forum](https://community.apify.com/). You can also find more of our actors on the [Actor Flow](https://apify.com/confidential_gnat) page. *** ### 🇮🇩 Scraper berita Kompas.com: panduan singkat (Bahasa Indonesia) Actor ini mengambil artikel berita dari **Kompas.com** secara otomatis. **Cara pakai:** 1. Isi **Start URLs** dengan halaman indeks (misalnya `https://indeks.kompas.com/`) atau langsung URL artikel. Jenis URL dideteksi otomatis. 2. Atur **Max items per start URL** untuk membatasi jumlah artikel per URL. 3. Aktifkan **Scrape the whole site** jika ingin menelusuri seluruh kanal utama Kompas. 4. Isi **Cache project name** (opsional) agar artikel yang sudah diambil pada run sebelumnya dilewati. 5. Klik **Start**, lalu unduh hasilnya dalam format JSON, CSV, atau Excel. **Data yang diambil:** judul, ringkasan, penulis, editor, tanggal terbit dan perubahan, kategori, kata kunci, gambar, serta isi lengkap artikel (termasuk artikel yang terbagi ke beberapa halaman). **Catatan:** actor ini hanya mengambil data yang memang tampil untuk publik di Kompas.com — tanpa login dan tanpa menembus konten berbayar. # Actor input Schema ## `startUrls` (type: `array`): Kompas listing URLs (indeks.kompas.com or channel front pages) or direct article URLs. The type of each URL is detected automatically. ## `maxItems` (type: `integer`): Maximum number of articles to scrape for each start URL. ## `cacheProjectName` (type: `string`): Optional. When set, articles already scraped under this project name in a previous run are skipped instead of being scraped again. ## `analyzeSentiment` (type: `boolean`): Classify each article as positive, negative, neutral or mixed, with a confidence score. Adds the fields sentiment, sentimentConfidence and sentimentScores to every article. ## `proxyConfiguration` (type: `object`): Kompas responds fine without a proxy. Enable datacenter proxies first if you start seeing blocks, and residential only if datacenter still fails. ## Actor input object example ```json { "startUrls": [ { "url": "https://kolom.kompas.com/?source=navbar" } ], "maxItems": 5, "analyzeSentiment": true, "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://kolom.kompas.com/?source=navbar" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("confidential_gnat/kompas-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://kolom.kompas.com/?source=navbar" }] } # Run the Actor and wait for it to finish run = client.actor("confidential_gnat/kompas-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://kolom.kompas.com/?source=navbar" } ] }' | apify call confidential_gnat/kompas-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,confidential_gnat/kompas-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/le7t4Rdz4eYhDmLHs/builds/TaxvdEDxvOD8fiPYH/openapi.json