Indonews Aggregator Scraper

📰 Indonews Aggregator Scraper – Indonesian News Data Extraction by Keyword & Date

The Indonews Aggregator Scraper collects Indonesian news articles by keyword and date range, returning each story as a structured record with headline, publisher, publication timestamp, article URL, lead image, extracted content, topical category, and pre-computed sentiment and emotion labels. It is built for media monitoring teams, political analysts, brand-reputation managers and researchers who need Indonesian news coverage in a machine-readable form rather than as a browser tab full of open articles.

Indonesian media is fragmented across dozens of national outlets, regional publishers and online-only newsrooms. Tracking a person, company or policy across that landscape by hand is impractical. This Indonesian news scraper aggregates coverage into a single dataset, so you can filter by publisher, chart sentiment over time, or feed the text into your own NLP pipeline.

Each run queries a news aggregation feed for your keyword, applies your date window and sort preference, and writes one dataset item per article. Results include a sentiment and emotion classification alongside the article body, which means basic media-tone analysis is available without a separate enrichment step.

---

📊 What Data Can You Extract with This Indonesian News Scraper?

Every dataset item represents a single news article. The nine output fields group into five useful categories:

Category	Fields	What you get
📝 Headline & body	title, content	The article headline as published, plus the extracted textual content of the story
🏢 Publisher	media_name	The name of the outlet that published the article, for example Kompas.com
🕒 Timing	published_at	The publication timestamp in ISO-style format, suitable for time-series grouping
🔗 Links & media	url, image	The canonical article URL and the lead image URL associated with the story
🎭 Tone & topic	sentiment, emotion, category	A sentiment label, an emotion label, and a topical category such as Government

The genuinely distinctive field here is sentiment arriving pre-computed alongside media_name. Because tone and publisher land in the same record, you can immediately answer a question that normally requires a whole analysis pipeline: which outlets are covering this topic favourably, and which are not.

---

🌟 Key Features of the Indonesian News Scraper

Feature	Description
🔍 Keyword-driven search	Supply any search phrase — a politician's name, a company, a policy term — and the scraper returns matching Indonesian news coverage
📅 Date-range filtering	Optional startDate and endDate fields scope results to a specific window, so you can isolate coverage around an event
🔀 Sort by time or relevance	Choose Time for a chronological feed or Relevance to surface the most on-topic articles first
📄 Pagination control	itemsPerPage and pageNumber let you page through larger result sets or pull a precise slice
🎭 Built-in sentiment labels	Each article arrives with a sentiment value, removing the need for a separate classification pass for basic tone tracking
😀 Emotion classification	An additional emotion field gives a second, finer-grained read on article tone beyond positive/negative
🗂️ Topical categorisation	The category field assigns each article a subject area such as Government, useful for segmenting a large corpus
🖼️ Lead image URLs	The image field supplies a direct link to the article's headline image for dashboards and monitoring reports
⚡ No browser required	The Actor runs on HTTP requests and HTML parsing rather than a headless browser, keeping runs light and fast

---

🚀 Why Choose This Indonesian News Scraper?

Sentiment and emotion arrive with the article, not after it. Most news scrapers hand you raw text and leave classification as an exercise. Here sentiment, emotion and category are populated in the same record as title and content, so a media-tone dashboard is a pivot table away rather than a machine-learning project.

Precise event windows. The startDate and endDate inputs let you carve out exactly the period that matters — the week around an election, the days following a product recall, a quarter of policy debate. Combined with sortBy: "Time", this produces a clean chronological narrative of how a story developed.

Publisher-level attribution. Every article carries media_name, which turns an undifferentiated pile of coverage into an analysable dataset. You can count articles per outlet, compare average sentiment between publishers, or filter to a specific set of national titles.

Simple, predictable input. Only keywords is required. Everything else has a sensible default — 50 items per page, page 1, sorted by time. You can get a useful first dataset with a single field filled in, then refine.

---

📥 Input

The Indonesian news scraper needs one required field, keywords. All other fields are optional refinements.

{
  "keywords": "prabowo subianto",
  "startDate": "2026-08-01",
  "endDate": "2026-08-15",
  "itemsPerPage": 50,
  "pageNumber": 1,
  "sortBy": "Time"
}

🔧 Indonesian News Scraper Input Fields

Field	Type	Required	Default	Description
keywords	string	✅ Yes	prabowo subianto	Enter the keyword and date range to fetch news data
startDate	string	No	—	Start of the date window, in YYYY-MM-DD format
endDate	string	No	—	End of the date window, in YYYY-MM-DD format
itemsPerPage	integer	No	50	Number of items to fetch
pageNumber	integer	No	1	Start page
sortBy	string (enum)	No	Time	Sort order. Accepts Time or Relevance

💡 Input Examples

Minimal run — keyword only:

{
  "keywords": "bank indonesia"
}

Event window, most relevant first:

{
  "keywords": "ibu kota nusantara",
  "startDate": "2026-07-01",
  "endDate": "2026-07-31",
  "sortBy": "Relevance",
  "itemsPerPage": 100
}

Paging deeper into a large result set:

{
  "keywords": "kurs rupiah",
  "itemsPerPage": 50,
  "pageNumber": 3,
  "sortBy": "Time"
}

---

📤 Output

Each article becomes one dataset item. Below is a real record from an actual run of the Indonews Aggregator Scraper (long URL and content values trimmed for readability):

{
  "title": "Prabowo Ajukan Destry Damayanti Jadi Gubernur BI ke DPR RI - Kompas.com",
  "media_name": "Kompas.com",
  "published_at": "2026-08-10T06:08:00",
  "url": "https://news.google.com/rss/articles/CBMisAFBVV95cUxQQ2QtNWdkOEd6NS1lbDFoTTNTVnBmUncxTUxYTzY4ckhCOX…",
  "image": "https://lh3.googleusercontent.com/J6_coFbogxhRI9iM864NL_liGXvsQp2AupsKei7z0cNNfDvGUmWUy20nuUhkREQyr…",
  "emotion": "Neutral",
  "sentiment": "Neutral",
  "category": "Government",
  "content": "<a href=\"https://news.google.com/rss/articles/CBMisAFBVV95cUxQQ2QtNWdkOEd6NS1lbDFoTTNTVnBmUncxTUxY…"
}

🧾 Indonesian News Output Fields

Field	Type	Description
title	string | null	Title of the item
media_name	string | null	Name of the media
published_at	string | null	When the item was published
url	string | null	Canonical URL of the scraped item
image	string | null	Image URL
emotion	string | null	Emotion of the item
sentiment	string | null	Sentiment of the item
category	string | null	Category of the item
content	string | null	Extracted textual content

The content field can contain HTML markup as delivered by the source feed. Strip tags in your pipeline if you need plain text for NLP work. All fields are nullable — an article without a lead image will return null for image rather than omitting the key.

---

💻 How to Use the Indonesian News Scraper (Step by Step)

Step 1: Open the Indonews Aggregator Scraper on Apify

Sign in to your Apify account and open the Actor page, then click Start to reveal the input form. If you are new to Apify, registration takes a minute and gives you a console where every run, log and dataset is retained. The console is also where scheduling, integrations and API tokens live, so it is worth a quick tour before your first serious run.

Step 2: Define your search keyword

The keywords field drives everything. Be deliberate here: a person's full name will pull coverage about that individual, while a broad topical term like ekonomi will return a much larger and noisier set. For monitoring a specific entity, use the form the Indonesian press actually prints — full names, official company names, or the common abbreviation if that is what outlets use. Overly narrow phrasing can miss coverage that refers to the subject differently.

Step 3: Set your date window

startDate and endDate both accept YYYY-MM-DD. Leaving them empty returns whatever the feed offers by default, which is usually recent coverage. Setting an explicit window is strongly recommended for any analytical use, because it makes your dataset reproducible: rerunning the same window later returns a comparable set rather than a moving target. For event analysis, extend the window a few days either side of the event itself to capture build-up and aftermath.

Step 4: Choose a sort order

sortBy accepts Time or Relevance. Use Time when you are building a chronology of how a story broke and developed — the resulting published_at sequence reads as a narrative. Use Relevance when your keyword is ambiguous or generic and you want the strongest matches at the top rather than a chronological mix of tangential mentions.

Step 5: Tune pagination for the volume you need

itemsPerPage defaults to 50 and pageNumber to 1. For a quick look, the defaults are fine. For a corpus, raise itemsPerPage or run several times incrementing pageNumber to walk deeper into the result set. Keep track of which pages you have already collected so you do not duplicate articles when you combine datasets — deduplicating on url afterwards is a reliable safety net.

Step 6: Run and review the dataset

Start the run and watch the log. When it completes, open the Output tab to see one row per article. Spot-check a few records: confirm the media_name values look like Indonesian outlets you recognise, that published_at falls inside your requested window, and that sentiment is populated. This two-minute check catches keyword mistakes before you build analysis on top of the data.

Step 7: Export or schedule for continuous monitoring

Export to JSON, CSV, Excel, XML or HTML from the dataset view. For ongoing media monitoring, set up a schedule in the Apify console so the same keyword runs daily or hourly, then append each run to a growing archive. Deduplicate on url as you append, and you have a continuously updating Indonesian news corpus with sentiment already attached.

---

🔌 API Access & Integrations

Trigger the Indonesian news scraper through the Apify API and receive dataset items in one synchronous request:

curl -X POST "https://api.apify.com/v2/acts/scrapers-hub~indonews-aggregator/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "keywords": "prabowo subianto",
    "startDate": "2026-08-01",
    "endDate": "2026-08-15",
    "itemsPerPage": 50,
    "pageNumber": 1,
    "sortBy": "Time"
  }'

The equivalent using the official Python client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")

run_input = {
    "keywords": "bank indonesia",
    "startDate": "2026-08-01",
    "endDate": "2026-08-15",
    "itemsPerPage": 50,
    "sortBy": "Relevance",
}

run = client.actor("scrapers-hub/indonews-aggregator").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["published_at"], item["media_name"], item["sentiment"], item["title"])

The Actor also connects to Zapier, Make, Google Sheets, Slack, Airtable and any custom endpoint through Apify webhooks — useful for posting new negative-sentiment articles into a monitoring channel the moment a run finishes.

---

💡 Best Use Cases for Indonesian News Data

🏛️ Political and Policy Monitoring

Track a minister, party or draft regulation by setting keywords to the relevant name or term and sortBy to Time. The combination of published_at, media_name and sentiment produces a day-by-day picture of how coverage tone shifted and which outlets drove it. The category field, which returns values such as Government, helps separate genuine policy reporting from incidental mentions.

🏢 Brand Reputation Management

Set keywords to your company or product name and run daily. Charting sentiment over published_at gives an early-warning signal: a cluster of negative-sentiment articles from major outlets is visible in the dataset before it becomes a full crisis. The url field lets your comms team jump straight to the source article for verification.

📈 Financial and Market Research

Terms like a listed company's name, a commodity, or kurs rupiah return coverage relevant to market sentiment. Analysts combine published_at with sentiment to build a news-tone series that can be aligned against price data, while content supplies the raw text for more sophisticated extraction of guidance, figures and named entities.

🎓 Academic and Media Studies Research

Researchers studying Indonesian media framing can assemble a corpus by running a fixed keyword across a defined date window, then analyse the distribution of media_name, category, emotion and sentiment. Because the date window makes runs reproducible, the resulting dataset is citable and another researcher can reconstruct it.

🚨 Crisis and Issue Tracking

During a developing incident, narrow startDate and endDate to the incident window and set itemsPerPage high. The chronological published_at sequence plus title reconstructs the timeline of what was reported when, and emotion highlights which coverage is escalating in tone rather than merely reporting facts.

📊 Competitive Intelligence

Run the scraper against a competitor's name and compare the resulting media_name distribution and sentiment mix against your own. This shows not just how much coverage a rival is getting, but where — a competitor consistently appearing in a particular set of outlets tells you something about their PR strategy that raw volume does not.

🔤 NLP Corpus Building

The content field supplies Indonesian-language article text at volume, which is genuinely scarce for anyone training or fine-tuning language models. Pairing content with the existing category and sentiment labels gives you a partially labelled corpus suitable for classification experiments straight out of the box.

---

⚙️ Tips for Better Indonesian News Scraping Results

• Always set an explicit date window for analysis. Without startDate and endDate, results shift every time you run, which makes comparisons unreliable. A fixed window makes a run reproducible.
• Use the phrasing Indonesian outlets actually use. Search coverage of a company under its registered Indonesian name as well as any English form — the two often return different article sets worth merging.
• Deduplicate on url when combining runs. Paging through results or scheduling repeated runs will produce overlap. The url field is the most reliable unique key.
• Strip HTML from content before text analysis. The field can carry markup from the source feed. A single cleaning step at ingestion saves trouble in every downstream consumer.
• Sort by Relevance for ambiguous keywords. If your search term is also a common word, Time will fill the dataset with tangential mentions. Relevance puts genuine matches first.
• Raise itemsPerPage before increasing pageNumber. Fetching more per page in a single run is simpler to manage than reconciling several paged runs.

---

🛠️ Troubleshooting

Why did my run return no articles? The most common cause is a date window with no coverage, or a keyword too specific to match any headline. Remove startDate and endDate and rerun to confirm the keyword itself returns results, then reintroduce the window.

Why does content contain HTML tags? The content is captured as the source feed supplies it, which sometimes includes anchor tags and markup. Run it through an HTML stripper such as BeautifulSoup's get_text() if you need clean prose.

Why is image sometimes null? Not every article publishes a lead image, and not every feed entry exposes one. Null is the honest representation of "no image available" — handle it explicitly rather than assuming a value is always present.

The url values look like aggregator redirect links rather than the publisher's domain. Article URLs are captured as the aggregation feed provides them, which can include redirect wrappers. Following the link in a browser resolves to the publisher's page. If you need the final destination programmatically, resolve redirects in your own pipeline.

Why do I see the same article twice across two runs? Paged and scheduled runs naturally overlap. Deduplicate on url when appending results to a combined dataset.

---

❓ Frequently Asked Questions About Indonesian News Scraping

What does the Indonews Aggregator Scraper do? It searches Indonesian news coverage for a keyword you supply and returns each matching article as a structured record containing headline, publisher, publication time, URL, image, content, category, sentiment and emotion.

What is the only required input? keywords. Everything else — date range, pagination and sort order — is optional and has a default.

Can I restrict results to a specific date range? Yes. Set startDate and endDate in YYYY-MM-DD format. This is the recommended approach for any analysis you intend to repeat or cite.

How many articles can I get in one run? That is governed by itemsPerPage, which defaults to 50. Increase it for a larger single-run dataset, or step pageNumber through successive pages.

What is the difference between sentiment and emotion? sentiment is a broad tone label for the article, while emotion provides a separate, finer-grained read. Both are returned per article, and both may be null if the source does not classify a given item.

Can I search in English instead of Indonesian? You can supply any keyword string, but coverage returned is Indonesian news. English-language search terms will match articles that contain them; for reliable recall, use the Indonesian phrasing the local press uses.

Does the Indonesian news scraper need proxy configuration? No. There is no proxy setting in the input and none is required.

What does the category field contain? A topical classification of the article, such as Government. It is supplied per article and is useful for segmenting large result sets by subject area.

How do I get the full text of an article? The content field carries the extracted textual content as delivered by the source. Note that it may include HTML markup and, depending on the publisher, may be a summary rather than the complete body.

What export formats are supported? JSON, CSV, Excel, XML, HTML table and RSS, available from the dataset view or through the Apify API.

Can I schedule this Indonesian news scraper to run automatically? Yes. Use the Schedules feature in the Apify console to run the same keyword configuration on a recurring basis, then append each run to a growing archive.

How is published_at formatted? As an ISO-style timestamp, for example 2026-08-10T06:08:00, which sorts and groups correctly without reformatting in most tools.

Can I search multiple keywords at once? The keywords field takes a single string. Run the scraper once per keyword and merge the datasets if you need to monitor several terms.

Is the sentiment classification always accurate? Sentiment and emotion labels are supplied by the source classification and, like any automated tone analysis, should be treated as indicative rather than definitive. For high-stakes conclusions, sample and verify against the content and title.

Is scraping Indonesian news data legal? The Actor collects publicly published news content. You are responsible for complying with copyright law, the publishers' terms, and applicable data protection rules in how you store, analyse and redistribute the data.

---

🆘 Support & Feedback

Hit an error, spotted a field returning unexpected values, or found a keyword that behaves strangely? Report it in the Issues tab on the Actor page so it can be tracked and fixed.

Need a custom build — additional fields, a different source set, a tailored output shape, or a scraper for another market — email scraperhubapi@gmail.com with the details.

If the Indonews Aggregator Scraper is useful to you, a review on the Apify Store genuinely helps. Feedback from real users shapes what gets improved next.

---

⚖️ Disclaimer

The Indonews Aggregator Scraper collects only publicly available Indonesian news content — headlines, article text, publication metadata and images that any visitor can access without authentication. It does not bypass paywalls, access subscriber-only material, or retrieve private data.

You are responsible for how you use the collected data. News articles are protected by copyright and remain the property of their publishers. Extracting content for internal analysis, monitoring or research is generally accepted practice, but republishing full article text may require permission from the rights holder. Review the terms of the publishers whose content appears in your dataset.

Where articles mention identifiable individuals — politicians, executives, private persons named in reporting — the resulting dataset contains personal data, and GDPR, Indonesia's Personal Data Protection Law and equivalent frameworks may apply to your processing of it. Ensure you have a lawful basis, retain data only as long as necessary, and apply appropriate safeguards.

This Actor is not affiliated with, endorsed by, or sponsored by any news publisher or aggregation platform. If you believe data collected by this Indonesian news scraper relates to you and you would like it removed, contact scraperhubapi@gmail.com and the request will be handled promptly.