# The Hindu News - Full Articles, Images & Comments (`ryyos/thehindu-news-scraper`) Actor \[$0.8/1000] Pull fully enriched The Hindu articles - every section, full body text, author, publish/update times, every image URL ready to download, tags, and reader comments (author + text). - **URL**: https://apify.com/ryyos/thehindu-news-scraper.md - **Developed by:** [Rio Dwi Saputra](https://apify.com/ryyos) (community) - **Categories:** News, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.80 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### What does The Hindu News Scraper do? **The Hindu News Scraper** extracts fully enriched articles from [thehindu.com](https://www.thehindu.com), one of India's largest English-language newspapers. Point it at any combination of 35 sections - India, World, Opinion/Editorial, Business, Cricket and every other sport, Sci-Tech, Entertainment, Education, Life & Style, Society and more - and it returns the full article: headline, byline or wire-agency credit, publish/update timestamps, complete body text, every image URL ready to download, topic tags, and reader comments (author username and full text, not just a count). Run it on demand, on a schedule, or wire it into a pipeline via the Apify API - proxy rotation, monitoring and storage are handled by the platform. ### Why use The Hindu News Scraper? - **Media monitoring** - track coverage of a company, policy, or public figure across India's national, state and business desks. - **Research & academia** - build a clean, structured corpus of Indian news and public opinion (editorials, letters, op-eds) without manual copy-pasting. - **Sentiment & discussion analysis** - reader comments are pulled with real usernames and full text, not just totals, so downstream NLP can work with actual language. - **Content aggregation** - repurpose or archive full articles, including every image, into your own systems. ### How to use The Hindu News Scraper 1. Click **Try for free** (or **Start**) on the Actor page. 2. On the **Input** tab, pick the sections you care about, or leave the default to cover the whole site. 3. Set `maxItemsPerSection` to control cost, and toggle `includeComments` off if you only need article text. 4. Click **Start** and watch the run in real time. 5. When it finishes, open the **Output** tab (or the API) to download your data as JSON, CSV, Excel, HTML, or RSS. ### Input | Field | Type | Description | |---|---|---| | `sections` | array | Section slugs to crawl (e.g. `national`, `sport-cricket`, `opinion-editorial`). Defaults to all 35. | | `maxItemsPerSection` | integer | Max articles to fetch per section per run. Default `20`. | | `maxPagesPerSection` | integer | Max section-listing pages to paginate through (~50 links each). Default `3`. | | `includeComments` | boolean | Fetch reader comments (author + text) per article. Default `true`. | | `proxyConfiguration` | object | Apify Proxy settings. Off by default - the site has shown no anti-bot blocking. | See the Input tab for the full schema with all section options. ### Output Each dataset item is one article: ```json { "section": "national", "articleId": "70224146", "headline": "Trump says it is time to get rid of U.S. Senate's filibuster custom", "summary": "Trump calls for abolishing the Senate filibuster to overcome Democratic obstacles amid a prolonged government shutdown.", "url": "https://www.thehindu.com/news/international/trump-says-it-is-time-to-get-rid-of-us-senates-filibuster-custom/article70224146.ece", "authorName": "Reuters", "publishedAt": "2025-10-31T08:54:41.000+05:30", "updatedAt": "2026-08-18T13:49:41.000+05:30", "tags": ["donald-trump"], "bodyText": "Republican U.S. President Donald Trump said on Thursday...", "heroImageUrl": "https://th-i.thgim.com/public/incoming/.../LANDSCAPE_1200/....JPG", "images": [{ "url": "https://th-i.thgim.com/...jpg", "caption": null }], "videos": [], "commentCount": 4, "comments": [ { "author": "Rajveer", "text": "Understood the whole idea...", "createdAt": "2026-08-10T12:56:00.751Z", "reactionCount": 0, "isReply": false } ], "scrapedAt": "2026-08-18T14:00:00.000Z" } ``` You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. ### Data table | Field | Description | |---|---| | `headline`, `summary`, `url` | Core article identity | | `authorName` | Byline, or wire-agency credit (Reuters/PTI/AFP) | | `publishedAt` / `updatedAt` | ISO-8601 timestamps | | `tags` | Topic tags | | `bodyHtml` / `bodyText` | Full article body | | `heroImageUrl` / `images` | Every directly downloadable image URL | | `videos` | Embedded video URLs, when present | | `commentCount` / `comments` | Reader discussion, with author username and full text | ### Pricing / Cost estimation This Actor uses the Pay-per-result pricing model at **$0.8 per 1,000 articles**. A run covering all 35 default sections at 20 articles each (700 articles) costs roughly $0.56. Turning off `includeComments` reduces the number of requests per article and speeds up large runs. ### Tips or Advanced options - Narrow `sections` to just what you need (e.g. `["sport-cricket"]`) to keep runs fast and cheap. - Raise `maxPagesPerSection` if you need deeper history within a section rather than just the latest items. - Leave Apify Proxy off unless you start seeing failures - thehindu.com's article and section pages are plain server-rendered HTML with no anti-bot challenge observed during development. ### FAQ, disclaimers, and support This Actor only reads publicly accessible pages and a public, anonymous comment API - no login, paywall bypass, or private data is involved. It is intended for research, monitoring, and archival use; respect thehindu.com's Terms of Service for your use case. Data field availability can vary by article (e.g. wire-service pieces may not have comments enabled). Found a bug or need a custom field? Use the Actor's **Issues** tab - custom modifications are also available on request. # Actor input Schema ## `sections` (type: `array`): Which The Hindu sections to crawl. Leave at the default to cover every major section on the site. ## `maxItemsPerSection` (type: `integer`): Maximum number of articles to fetch full detail for, per section, on each run. ## `maxPagesPerSection` (type: `integer`): How many section-listing pages to paginate through (each page has ~50 article links) before stopping, in addition to the max articles cap. ## `includeComments` (type: `boolean`): Fetch reader comments (author username + full text, not just counts) for each article via The Hindu's Coral comment platform. Adds one extra request per article. ## `proxyConfiguration` (type: `object`): thehindu.com has not shown anti-bot blocking during testing (plain server-rendered HTML, public comment API), so this defaults off. Switch on Apify Proxy if you see failures. ## Actor input object example ```json { "sections": [ "national", "international", "states", "cities", "opinion-editorial", "opinion-columns", "opinion-comment", "opinion-interview", "opinion-lead", "opinion-letters", "opinion-open-page", "business", "business-agri", "business-markets", "business-budget", "sport-cricket", "sport-football", "sport-hockey", "sport-tennis", "sport-athletics", "sport-motorsport", "sport-other", "health", "science", "technology", "entertainment-movies", "entertainment-music", "entertainment-art", "entertainment-theatre", "entertainment-dance", "entertainment-reviews", "education", "life-and-style", "society", "data" ], "maxItemsPerSection": 20, "maxPagesPerSection": 3, "includeComments": true, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): No description ## `runInfo` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("ryyos/thehindu-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("ryyos/thehindu-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call ryyos/thehindu-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,ryyos/thehindu-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/gpvU3rruvyGXgQLE5/builds/wM2KZSWROZvGzrOUj/openapi.json