# Hacker News Scraper: Stories, Comments & Search (`scrapemint/hacker-news-scraper`) Actor Search and scrape Hacker News: stories and comments by keyword, author, points, date or category (front page, Ask HN, Show HN). One clean row per story or comment with points, comment count, author and links. No API key. - **URL**: https://apify.com/scrapemint/hacker-news-scraper.md - **Developed by:** [Ken M](https://apify.com/scrapemint) (community) - **Categories:** News, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hacker News Scraper: Stories, Comments & Search Search and scrape [Hacker News](https://news.ycombinator.com) - stories and comments - by keyword, author, points, date or category. No API key, no login, no browser. Built on the official Algolia HN Search API. ### What you get **Stories** - one row each with title, external link, points, comment count, author, post text (for Ask/Show HN), tags, and the HN discussion link. **Comments** - one row each with the comment text, author, the story it belongs to (title and URL), and links. ### Ways to use it ```json { "queries": ["kubernetes"], "contentType": "stories", "minPoints": 50, "sortBy": "relevance" } ```` - **queries** - keywords (one search per line); leave empty to browse a category - **contentType** - stories, comments, or both - **category** - front page, Ask HN, Show HN, or polls - **author** - only a specific user's posts - **minPoints / minComments** - only items that got traction - **sinceDays** - recent items only - **sortBy** - best match or newest Examples: the current front page, the top "Show HN" launches over 100 points, every comment mentioning your product, or one author's whole history. ### Who uses this - **Founders and marketers**: monitor mentions of your product or competitors and find launches gaining traction. - **Developers and researchers**: track what the tech community is discussing, and pull discussions for analysis. - **Trend and market analysts**: measure attention on a technology or company over time by points and comments. - **Recruiters and community teams**: follow active authors and threads in a niche. Complements our Hacker News lead actors (which focus on hiring posts and lead alerts) with general story and comment scraping. ### Pricing A small fee per row (one per story or comment). Searches that match nothing are free note rows, and the first 2 rows of every run are free. ### Notes - Source: Algolia Hacker News Search API, the same search that powers HN's own search box. Points and comment counts reflect the last time Algolia indexed the item. - Relevance search reaches the first ~1,000 results per query; narrow with filters (points, date, author) or use "newest" to page deeper. # Actor input Schema ## `queries` (type: `array`): Keywords to search for, one per line ("kubernetes", "rust async"). Leave empty to just browse a category (e.g. the front page) using the filters below. ## `contentType` (type: `string`): Return stories, comments, or both. ## `category` (type: `string`): Limit to a Hacker News category (stories only). ## `author` (type: `string`): Optional: only items by this Hacker News username (e.g. "pg"). ## `minPoints` (type: `integer`): Only items with at least this many points. 0 = no minimum. ## `minComments` (type: `integer`): Stories only: at least this many comments. 0 = no minimum. ## `sinceDays` (type: `integer`): Only items from the last N days. 0 = no date limit. ## `sortBy` (type: `string`): Best match (relevance) or newest first. ## `maxPerQuery` (type: `integer`): How many items to return per search term (or per category browse). ## `maxRows` (type: `integer`): Stop after this many rows in total. ## Actor input object example ```json { "queries": [ "kubernetes" ], "contentType": "stories", "category": "any", "minPoints": 0, "minComments": 0, "sinceDays": 0, "sortBy": "relevance", "maxPerQuery": 50, "maxRows": 1000 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "kubernetes" ] }; // Run the Actor and wait for it to finish const run = await client.actor("scrapemint/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["kubernetes"] } # Run the Actor and wait for it to finish run = client.actor("scrapemint/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "kubernetes" ] }' | apify call scrapemint/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapemint/hacker-news-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hacker News Scraper: Stories, Comments & Search", "description": "Search and scrape Hacker News: stories and comments by keyword, author, points, date or category (front page, Ask HN, Show HN). One clean row per story or comment with points, comment count, author and links. No API key.", "version": "0.1", "x-build-id": "9YzHYvgOHMBMZ5FTT" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapemint~hacker-news-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapemint-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapemint~hacker-news-scraper/runs": { "post": { "operationId": "runs-sync-scrapemint-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapemint~hacker-news-scraper/run-sync": { "post": { "operationId": "run-sync-scrapemint-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "queries": { "title": "Search terms", "type": "array", "description": "Keywords to search for, one per line (\"kubernetes\", \"rust async\"). Leave empty to just browse a category (e.g. the front page) using the filters below.", "items": { "type": "string" } }, "contentType": { "title": "Content type", "enum": [ "stories", "comments", "all" ], "type": "string", "description": "Return stories, comments, or both.", "default": "stories" }, "category": { "title": "Category", "enum": [ "any", "front_page", "ask_hn", "show_hn", "poll" ], "type": "string", "description": "Limit to a Hacker News category (stories only).", "default": "any" }, "author": { "title": "Author", "type": "string", "description": "Optional: only items by this Hacker News username (e.g. \"pg\")." }, "minPoints": { "title": "Minimum points", "minimum": 0, "maximum": 100000, "type": "integer", "description": "Only items with at least this many points. 0 = no minimum.", "default": 0 }, "minComments": { "title": "Minimum comments", "minimum": 0, "maximum": 100000, "type": "integer", "description": "Stories only: at least this many comments. 0 = no minimum.", "default": 0 }, "sinceDays": { "title": "Posted within (days)", "minimum": 0, "maximum": 7300, "type": "integer", "description": "Only items from the last N days. 0 = no date limit.", "default": 0 }, "sortBy": { "title": "Sort by", "enum": [ "relevance", "newest" ], "type": "string", "description": "Best match (relevance) or newest first.", "default": "relevance" }, "maxPerQuery": { "title": "Results per search", "minimum": 1, "maximum": 1000, "type": "integer", "description": "How many items to return per search term (or per category browse).", "default": 50 }, "maxRows": { "title": "Max rows", "minimum": 1, "maximum": 20000, "type": "integer", "description": "Stop after this many rows in total.", "default": 1000 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```