# Hacker News Scraper — Stories, Comments & Ask HN by Keyword (`convertfleetdotonline/hacker-news-scraper`) Actor Search and export Hacker News stories, comments, Ask HN, Show HN and polls by keyword. Points, author, comment counts, URLs & timestamps. Official public API — no login, no key, no proxy. JSON/CSV export. - **URL**: https://apify.com/convertfleetdotonline/hacker-news-scraper.md - **Developed by:** [Hasnain Nisar](https://apify.com/convertfleetdotonline) (community) - **Categories:** News, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hacker News Scraper — Stories, Comments, Ask/Show HN by Keyword Search and export **Hacker News** content as clean structured data. Give it any keyword and get back **stories, comments, Ask HN, Show HN and polls** with points, author, comment counts, URLs and timestamps — ready for JSON, CSV, Excel, Google Sheets or your database. Built on Hacker News' **official public search API**, so it is fast, stable and needs **no login, no API key and no proxy**. ### 🔍 What you can do - Track every mention of a **product, company, competitor or topic** on Hacker News - Pull the **front page** on a schedule for a news/monitoring dashboard - Collect **Ask HN / Show HN** threads for product and market research - Export **top comments** for sentiment and community analysis - Feed Hacker News discussions into an **LLM / RAG** pipeline ### ⚙️ Input ```json { "queries": ["openai", "rust"], "searchType": "story", "sortBy": "relevance", "maxItemsPerQuery": 100, "minPoints": 50 } ```` | Field | Description | |---|---| | `queries` | Keywords to search. Empty → current front page. | | `searchType` | `story`, `comment`, `ask_hn`, `show_hn`, `poll` or `front_page`. | | `sortBy` | `relevance` or `date` (newest first). | | `maxItemsPerQuery` | 1–1000 items per query. | | `minPoints` | Only items with at least this many points. | ### 📤 Output (one item per result) ```json { "objectID": "38765432", "type": "story", "title": "Show HN: I built …", "url": "https://example.com", "author": "pg", "points": 512, "num_comments": 231, "text": null, "created_at": "2026-01-04T12:00:00.000Z", "hn_url": "https://news.ycombinator.com/item?id=38765432", "query": "openai" } ``` ### ❓ FAQ **Do I need a Hacker News or Algolia account?** No. This Actor uses the free public Hacker News search API — no key, no login. **How fresh is the data?** Real time. `sortBy: "date"` returns items the moment they are indexed. **Can I get every comment on a thread?** Use `searchType: "comment"` with the thread keyword, or search by the story author. Very large threads are paginated automatically up to your item cap. **How do I run it on a schedule?** Use the built-in Apify Scheduler or call the run-sync API endpoint from Make, Zapier, n8n or your own code. ### ⚖️ Disclaimer Not affiliated with Y Combinator or Hacker News. Reads only publicly available data via the official public API. You are responsible for how you use the data. ### 🔌 Integrations Works with the Apify API, Scheduler, webhooks, Make, Zapier, n8n, Google Sheets and Slack. # Actor input Schema ## `queries` (type: `array`): Keywords to search Hacker News for (e.g. "openai", "rust"). Leave empty to fetch the current front page. ## `searchType` (type: `string`): Which Hacker News item type to return. ## `sortBy` (type: `string`): Rank by relevance, or return newest first. ## `maxItemsPerQuery` (type: `integer`): Upper bound of items returned for each query (1–1000). ## `minPoints` (type: `integer`): Only return stories/comments with at least this many points. ## Actor input object example ```json { "queries": [ "openai", "rust", "startup" ], "searchType": "story", "sortBy": "relevance", "maxItemsPerQuery": 100, "minPoints": 0 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "openai" ] }; // Run the Actor and wait for it to finish const run = await client.actor("convertfleetdotonline/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["openai"] } # Run the Actor and wait for it to finish run = client.actor("convertfleetdotonline/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "openai" ] }' | apify call convertfleetdotonline/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=convertfleetdotonline/hacker-news-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hacker News Scraper — Stories, Comments & Ask HN by Keyword", "description": "Search and export Hacker News stories, comments, Ask HN, Show HN and polls by keyword. Points, author, comment counts, URLs & timestamps. Official public API — no login, no key, no proxy. JSON/CSV export.", "version": "1.0", "x-build-id": "QwEoI1Q28qm5jNuo8" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/convertfleetdotonline~hacker-news-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-convertfleetdotonline-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/convertfleetdotonline~hacker-news-scraper/runs": { "post": { "operationId": "runs-sync-convertfleetdotonline-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/convertfleetdotonline~hacker-news-scraper/run-sync": { "post": { "operationId": "run-sync-convertfleetdotonline-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "queries": { "title": "Search queries / keywords", "type": "array", "description": "Keywords to search Hacker News for (e.g. \"openai\", \"rust\"). Leave empty to fetch the current front page.", "items": { "type": "string" } }, "searchType": { "title": "What to search", "enum": [ "story", "comment", "ask_hn", "show_hn", "poll", "front_page" ], "type": "string", "description": "Which Hacker News item type to return.", "default": "story" }, "sortBy": { "title": "Sort by", "enum": [ "relevance", "date" ], "type": "string", "description": "Rank by relevance, or return newest first.", "default": "relevance" }, "maxItemsPerQuery": { "title": "Max items per query", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Upper bound of items returned for each query (1–1000).", "default": 100 }, "minPoints": { "title": "Minimum points", "minimum": 0, "type": "integer", "description": "Only return stories/comments with at least this many points.", "default": 0 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```