# Hacker News Scraper | Stories, Ask HN, Show HN & Who Is Hiring (`resounding_diplomacy/hacker-news-scraper`) Actor Scrape Hacker News front page stories, Ask HN, Show HN posts, and Who Is Hiring threads. Extract titles, URLs, points, authors, comment counts, and job postings. Sort by points or comments. Export scraped data as JSON or CSV, run via API, schedule recurring runs, or integrate with Zapier and Make. - **URL**: https://apify.com/resounding\_diplomacy/hacker-news-scraper.md - **Developed by:** [alars num](https://apify.com/resounding_diplomacy) (community) - **Categories:** News, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hacker News Scraper Scrape top stories from Hacker News (Y Combinator) — front page posts, Ask HN, Show HN, and the monthly "Who Is Hiring" thread. Extract titles, points, comment counts, authors, and timestamps as structured JSON. No API key required. This actor uses CheerioCrawler for fast HTTP-based scraping, automatically paginates through results, and supports filtering and sorting so you get exactly the data you need. Export to JSON, CSV, Excel, or via API. ### Features - Scrape front page stories with automatic pagination - Filter stories by minimum upvote threshold (`minPoints`) - Sort results by points or comment count - Automatic detection of `Show HN`, `Ask HN`, and standard story types - "Who Is Hiring" mode to extract job postings from the monthly Ask HN thread - Extracts story ID, title, URL, points, author, comment count, and timestamp - No API key, authentication, or login required ### Input Configure the actor through the input schema. All fields are optional with sensible defaults. | Field | Type | Default | Description | |-------|------|---------|-------------| | `maxItems` | Integer | `30` | Maximum number of stories to scrape | | `minPoints` | Integer | `0` | Only include stories with at least this many points | | `sortBy` | String | `points` | Sort order: `points` or `comments` | | `includeComments` | Boolean | `false` | Include comment data (reserved for future use) | | `whoIsHiring` | Boolean | `false` | Scrape the latest "Ask HN: Who is Hiring" thread for job postings | #### Example input ```json { "maxItems": 50, "minPoints": 100, "sortBy": "points", "whoIsHiring": false } ```` ### Output Results are stored in the default dataset and can be exported as JSON, CSV, Excel, or via API. #### Story output (default mode) ```json { "id": "39482071", "title": "Show HN: A new way to build web apps", "url": "https://example.com/project", "points": 342, "author": "pg", "commentsCount": 128, "timeAgo": "2026-07-04T10:23:00", "isInternal": false, "type": "show_hn" } ``` #### Job posting output (Who Is Hiring mode) ```json { "text": "Company Name | Senior Engineer | Remote | $120k-$180k...", "type": "job_posting" } ``` ### Use cases - **Market research** — track trending tech topics and discussions - **Lead generation** — identify active founders, makers, and companies launching products via Show HN - **Job monitoring** — extract job postings from the monthly Who Is Hiring thread - **Content discovery** — surface viral stories and breaking news for newsletters or aggregators - **Sentiment analysis** — feed story data into NLP pipelines to gauge developer community trends - **Competitive intelligence** — monitor competitor launches and community reactions ### How to use 1. Open the actor on the [Apify Store](https://apify.com/store). 2. Click **Try for free** or **Start with Free Trial**. 3. Adjust the input fields (max items, minimum points, sort order) — or leave defaults. 4. Click **Start** and wait for the run to complete. 5. Export the results to JSON, CSV, or connect via the Apify API. For scheduled runs, use Apify's scheduler to scrape Hacker News daily or weekly and auto-export to your webhook, database, or Google Sheets. # Actor input Schema ## `maxItems` (type: `integer`): Maximum number of stories to scrape ## `minPoints` (type: `integer`): Only return stories with at least this many points ## `sortBy` (type: `string`): Sort results by points (highest first) or comments (most discussed first) ## `whoIsHiring` (type: `boolean`): Scrape the latest 'Ask HN: Who is Hiring' thread for job postings ## Actor input object example ```json { "maxItems": 30, "minPoints": 0, "sortBy": "points", "whoIsHiring": false } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("resounding_diplomacy/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("resounding_diplomacy/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call resounding_diplomacy/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=resounding_diplomacy/hacker-news-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hacker News Scraper | Stories, Ask HN, Show HN & Who Is Hiring", "description": "Scrape Hacker News front page stories, Ask HN, Show HN posts, and Who Is Hiring threads. Extract titles, URLs, points, authors, comment counts, and job postings. Sort by points or comments. Export scraped data as JSON or CSV, run via API, schedule recurring runs, or integrate with Zapier and Make.", "version": "0.0", "x-build-id": "wdNfdfBpRnWpbbPmp" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/resounding_diplomacy~hacker-news-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-resounding_diplomacy-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/resounding_diplomacy~hacker-news-scraper/runs": { "post": { "operationId": "runs-sync-resounding_diplomacy-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/resounding_diplomacy~hacker-news-scraper/run-sync": { "post": { "operationId": "run-sync-resounding_diplomacy-hacker-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "maxItems": { "title": "Max Items", "type": "integer", "description": "Maximum number of stories to scrape", "default": 30 }, "minPoints": { "title": "Minimum Points", "type": "integer", "description": "Only return stories with at least this many points", "default": 0 }, "sortBy": { "title": "Sort By", "enum": [ "points", "comments" ], "type": "string", "description": "Sort results by points (highest first) or comments (most discussed first)", "default": "points" }, "whoIsHiring": { "title": "Who Is Hiring Mode", "type": "boolean", "description": "Scrape the latest 'Ask HN: Who is Hiring' thread for job postings", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```