# Hacker News Scraper – Stories, Comments & Who Is Hiring Jobs (`openharvest/hacker-news-scraper`) Actor Hacker News scraper: search stories and comments by keyword, points and date, get top/new/best/Ask HN/Show HN/job lists, full comment threads (flattened with depth and parentId), user profiles and parsed 'Ask HN: Who is hiring?' job posts. Official HN APIs, no login. - **URL**: https://apify.com/openharvest/hacker-news-scraper.md - **Developed by:** [Enes BulanΔ±k](https://apify.com/openharvest) (community) - **Categories:** News, Developer tools, Jobs - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $0.20 / 1,000 item scrapeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Hacker News Scraper – Stories, Comments & Who Is Hiring Jobs Hacker News scraper: search stories and comments by keyword, points and date, get top/new/best/Ask HN/Show HN/job lists, full comment threads (flattened with depth and parentId), user profiles and parsed 'Ask HN: Who is hiring?' job posts. Official HN APIs, no login. **Price: $0.20 per 1,000 items.** No login, no API key, no browser. Empty searches and invalid inputs are never charged. ### What data do you get? - πŸ“° **Stories:** title, URL, domain, points, comment count, author, date, HN link, Ask/Show HN text, list rank - πŸ’¬ **Comments:** plain text (HTML decoded, links expanded), author, date, `storyId`, `storyTitle`, `parentId`, `depth` (1 = top-level, 2 = reply …) - 🧡 **Threads:** story plus all comments in Hacker News order, depth-first, with optional depth/size limits - πŸ’Ό **Who is hiring job posts:** `company`, `location`, `remote` / `hybrid` / `onsite`, `roles[]`, `employmentType[]`, `salary`, `visaSponsorship`, `applyUrl`, `urls[]`, full `text`, thread month - πŸ‘€ **Users:** karma, account creation date, bio (contact details removed), submission count, recent submission IDs Data comes from the two **official Hacker News APIs**: the Algolia HN Search API (search, full threads) and the HN Firebase API (live front-page lists, items, users). ### Use cases - **Tech trend and brand monitoring:** track every mention of your product, competitors or a technology, sorted by date. - **Recruiting and job-market research:** each month, export all "Who is hiring?" posts as a spreadsheet and filter by remote, salary, Rust, Python, Berlin and so on. - **Sentiment and NLP datasets:** download whole discussion trees with parent/child structure for analysis or fine-tuning. - **Content research:** find the highest-scoring posts on any topic and see what the community said. - **AI agents and RAG:** give an LLM fresh, structured HN discussions on demand. ### How to use it 1. Click **Try for free** or **Start**. 2. Choose a **Mode**: `search`, `list`, `item`, `whoIsHiring` or `user`. 3. Fill in the fields for that mode (each field says which mode it applies to) and set **Max items**. 4. Click **Start**, then export as JSON, CSV or Excel or connect via API, Make, Zapier, n8n or Google Sheets. ### Input | Field | Mode | Description | |---|---|---| | `mode` | all | `search` (default), `list`, `item`, `whoIsHiring`, `user` | | `searchQuery` | search | Keywords, e.g. `AI agents`. Empty = browse by filters. | | `searchType` | search | `story` (default), `comment`, `ask_hn`, `show_hn`, `job`, `poll`, `front_page`, `all` | | `sortBy` | search | `relevance` (popular first, max 1,000) or `date` (newest first, no cap) | | `minPoints`, `minComments` | search, list | Minimum points / comments | | `onlyNewerThan` | search, list | `2026-01-01` or `7 days`, `1 month` … | | `author` | search | Only items by this username | | `listType` | list | `top`, `new`, `best`, `ask`, `show`, `job` | | `itemIds` | item | IDs or `news.ycombinator.com/item?id=…` URLs | | `includeComments` | search, list | Also return each story's comments | | `maxCommentsPerStory` | search, list, item | Default `100`, `0` = all | | `maxCommentDepth` | search, list, item | `1` = top-level only, `0` = unlimited | | `hiringMonth` | whoIsHiring | `YYYY-MM`. Empty = latest thread | | `hiringKeyword` | whoIsHiring | Only posts containing all these words, e.g. `rust remote` | | `remoteOnly` | whoIsHiring | Only remote-friendly posts | | `usernames` | user | Usernames or profile URLs | | `maxItems` | all | Total item limit (= cost cap). Default `100` | Example inputs: ```json { "mode": "search", "searchQuery": "postgres", "sortBy": "date", "minPoints": 50, "onlyNewerThan": "30 days" } ``` ```json { "mode": "whoIsHiring", "hiringKeyword": "python", "remoteOnly": true, "maxItems": 300 } ``` ```json { "mode": "item", "itemIds": ["https://news.ycombinator.com/item?id=8863"], "maxCommentDepth": 2 } ``` ### Output Every item has a `type`: `story`, `comment`, `job`, `poll`, `job-post`, `user` or `error` (free). This is real output (shortened): **Story** (mode `list`) ```json { "type": "story", "id": 49991227, "title": "Shipping JPEG XL in Chrome", "url": "https://developer.chrome.com/blog/jpeg-xl-in-chrome", "domain": "developer.chrome.com", "hnUrl": "https://news.ycombinator.com/item?id=49991227", "text": null, "author": "AshleysBrain", "points": 397, "numComments": 254, "createdAt": "2026-10-07T11:25:02.000Z", "storyId": 49991227, "parentId": null, "depth": 0, "rank": 6, "listType": "top", "tags": ["story"] } ``` **Comment** (thread, flattened) ```json { "type": "comment", "id": 49995760, "text": "There are lots in there with proper ASCII check out https://ascii.rest/julia-set/ for example", "author": "elevaet", "createdAt": "2026-10-07T17:13:42.000Z", "storyId": 49993857, "storyTitle": "Animated ASCII Art for Web Pages", "storyUrl": "https://ascii.rest/", "parentId": 49995373, "depth": 2, "hnUrl": "https://news.ycombinator.com/item?id=49995760", "links": ["https://ascii.rest/julia-set/"] } ``` **Who is hiring job post** ```json { "type": "job-post", "id": 49995549, "hnUrl": "https://news.ycombinator.com/item?id=49995549", "company": "Mastra", "location": "REMOTE (AMER or EMEA time zones)", "remote": true, "hybrid": null, "onsite": null, "roles": ["AI Engineer"], "employmentType": ["full-time"], "salary": null, "visaSponsorship": null, "applyUrl": "https://jobs.ashbyhq.com/mastra/531a3fe8-556f-4b83-bee7-20374f67f980", "urls": ["https://github.com/mastra-ai/mastra", "https://jobs.ashbyhq.com/mastra/531a3fe8-…"], "text": "Mastra | AI Engineer | REMOTE (AMER or EMEA time zones) | Full-time\nMastra is the open-source TypeScript framework for building AI agents …", "author": "RitaSand", "createdAt": "2026-10-07T16:59:27.000Z", "numReplies": 0, "threadId": 49922569, "threadTitle": "Ask HN: Who is hiring? (October 2026)", "threadMonth": "October 2026" } ``` **User** ```json { "type": "user", "id": "pg", "karma": 157316, "createdAt": "2006-10-09T18:21:32.000Z", "text": "Bug fixer.", "submittedCount": 15565, "hnUrl": "https://news.ycombinator.com/user?id=pg" } ``` **How "Who is hiring" parsing works:** the Actor finds the latest (or chosen month's) thread posted by `whoishiring` and reads every top-level comment. It splits the `Company | Role | Location | Remote | Salary | URL` header that most posts use, falls back to labeled lines such as `Location:` and `Salary:` in the body, and detects remote/hybrid/onsite, employment type, visa sponsorship and salary ranges in USD, EUR, GBP and other currencies. Funding amounts like "raised $42M" are ignored. Meta comments that aren't job posts are skipped and not charged. Parsing is heuristic. On the October 2026 thread (239 top-level comments), 227 were classified as job posts. Of those, 96% had a location detected, 86% a remote/onsite flag, 86% an apply link, 77% roles and 30% a salary (which matches how often posts state one). The full original `text` is always included. Dataset views in Apify Console: **Overview**, **Who is hiring**, **Users**, **Errors**. ### Pricing Pay per event, with platform usage included. | Event | Price | When it is charged | |---|---|---| | `item` | **$0.0002** per item ($0.20 / 1,000) | Each story, comment, job post or user saved | | Errors / no results | **Free** | Invalid IDs, missing users, empty searches | | Job | Cost | |---|---| | Top 30 front-page stories | **$0.015** | | One month of "Who is hiring?" (~250 posts) | **~$0.13** | | A 500-comment thread | **$0.25** | | 10,000 search results | **$5.00** | Set **Max items** and Apify's **Max cost per run**. The Actor stops cleanly at the limit. ### Use with AI agents (MCP) Add the Actor as a tool in the [Apify MCP server](https://mcp.apify.com) (`https://mcp.apify.com?tools=openharvest/hacker-news-scraper`), or let agents find it with `search-actors` ("hacker news"). Example `call-actor` input for "What is HN saying about the new Rust release?": ```json { "mode": "search", "searchQuery": "rust release", "sortBy": "date", "onlyNewerThan": "14 days", "includeComments": true, "maxCommentsPerStory": 20, "maxItems": 60 } ``` Example input for "Find remote Python jobs from this month's Who is hiring": ```json { "mode": "whoIsHiring", "hiringKeyword": "python", "remoteOnly": true, "maxItems": 100 } ``` ### FAQ **Is there a limit on search results?** Algolia returns at most 1,000 results per query in relevance order. With `sortBy: "date"` the Actor pages back in time past that limit, so you can collect complete history for a keyword. **Why do comments from search not have `depth`?** Search returns individual comments without their position in the tree (`depth: null`, but `parentId` and `storyId` are set). Use mode `item` or `includeComments` to get the full thread with depths. **In which order are thread comments returned?** Top-level comments follow Hacker News's own ranking. Replies come right after their parent, depth-first, so a thread reads like it does on the site. **Does it include dead or deleted comments?** No. Dead and deleted comments are skipped, but their replies are kept. **Can it parse "Who wants to be hired?" threads?** No. Those posts are written by individuals and contain personal contact details, so the Actor only parses company "Who is hiring?" posts. **How fresh is the data?** Lists, items and users come live from the official Firebase API. Search and threads come from Algolia, which typically indexes new items within a minute or two. ### Is it legal to scrape Hacker News? The Actor only uses Hacker News's **official public APIs** (Algolia HN Search and the HN Firebase API, both provided for public use) and only reads public posts. It does not log in. User bios are returned with emails and phone numbers removed, and the Actor does not harvest contact data. Posts are written by HN users. Respect their rights and the [Hacker News guidelines](https://news.ycombinator.com/newsguidelines.html), and comply with applicable laws such as GDPR when you process the data. **Do not use the data for spam or unsolicited outreach.** ### Related actors - Polymarket, Kalshi & Manifold Scraper – Prediction Markets, Odds & Price History *(coming soon)* - GitHub Scraper *(coming soon)* - Reddit Scraper *(coming soon)* - Google News Scraper *(coming soon)* ### Support & feedback Found a bug or a missing field? Open an issue in the **Issues** tab and we usually reply within a day. # Actor input Schema ## `mode` (type: `string`): search = keyword search over stories/comments (Algolia). list = current top/new/best/Ask/Show/job lists. item = specific stories with their full comment thread. whoIsHiring = parsed job posts from the monthly 'Ask HN: Who is hiring?' thread. user = user profiles. ## `searchQuery` (type: `string`): Mode search: keywords to search for, e.g. AI agents, rust, postgres. Leave empty to browse by the filters below. ## `searchType` (type: `string`): Mode search: what to search. ## `sortBy` (type: `string`): Mode search: relevance (popular first, max 1,000 results) or date (newest first, no result cap). ## `minPoints` (type: `integer`): Modes search and list: only stories with at least this many points. ## `minComments` (type: `integer`): Mode search: only stories with at least this many comments. ## `onlyNewerThan` (type: `string`): Modes search and list: skip items older than this. Absolute date (YYYY-MM-DD) or relative period like 7 days, 1 month. ## `author` (type: `string`): Mode search: only items posted by this HN username. ## `listType` (type: `string`): Mode list: which Hacker News list to read (same order as on the site). ## `itemIds` (type: `array`): Mode item: Hacker News item IDs or URLs like https://news.ycombinator.com/item?id=8863. Returns the item plus its whole comment thread, flattened with depth and parentId. ## `includeComments` (type: `boolean`): Modes search and list: after each story, also return its comments (each comment is one charged item). Mode item always includes comments. ## `maxCommentsPerStory` (type: `integer`): Limit the number of comments per story (threads are walked depth-first in Hacker News order). 0 = all comments. ## `maxCommentDepth` (type: `integer`): 1 = only top-level comments, 2 = plus direct replies, ... 0 = unlimited. ## `hiringMonth` (type: `string`): Mode whoIsHiring: thread month as YYYY-MM (e.g. 2026-10). Leave empty for the latest thread. ## `hiringKeyword` (type: `string`): Mode whoIsHiring: only job posts containing all these words (case-insensitive), e.g. rust, python berlin. ## `remoteOnly` (type: `boolean`): Mode whoIsHiring: only job posts detected as remote-friendly. ## `includeNonJobComments` (type: `boolean`): Mode whoIsHiring: also return top-level comments that don't look like job posts (meta discussion). Off by default. ## `usernames` (type: `array`): Mode user: Hacker News usernames or profile URLs. Returns karma, account creation date, bio and submission count. ## `maxItems` (type: `integer`): Maximum number of items (stories, comments, job posts or users) in total. You pay per item, so this also caps your cost. ## Actor input object example ```json { "mode": "search", "searchQuery": "rust", "searchType": "story", "sortBy": "date", "minPoints": 100, "minComments": 20, "onlyNewerThan": "30 days", "author": "dang", "listType": "top", "itemIds": [ "https://news.ycombinator.com/item?id=8863" ], "includeComments": true, "maxCommentsPerStory": 50, "maxCommentDepth": 2, "hiringMonth": "2026-10", "hiringKeyword": "rust", "remoteOnly": true, "includeNonJobComments": false, "usernames": [ "pg", "dang" ], "maxItems": 50 } ``` # Actor output Schema ## `results` (type: `string`): All scraped items in the default dataset (JSON, CSV, Excel and more via the dataset export). ## `files` (type: `string`): Records saved in the default key-value store, e.g. the run summary. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "searchQuery": "AI agents", "maxItems": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("openharvest/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "searchQuery": "AI agents", "maxItems": 50, } # Run the Actor and wait for it to finish run = client.actor("openharvest/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "searchQuery": "AI agents", "maxItems": 50 }' | apify call openharvest/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,openharvest/hacker-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/jxIDgcdHOvYzfT5EM/builds/0AJ2LdbernARKRmJw/openapi.json