# Hacker News Data Scraper (`maximedupre/hacker-news-scraper`) Actor Collect public Hacker News stories and posts by feed, topic, user, date, or listing. Export structured fields such as titles, links, authors, scores, ranks, publication times, and optional top-level comments to an Apify dataset. - **URL**: https://apify.com/maximedupre/hacker-news-scraper.md - **Developed by:** [Maxime Dupré](https://apify.com/maximedupre) (community) - **Categories:** News, Developer tools, Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.05 / 1,000 posts This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### 📰 Turn Hacker News into usable data Hacker News Scraper is for developers, researchers, founders, and news curators who need public Hacker News stories and posts in structured dataset rows. Choose a feed, topic, user, date, or listing to collect titles, links, authors, scores, ranks, publication times, and optional top-level comments for analysis or reuse. - Collect a ranked front-page snapshot with **[Hacker News top stories](https://apify.com/maximedupre/hacker-news-scraper/examples/hacker-news-top-stories)**. - Find stories about a topic with **[Hacker News search](https://apify.com/maximedupre/hacker-news-scraper/examples/hacker-news-search)** and review their titles, links, and scores. - Read a known discussion with **[Hacker News comments](https://apify.com/maximedupre/hacker-news-scraper/examples/hacker-news-comments)** when top-level comments matter. - Review public hiring posts with **[Hacker News job listings](https://apify.com/maximedupre/hacker-news-scraper/examples/hacker-news-job-listings)**. - Open a known item with **[Hacker News scraper](https://apify.com/maximedupre/hacker-news-scraper/examples/hacker-news-scraper)** when you have its ID or URL. #### 📊 See the Hacker News fields you need Each saved row represents a public Hacker News story or post found by the selected discovery method. The fields keep source values such as the item ID, title, publication time, author, score, and comment count. A topic search can also return the outbound URL and result rank. You can turn on top-level public comments when you need their author, text, posting time, and reply IDs. #### ▶️ Choose how to find Hacker News stories Choose one discovery method for each run. Fill only the fields for that method because values in other method sections are ignored. 1. Use a **Standard feed** for the Front page, Top stories, Newest stories, Best stories, Ask HN, Show HN, or Jobs feed. 2. Use **Topic search** for a full-text topic or phrase. Add an order, date range, minimum score, or minimum comment count when needed. 3. Use **User submissions** to collect public stories submitted by one Hacker News username. 4. Use **Historical content** to collect public stories for a start date and an optional end date. 5. Use **Specific listings** to submit one or more Hacker News item IDs or public item URLs. Run a small collection first when you want to check the fields. Set **Maximum stories** to a positive number to stop after that many stories. Leave it empty to collect all available results until the source is exhausted. #### ⚙️ Input The public input form has one discovery method and matching options for each method. **Input fields** | Field | Type | What it does | | --- | --- | --- | | `discoveryMethod` | select | Chooses `feed`, `search`, `user`, `date`, or `listing`. Use one method per run. | | `feed` | select | Chooses the public feed: `front`, `top`, `new`, `best`, `ask`, `show`, or `jobs`. Used only for `feed`. | | `searchQuery` | string | Sets the full-text topic or phrase. Used only for `search`. | | `searchOrder` | select | Orders topic results by `relevance` or `newest`. | | `searchStartDate` | date string | Sets the first publication date for topic search in `YYYY-MM-DD` format. Leave it empty for no start-date filter. | | `searchEndDate` | date string | Sets the last publication date for topic search in `YYYY-MM-DD` format. Leave it empty for no end-date filter. | | `minScore` | integer | Keeps topic-search stories with at least this many points. Leave it empty for no score filter. | | `minComments` | integer | Keeps topic-search stories with at least this many comments. Leave it empty for no comment filter. | | `userHandle` | string | Sets one Hacker News username whose public story submissions should be collected. Used only for `user`. | | `historyStartDate` | date string | Sets the first publication date for historical content in `YYYY-MM-DD` format. It is required for `date`. | | `historyEndDate` | date string | Sets the last publication date for historical content in `YYYY-MM-DD` format. Use the same date as `historyStartDate` for one day. | | `listingTargets` | array of strings | Adds one or more public Hacker News item IDs or item URLs. Used only for `listing`. | | `includeComments` | boolean | Includes available top-level public comments with their author, text, posting time, and reply relationships. | | `maxItems` | integer | Stops after this many stories. Leave empty to collect all available results until the source is exhausted. | **Input example** This is the public input from a successful default-input QA run: ```json { "discoveryMethod": "feed", "feed": "front", "searchOrder": "relevance", "includeComments": false, "maxItems": 100 } ``` #### 🧾 Output The `dataset` output links to the collected Hacker News rows. Use the dataset after the run or retrieve it through the Apify API. **Run output** | Field | Type | What it does | | --- | --- | --- | | `dataset` | URL | Links to the collected Hacker News items in the dataset. | **Standard Hacker News rows** Rows from feeds, topic search, user submissions, historical content, and specific listings use this shape. Fields marked optional are included when Hacker News supplies them for the item or selected method. | Field | Type | What it does | | --- | --- | --- | | `id` | integer | Stable Hacker News item ID. | | `type` | string | Source-provided item type, such as `story` or `job`. | | `title` | string | Title supplied by Hacker News. | | `publishedAt` | date-time string | Publication time supplied by Hacker News in ISO 8601 format. | | `text` | string, optional | Public post text supplied by Hacker News when available. The source may format it as HTML. | | `url` | URL, optional | Outbound URL supplied by Hacker News when available. | | `author` | string, optional | Hacker News username supplied for the item when available. | | `score` | integer, optional | Current score supplied by Hacker News when available. | | `commentCount` | integer, optional | Current number of comments supplied by Hacker News when available. | | `rank` | integer, optional | Position in the selected feed or topic-search results when available. | **Example standard row** This is a genuine row from a successful current-beta front-page run. ```json { "id": 49554643, "type": "story", "title": "GPT-6 Astra", "publishedAt": "2026-09-03T18:41:05.000Z", "text": "System Card: https://deploymentsafety.openai.com/gpt-6-astra

Related ongoing threads:

OpenAI's GPT-6 Astra on ARC-AGI-3 - https://news.ycombinator.com/item?id=49555691

GPT-6 Astra makes major gains in the Artificial Analysis Coding Agent Index - https://news.ycombinator.com/item?id=49556147", "url": "https://openai.com/index/gpt-6-astra/", "author": "kibae", "score": 665, "commentCount": 393, "rank": 1 } ``` **Rows with top-level comments** When `includeComments` is on, the row keeps the standard fields and adds available top-level public comments. Each comment can include direct reply IDs. Replies are linked by ID and are not rewritten into a second row shape. | Field | Type | What it does | | --- | --- | --- | | `id` | integer | Stable Hacker News item ID. | | `type` | string | Source-provided item type, such as `story` or `job`. | | `title` | string | Title supplied by Hacker News. | | `publishedAt` | date-time string | Publication time supplied by Hacker News in ISO 8601 format. | | `text` | string, optional | Public post text supplied by Hacker News when available. The source may format it as HTML. | | `url` | URL, optional | Outbound URL supplied by Hacker News when available. | | `author` | string, optional | Hacker News username supplied for the item when available. | | `score` | integer, optional | Current score supplied by Hacker News when available. | | `commentCount` | integer, optional | Current number of comments supplied by Hacker News when available. | | `rank` | integer, optional | Position in the selected feed or topic-search results when available. | | `comments` | array | Available top-level public comments when comment collection is enabled. | | `comments[].id` | integer | Stable Hacker News comment ID. | | `comments[].author` | string, optional | Hacker News username supplied for the comment when available. | | `comments[].text` | string, optional | Public comment text supplied by Hacker News when available. The source may format it as HTML. | | `comments[].postedAt` | date-time string, optional | Comment posting time supplied by Hacker News in ISO 8601 format when available. | | `comments[].replies` | array of integers, optional | IDs of available direct replies to the comment. | **Example row with comments** The comments list below is shortened. The first comment and all other shown values are genuine. The string `"..."` marks omitted comments. ```json { "id": 8863, "type": "story", "title": "My YC app: Dropbox - Throw away your USB drive", "publishedAt": "2007-04-04T19:16:40.000Z", "url": "http://www.getdropbox.com/u/2/screencast.html", "author": "dhouston", "score": 104, "commentCount": 71, "comments": [ { "id": 9224, "author": "BrandonM", "text": "I have a few qualms with this app:

1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.

2. It doesn't actually replace a USB drive. Most people I know e-mail files to themselves or host them somewhere online to be able to perform presentations, but they still carry a USB drive in case there are connectivity problems. This does not solve the connectivity issue.

3. It does not seem very \"viral\" or income-generating. I know this is premature at this point, but without charging users for the service, is it reasonable to expect to make money off of this?", "postedAt": "2007-04-05T15:16:54.000Z", "replies": [ 9272 ] }, "..." ] } ``` #### 💳 Pricing **Saved-row charge** The Actor uses pay-per-event pricing. One event is charged for each successfully collected public story or post saved to the dataset. A run that saves no story or post does not create this item event. The current rate for your Apify plan is shown in the Pricing tab. Use `maxItems` to bound the number of saved stories. Leave it empty to collect all available results until the source is exhausted. #### 🔌 Integrations **Dataset access** Open the `dataset` output after a run, or use the Apify API to retrieve the rows in your workflow. https://www.youtube.com/watch?v=bNACk1\_S\_6w\&list=PLObrtcm1Kw6MUrlLNDbK9QRg8VDJg0gOW\&index=4 #### ❓ FAQ ##### Can I narrow a topic search by date, score, or comment count? Yes. Use `searchStartDate` and `searchEndDate` for a publication-date range, or use `minScore` and `minComments` for topic-search thresholds. These fields apply only to Topic search. ##### Can I collect one known Hacker News item? Yes. Choose Specific listings and add an item ID such as `8863` or a public item URL. You can add more than one target in the same list. ##### What does including comments add? It adds available top-level public comments with their author, text, posting time, and direct reply IDs. The replies stay linked under their parent comment, and unavailable source values are not filled in. ##### What happens when an item has no score or comments? Those fields are optional. When Hacker News does not provide a value for an item, the field is absent from that row rather than filled with a made-up value. ##### Can I use several discovery methods in one run? No. Choose one method for each run. Values in the other option sections are ignored. ##### Do I need a Hacker News login or API key? No. The Actor collects public Hacker News data without buyer-supplied source credentials. ##### Does the Actor scrape the websites linked from Hacker News? No. It collects public Hacker News stories and posts. It does not fully extract content from outbound websites. ##### What does leaving Maximum stories empty do? It returns all available results until the source is exhausted. Set a positive value when you want a bounded run. ### 📝 Changelog **v0.0** (03-09-2026) - Initial release. ### 🆘 Support For issues, questions, or feature requests, [file a ticket](https://console.apify.com/actors/maximedupre~hacker-news-scraper/issues) and I'll fix or implement it in less than 24h 🫡 ### 🔗 Related Actors - [Reddit Scraper](https://apify.com/maximedupre/reddit-scraper) collects public Reddit posts, comments, communities, and user data when you need a broader community source. - [Reddit Comments Search Scraper](https://apify.com/maximedupre/reddit-comments-search-scraper) searches public discussion comments by keyword when comment text is your focus. - [Reddit Historical Archive Scraper for Posts and Comments](https://apify.com/maximedupre/reddit-historical-archive-scraper) finds dated public posts and comments when you need historical context. - [GitHub Discussions Scraper](https://apify.com/maximedupre/github-discussions) collects public developer discussions, replies, and reactions for product feedback. - [Product Hunt Scraper](https://apify.com/maximedupre/product-hunt-scraper) collects public product launches, topics, and engagement around new tools. **Made with ❤️ by Maxime Dupré** # Actor input Schema ## `discoveryMethod` (type: `string`): Choose how to find stories. Use one method per run. Fields for other methods are ignored. ## `feed` (type: `string`): Choose a public Hacker News feed: Front page, Top stories, Newest stories, Best stories, Ask HN, Show HN, or Jobs. ## `searchQuery` (type: `string`): Enter a topic or phrase for full-text search. This field is used only for Topic search. ## `searchOrder` (type: `string`): Choose how Topic search results are ordered. Relevance puts closer matches first; Newest first uses publication time. ## `searchStartDate` (type: `string`): Optionally enter the first publication date for Topic search. Use YYYY-MM-DD. Leave empty to start without a date filter. ## `searchEndDate` (type: `string`): Optionally enter the last publication date for Topic search. Use YYYY-MM-DD. Leave empty to end without a date filter. ## `minScore` (type: `integer`): Keep Topic search stories with at least this many points. Leave empty to use no score filter. ## `minComments` (type: `integer`): Keep Topic search stories with at least this many comments. Leave empty to use no comment filter. ## `userHandle` (type: `string`): Enter one Hacker News username to collect that user's public story submissions. This field is used only for User submissions. ## `historyStartDate` (type: `string`): Enter the first publication date to collect for Historical content. Use YYYY-MM-DD. This field is required for that method. ## `historyEndDate` (type: `string`): Optionally enter the last publication date for Historical content. Use YYYY-MM-DD. Use the same date as Start date for one day. ## `listingTargets` (type: `array`): Add one or more public Hacker News item IDs or URLs, such as 8863 or https://news.ycombinator.com/item?id=8863. This field is used only for Specific listings. ## `includeComments` (type: `boolean`): Include available top-level public comments with their author, text, posting time, and reply relationships. ## `maxItems` (type: `integer`): Stop after this many stories. Leave empty to collect all available results until the source is exhausted. ## Actor input object example ```json { "discoveryMethod": "feed", "feed": "front", "searchOrder": "relevance", "includeComments": false, "maxItems": 100 } ``` # Actor output Schema ## `dataset` (type: `string`): Open the collected Hacker News items in the dataset. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "discoveryMethod": "feed", "feed": "front", "maxItems": 100 }; // Run the Actor and wait for it to finish const run = await client.actor("maximedupre/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "discoveryMethod": "feed", "feed": "front", "maxItems": 100, } # Run the Actor and wait for it to finish run = client.actor("maximedupre/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "discoveryMethod": "feed", "feed": "front", "maxItems": 100 }' | apify call maximedupre/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,maximedupre/hacker-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/POH7StBMMb9sMNIJS/builds/fGmd8R5gT4aG7v4p7/openapi.json