# Hacker News Scraper: Stories, Comments, Search, Monitor (`themineworks/hacker-news-scraper`) Actor Scrape Hacker News top, new, best, Ask HN, Show HN and jobs, or search by keyword with a date range and minimum points. One row per story and per comment with depth, plus user submissions, full threads and a monitor mode for new items only. Free public APIs, no login. - **URL**: https://apify.com/themineworks/hacker-news-scraper.md - **Developed by:** [The Mine Works](https://apify.com/themineworks) (community) - **Categories:** News, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.18 / 1,000 items This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Hacker News Scraper: Stories, Comments, Search, Monitor [![50 Hacker News front page stories in 3 seconds, with points, comments, domain and author](https://api.apify.com/v2/key-value-stores/cUXz95yxflDho41nn/records/hacker-news-scraper-hero.png)](https://console.apify.com/actors/lP6YouUHhJ3EA0xCI/input?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) From **The Mine Works**, makers of [Reddit Scraper](https://apify.com/themineworks/reddit-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) and [GitHub Trending Scraper](https://apify.com/themineworks/github-trending-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral), with over 140,000 runs across 170+ public actors. Updated 21 Oct 2026: first release. Top, New, Best, Ask HN, Show HN and Jobs lists, keyword search with a date range and minimum points, a user's submissions, full comment threads with depth, and a monitor mode that returns only new items. Bookmark this Actor on its Store page so it is one click away next time. ### Why choose this actor? - **Every list, every search, every thread.** The six live lists in the order the site shows them, keyword search over stories or comments newest first back to 2007, a date range, minimum points and comments, a user's stories and comments, and any story or comment with the replies under it. One input field picks which. - **Comments as real rows, not a nested blob.** With `includeComments` on, each story is followed by its comments in thread order, each with `depth`, `parent_id`, `story_id`, `reply_count` and its own link. You choose how many levels deep and how many per story. Filter on `depth: 1` for direct replies or rebuild the tree from `parent_id`. - **Monitor mode for scheduled runs.** Watch a keyword (your product, a competitor, a library) or the front page, and each run returns only the stories and comments no earlier run delivered. Items skipped this way are not charged. - **Cheap and fast, nothing to set up.** $0.18 per 1,000 rows on every plan, read from Hacker News's own free public APIs. No account, no API key, no cookies. [![Run it on Apify](https://api.apify.com/v2/key-value-stores/cUXz95yxflDho41nn/records/button-run.png)](https://console.apify.com/actors/lP6YouUHhJ3EA0xCI/input?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Part of The Mine Works Social media and video family:** [Threads Scraper](https://apify.com/themineworks/threads-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral), [Reddit Scraper](https://apify.com/themineworks/reddit-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral), [Threads Search Scraper](https://apify.com/themineworks/threads-search-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral), [Instagram Profile Scraper](https://apify.com/themineworks/instagram-profile-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral), [Instagram Followers & Following](https://apify.com/themineworks/instagram-followers-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral), [Reddit Search Scraper](https://apify.com/themineworks/reddit-search-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral). ### Try it in one minute Paste this into the input (JSON view) and press Start. It returns the first 50 stories of the Hacker News front page, one row each. It took 3 seconds in our test run: ```json { "mode": "top", "maxItems": 50 } ``` Switch `mode` to `search` and put words in `keywords` to search instead, or turn on `includeComments` to get the discussion under each story. Apify's free plan includes $5 of credit every month, which covers about 27,000 rows at this actor's price. #### Copy to your AI assistant ```text themineworks/hacker-news-scraper on Apify. Reads Hacker News through its free public APIs and returns one flat row per story and per comment, no login. Call ApifyClient("TOKEN").actor("themineworks/hacker-news-scraper").call(run_input={"mode": "search", "keywords": ["rust"], "dateFrom": "2026-10-01", "maxItems": 100}), then client.dataset(run["defaultDatasetId"]).list_items().items. mode is one of top, new, best, ask, show, jobs (live lists), search (needs keywords, or a date range or minPoints), user (needs usernames), item (needs itemIds, ids or news.ycombinator.com/item?id= links). Search options: searchIn (story, ask_hn, show_hn, comment, all; default story), sortBy (date or relevance; default date), dateFrom and dateTo (YYYY-MM-DD UTC or relative like "7 days"), minPoints, minComments. User option: userContent (stories, comments, all). maxItems (default 100, 1 to 10000) is per list, keyword or user. includeComments (default false; always on in item mode), maxCommentDepth (default 5), maxCommentsPerStory (default 100). monitorMode (default false) returns only items no earlier monitor run with the same mode and keywords, users or items delivered; then maxItems is the window checked. Rows have record_type "story" or "comment". Story rows: id, title, url, domain, points, author, comment_count, created_at, created_at_unix, type (story, ask, show, launch, job, poll), hn_url, text, text_html, rank, position, source, source_query, comments_fetched. Comment rows: id, parent_id, story_id, story_title, story_url, author, text, text_html, created_at, created_at_unix, depth, reply_count, thread_position, hn_url, position, source, source_query. Each run also writes one row with _type "summary", one with _type "info" when rows were delivered, and a row with _type "error" for each input that could not be read; none of these is charged. Full spec: GET https://api.apify.com/v2/acts/themineworks~hacker-news-scraper/builds/default (Bearer TOKEN), which returns inputSchema and readme. Token: https://console.apify.com/account/integrations?fpr=ymnoit&utm_source=apify-readme&utm_medium=referral ``` ### Key features - **Six live lists.** `top` (the front page and beyond, 500 stories), `new` (500), `best` (200), `ask`, `show` and `jobs`, in the order Hacker News shows them, with `rank` on each row. - **Keyword search with dates.** Search titles, links and story text, or comments. Newest first goes back as far as Hacker News does, past the usual 1,000 result limit of the search API: in our test one keyword returned 2,500 stories in 6 seconds. Most relevant first is also there, up to 1,000 per keyword. Typo matching is off, so `rust` does not also return posts about `just`. - **Filters that matter.** `dateFrom` and `dateTo` as dates or as "7 days", `minPoints` and `minComments`, and `searchIn` to search only Ask HN, only Show HN or only comments. - **Users and threads.** `user` mode returns a user's stories, comments or both, newest first. `item` mode takes any story or comment id or link and returns it with every reply under it, up to your limits; for a comment it also finds the story and the comment's depth. - **Readable text.** `text` is plain text with paragraphs and links written out; `text_html` keeps the original Hacker News HTML. - **Stops cleanly.** A list or search ends when it runs out, when `maxItems` is reached, or when two pages in a row bring nothing new. Eight failed requests in a row end the run, and the run stops before its time limit and keeps every row. The summary row says why each source stopped. ### How to use it #### Front page with the discussion ```json { "mode": "top", "maxItems": 30, "includeComments": true, "maxCommentDepth": 2, "maxCommentsPerStory": 50 } ``` 30 stories, each followed by up to 50 comments, direct replies and the replies to them. In our test, best 200 stories with up to 50 comments each, 3 levels deep, gave 5,221 rows in 47 seconds. #### Everything said about a topic last month ```json { "mode": "search", "keywords": ["rust", "postgres"], "dateFrom": "2026-09-01", "dateTo": "2026-09-30", "maxItems": 1500 } ``` Each keyword is its own search, labelled in `source_query`; a story both keywords find is delivered once. In our test this returned all 590 matching stories in 3 seconds. Set `searchIn` to `comment` to get what people wrote in the comments instead. #### Popular posts only ```json { "mode": "search", "keywords": ["linux"], "dateFrom": "7 days", "minPoints": 50 } ``` Stories about Linux from the last 7 days with 50 points or more. Leave `keywords` empty with a date range and `minPoints` to list every popular story of a period. #### A user's history ```json { "mode": "user", "usernames": ["pg", "https://news.ycombinator.com/user?id=dang"], "userContent": "all", "maxItems": 500 } ``` Stories and comments of each user, newest first. The summary row also gives each user's karma and when the account was made. #### One thread, all of it ```json { "mode": "item", "itemIds": ["https://news.ycombinator.com/item?id=8863"], "maxCommentDepth": 100, "maxCommentsPerStory": 5000 } ``` The story row, then every comment in thread order. Rebuild the tree with `parent_id`, or sort by `thread_position` to read it as the page does. #### Keyword monitor on a schedule ```json { "mode": "search", "keywords": ["your product name"], "searchIn": "all", "maxItems": 200, "monitorMode": true } ``` Save it as a task and schedule it every hour. The first run returns the 200 newest matches; each later run checks the 200 newest again and returns only the ones not delivered before, stories and comments. Nothing new means nothing charged, not even the run fee. ### Input parameters | Parameter | Type | Default | What it does | |---|---|---|---| | `mode` | string | `top` | `top`, `new`, `best`, `ask`, `show`, `jobs`, `search`, `user` or `item` | | `keywords` | array of strings | empty | Search mode: one search per line, all words of a line must match | | `searchIn` | string | `story` | Search mode: `story`, `ask_hn`, `show_hn`, `comment` or `all` | | `sortBy` | string | `date` | Search mode: `date` (newest first, no depth limit) or `relevance` (up to 1,000 per keyword) | | `dateFrom`, `dateTo` | string | empty | Search and user modes: `YYYY-MM-DD` in UTC, or relative such as `7 days`. `dateTo` includes the whole day | | `minPoints`, `minComments` | integer | empty | Search and user modes: only stories with at least this many points or comments | | `usernames` | array of strings | empty | User mode: names or profile links, case sensitive | | `userContent` | string | `stories` | User mode: `stories`, `comments` or `all` | | `itemIds` | array of strings | empty | Item mode: story or comment ids or links | | `maxItems` | integer | `100` | Most stories (or comments when you search comments) per list, keyword or user, 1 to 10,000 | | `includeComments` | boolean | `false` | Adds the comments under each story as rows. Always on in item mode | | `maxCommentDepth` | integer | `5` | Levels of replies to read, 1 = direct replies only | | `maxCommentsPerStory` | integer | `100` | Most comment rows under one story or item, 1 to 10,000 | | `monitorMode` | boolean | `false` | Only items no earlier monitor run with the same watch delivered | Up to 100 keywords, users or items per run. ### What data do you get? One row per story and one per comment, told apart by `record_type`. Data rows have no `_type` field; the summary, info and error rows do, so a filter on `_type` separates them. **Story rows** (`record_type: "story"`) | Field | What it is | |---|---| | `id` | Hacker News item id | | `title` | Story title | | `url`, `domain` | Link the story points to and its domain; text posts point to their own page | | `points` | Points (score) | | `author` | Username of the poster | | `comment_count` | Comments on the story, all levels | | `created_at`, `created_at_unix` | When it was posted, ISO 8601 UTC and unix seconds | | `type` | `story`, `ask`, `show`, `launch`, `job` or `poll` | | `hn_url` | Discussion page on news.ycombinator.com | | `text`, `text_html` | Body of Ask HN, Show HN and text posts, as plain text and as HTML | | `rank` | Place on the live list when it was read (list modes) | | `comments_fetched` | Comment rows delivered under it (with `includeComments`) | **Comment rows** (`record_type: "comment"`) | Field | What it is | |---|---| | `id` | Hacker News item id of the comment | | `parent_id` | The story or comment it replies to | | `story_id`, `story_title` | The story it belongs to (`story_url` too for search results) | | `author` | Username of the commenter | | `text`, `text_html` | The comment as plain text and as HTML | | `created_at`, `created_at_unix` | When it was posted | | `depth` | Level in the thread, 1 = direct reply to the story (empty for comments found by search) | | `reply_count` | Direct replies to it | | `thread_position` | Place in the thread as the page shows it | | `hn_url` | Link to the comment | **On every row:** `source` (`top`, `new`, `best`, `ask`, `show`, `jobs`, `search`, `user` or `item`), `source_query` (the keyword, username or item id), `position` (place within that source, stories and searched comments) and `scraped_at`. **Summary row** (`_type: "summary"`, never charged): `sources_requested`, `sources_read`, `sources_failed`, `pages_read`, `stories_found`, `stories_delivered`, `comments_delivered`, `items_delivered`, `charged_for`, `duplicates_skipped`, `known_from_earlier_runs`, `comments_dead_or_deleted`, `comment_threads_cut_at_cap`, `monitor`, `http_requests`, `http_megabytes`, `ended_on_deadline`, `stopped_by_max_cost`, `shortfall_reason` when fewer rows came back than asked, and `per_source`, one entry per list, keyword, user or item with `pages`, `found`, `delivered`, `total_matches` for searches, `karma` for users, and why it `stopped`. The same summary is saved as the `OUTPUT` record of the run. **Error rows** (`_type: "error"`, never charged): one per input that could not be read, with `error_type` (`invalid_item`, `invalid_username`, `invalid_date`, `not_found`, `rate_limited`, `fetch_error`), `identifier` and `reason`. #### Stable fields for automations These were present in every story row of our test runs (4,074 stories across all modes), and their names will not change: `record_type`, `id`, `title`, `url`, `author`, `created_at`, `created_at_unix`, `type`, `hn_url`, `source`, `position`, `scraped_at`. Every comment row (5,601 in our tests) had `record_type`, `story_title`, `id`, `parent_id`, `story_id`, `author`, `text`, `created_at`, `hn_url`, `source` and `scraped_at`; comments read from a thread also had `depth`, `reply_count` and `thread_position`. Optional, present only when Hacker News has a value: `points` and `comment_count` (job posts have no comments), `text` on stories (only text posts have a body), `rank` (list modes only), `depth` on comments found by search, and `domain` when a link cannot be read as a web address. #### Output examples The first story of the front page (test run, 21 Oct 2026): ```json { "record_type": "story", "id": 50170593, "title": "SimCity, but with Permits", "url": "https://permit.petercai.com/", "domain": "permit.petercai.com", "points": 366, "author": "pcai", "comment_count": 196, "created_at": "2026-10-20T18:29:09.000Z", "created_at_unix": 1792520949, "type": "story", "hn_url": "https://news.ycombinator.com/item?id=50170593", "rank": 1, "source": "top", "source_query": "top", "position": 1, "scraped_at": "2026-10-20T22:15:36.690Z" } ``` A reply two levels deep in the Dropbox launch thread, from item mode (same day, text trimmed): ```json { "record_type": "comment", "id": 9272, "parent_id": 9224, "story_id": 8863, "story_title": "My YC app: Dropbox - Throw away your USB drive", "author": "dhouston", "text": "1. re: the first part, many people want something plug and play. and even if they were plug and play, the problem is that the user experience (on windows at least) with online drives generally sucks...", "created_at": "2007-04-05T16:47:01.000Z", "created_at_unix": 1175791621, "depth": 2, "reply_count": 1, "thread_position": 2, "hn_url": "https://news.ycombinator.com/item?id=9272", "source": "item", "source_query": "8863" } ``` The summary row of a relevance search for "claude code" with 5 comments per story, trimmed: ```json { "_type": "summary", "mode": "search", "sources_read": 1, "stories_delivered": 30, "comments_delivered": 150, "items_delivered": 180, "comment_threads_cut_at_cap": 30, "http_requests": 241, "run_seconds": 7, "per_source": [ { "source": "claude code", "pages": 1, "found": 50, "delivered": 30, "rows_delivered": 180, "stopped": "max_items_reached", "total_matches": 347 } ] } ``` An error row for an id that does not exist (nothing charged): ```json { "_type": "error", "error_type": "not_found", "identifier": "9999999999", "reason": "Hacker News has no such item.", "http_status": 404 } ``` ### Pricing Pay per event: you pay for the rows you receive, plus a flat $0.003 per run, whatever memory you choose. Story rows and comment rows cost the same. | Event | When it is charged | Free plan | Bronze (Starter) | Silver (Scale) | Gold (Business) | |---|---|---|---|---|---| | `item-scraped` | Once per story or comment row stored in your dataset | $0.18 per 1,000 | $0.18 per 1,000 | $0.18 per 1,000 | $0.18 per 1,000 | | `run-start` | Once per run, after the first page returned rows | $0.003 | $0.003 | $0.003 | $0.003 | The price of one row is the price per 1,000 divided by 1,000: all 500 top stories with 50 comments each, about 25,500 rows, cost about $4.59. Never charged: - inputs that are not a Hacker News id, link, username or date (each gets an error row); - a story or comment that a second keyword of the same run finds again; - items an earlier monitor run already delivered; - deleted and flagged comments (no row is written for them); - rows past your `maxItems` or `maxCommentsPerStory`; - the summary, info and error rows; - a run with no valid input, or one that finds nothing: the start fee is charged only once a page has returned rows; - anything past your "Max cost per run". The actor checks the budget before each batch of rows and stops while the total is still below your limit, so the run ends SUCCEEDED with a partial result and a status message saying so. ### Run it on a schedule 1. Open the actor in Apify Console, set your input, turn on `monitorMode` and click **Save as a new task**. 2. In the task, open **Schedules**, click **Add schedule** and pick how often (every hour, every morning at 08:00 and so on). 3. Set `maxItems` above the number of new items you expect between two runs; each run checks that many of the newest items and returns the ones not delivered before. 4. Optional: add an integration (Slack, Google Sheets, a webhook) so each new mention lands where your team works. ### FAQ #### What does this actor read? Hacker News's two free public APIs: the official Firebase API (hacker-news.firebaseio.com) for the live lists, items, users and comment threads, and the Algolia HN Search API (hn.algolia.com) for keyword, date and user searches. It does not log in, vote, post or read anything that needs an account. #### How far back can a search go? To the first Hacker News posts of October 2006. With newest first (the default), the actor pages backwards through time, so one keyword can return up to 10,000 stories per run and more over several runs with `dateTo`. Most relevant first stops at 1,000 per keyword, a limit of the search API. #### Why does a story say 196 comments and I got fewer? `comment_count` counts every comment on the story. You get at most `maxCommentsPerStory` rows, only `maxCommentDepth` levels deep, and no rows for deleted or flagged comments. Raise both limits to get the whole thread; when a thread is cut, the higher levels are kept first and the summary counts it in `comment_threads_cut_at_cap`. #### How does monitor mode decide what is new? It remembers the id of every story and comment it delivered, per watch: the same mode with the same keywords, users or items and the same `searchIn` or `userContent`. Two tasks with different keywords keep separate memories. Each run checks the first `maxItems` items of each list or search and returns those not delivered before. Changing `maxItems`, dates or comment settings keeps the memory; changing the keywords starts a new one. #### Are the points and comment counts live? List and item modes read them straight from Hacker News at run time. Search and user modes read them from the Algolia index, which can lag the site by a few minutes for very new posts. #### How long does a run take? In our tests: 3 seconds for the "Try it" input (50 front page stories), 8 seconds for all 500 top stories, 6 seconds for 2,500 search results, and 47 seconds for 200 best stories with 5,021 comments. The default time limit is 3,600 seconds; the actor stops before it and keeps every row. #### Do I need an account, an API key or a proxy? No. Both APIs are free and keyless. If the search API ever answers "too many requests", the actor moves its search requests to Apify Proxy on its own, at no cost to you. #### Which formats can I download? JSON, CSV, Excel, XML, RSS and HTML from the run's Storage tab or through the API. The dataset has three ready views: Stories, Comments and Links. #### Can I use it from Claude, ChatGPT or Cursor (MCP)? Yes, through Apify's MCP server. Connector URL: https://mcp.apify.com/?tools=themineworks/hacker-news-scraper - Claude: Settings > Connectors > Add custom connector, paste the URL, sign in with Apify. - ChatGPT: turn on developer mode, add an MCP connector with the URL, sign in with Apify. - Cursor or VS Code: add it as an HTTP MCP server with that URL. - Claude Code: `claude mcp add -t http hacker-news-scraper "https://mcp.apify.com/?tools=themineworks/hacker-news-scraper"` #### Is it legal to scrape Hacker News? The actor reads only public posts through the APIs Hacker News and Algolia publish for this purpose. Comments are written by people and carry usernames; you are responsible for how you use the data, including data protection law such as GDPR and CCPA. This is general information, not legal advice. ### Integrations - **Slack and email**: schedule a monitor task and send each run's new rows to a channel with Apify's Slack integration or a webhook, for example "every new mention of our product on Hacker News". - **Google Sheets**: send stories to a sheet; one row per story fits a sheet as is. - **Make, Zapier and n8n**: start a run and read its dataset with the Apify apps and nodes. - **API and SDKs**: run it from Python or JavaScript with the Apify client, or with a plain HTTP call (see "Copy to your AI assistant" above). - **MCP clients**: Claude, ChatGPT, Cursor and VS Code can run it as a tool through the connector URL in the FAQ. ### More from The Mine Works **Social media and video** - [Threads Scraper](https://apify.com/themineworks/threads-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Reddit Scraper](https://apify.com/themineworks/reddit-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Threads Search Scraper](https://apify.com/themineworks/threads-search-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Instagram Profile Scraper](https://apify.com/themineworks/instagram-profile-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Instagram Followers & Following](https://apify.com/themineworks/instagram-followers-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Reddit Search Scraper](https://apify.com/themineworks/reddit-search-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Twitter / X Scraper](https://apify.com/themineworks/twitter-x-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [YouTube Transcript](https://apify.com/themineworks/youtube-transcript-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Xiaohongshu (RED) Scraper](https://apify.com/themineworks/xiaohongshu-explore-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Telegram Channel Scraper](https://apify.com/themineworks/telegram-channel-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Telegram Channel Finder](https://apify.com/themineworks/telegram-channel-finder?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Pinterest Profile Scraper](https://apify.com/themineworks/pinterest-profile-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Leads and business directories** - [B2B Leads Finder](https://apify.com/themineworks/b2b-leads-finder?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Skip Trace Lookup](https://apify.com/themineworks/skip-trace-lookup?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Google Maps Email Scraper](https://apify.com/themineworks/maps-leads?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [2GIS Places Scraper](https://apify.com/themineworks/2gis-places-search?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Marketing, SEO and reviews** - [Facebook Ad Library Scraper](https://apify.com/themineworks/meta-ad-library-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Google Ads Transparency Scraper](https://apify.com/themineworks/google-ads-transparency?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Similarweb Scraper](https://apify.com/themineworks/similarweb-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Google News Scraper](https://apify.com/themineworks/google-news?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **LinkedIn** - [LinkedIn Company Scraper](https://apify.com/themineworks/linkedin-company-details?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [LinkedIn Post Scraper](https://apify.com/themineworks/linkedin-post-search?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [LinkedIn Employees Scraper](https://apify.com/themineworks/linkedin-employees?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [LinkedIn Profile Scraper](https://apify.com/themineworks/linkedin-profile-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Real estate** - [Zillow Rentals Scraper](https://apify.com/themineworks/zillow-rental-listings?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Zillow Sold Comps Scraper](https://apify.com/themineworks/zillow-recently-sold?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [India Real Estate MCP](https://apify.com/themineworks/india-real-estate-mcp?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Housing.com Scraper](https://apify.com/themineworks/housing-com-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Science, health and government data** - [Academic Research MCP](https://apify.com/themineworks/academic-research-mcp?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [CourtListener Scraper](https://apify.com/themineworks/courtlistener-court-records?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Socrata Open Data Scraper](https://apify.com/themineworks/socrata-open-data?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [OpenAlex Scraper](https://apify.com/themineworks/openalex-scholarly-works?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Jobs and hiring** - [Foundit Monster India Jobs](https://apify.com/themineworks/foundit-jobs-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Hirist Jobs Scraper](https://apify.com/themineworks/hirist-jobs-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [India Jobs MCP](https://apify.com/themineworks/india-jobs-mcp?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Naukri Jobs Scraper](https://apify.com/themineworks/naukri-jobs?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Company and business data** - [Company Domain Finder](https://apify.com/themineworks/company-domain-finder?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [GST Taxpayer Lookup](https://apify.com/themineworks/gst-taxpayer-lookup?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Company KYB Resolver](https://apify.com/themineworks/company-identity-resolver?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [SEC EDGAR Filings Scraper](https://apify.com/themineworks/sec-edgar-filings?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **E-commerce and marketplaces** - [Ozon.ru Scraper](https://apify.com/themineworks/ozon-product-search?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [⭐ Amazon Reviews Scraper](https://apify.com/themineworks/amazon-reviews?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Amazon Product Scraper](https://apify.com/themineworks/amazon-products?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Etsy Listings Scraper](https://apify.com/themineworks/etsy-search-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Food and local services** - [NoBroker Scraper](https://apify.com/themineworks/nobroker-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Swiggy Restaurant Scraper](https://apify.com/themineworks/swiggy-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Zomato Scraper](https://apify.com/themineworks/zomato-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **Developer and AI tools** - [Website to Markdown Crawler](https://apify.com/themineworks/rag-crawler?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [GitHub Skill Finder](https://apify.com/themineworks/github-skill-discovery?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [GitHub Repo Scraper](https://apify.com/themineworks/github-repo-intelligence?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [GitHub Trending Scraper](https://apify.com/themineworks/github-trending-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) **More tools** - [G2 Reviews Scraper](https://apify.com/themineworks/g2-reviews-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Naver Blog Scraper](https://apify.com/themineworks/naver-blog-posts-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [WeChat Article Scraper 微信公众号文章](https://apify.com/themineworks/wechat-article-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) - [Taobao Products Scraper 淘宝 天猫](https://apify.com/themineworks/taobao-products-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral) ### Support Found a story or thread that comes back with fewer rows than the site shows, or a field that is empty? Open an issue in the Issues tab with the input and the run ID, and we will look at it. For a field you need or a new source, email dmineworks@gmail.com. *Get Hacker News stories and comments as clean rows: the front page and every live list, keyword search with dates and minimum points, users and full threads, and only new items on scheduled runs.* # Actor input Schema ## `mode` (type: `string`): Top, New, Best, Ask HN, Show HN and Jobs read the live list in the order the site shows it. Search uses the keywords and dates below. User reads the usernames below. Item reads the stories or comments in Item ids with the comments under them. ## `keywords` (type: `array`): Words to search for in titles, links, story text and comments, one search per line. Example: rust, postgres, claude code. All words of a line must match. Leave empty and set a date range or minimum points to list every story in that range. ## `searchIn` (type: `string`): What the keyword search returns. Comments come back as comment rows with the story they belong to. ## `sortBy` (type: `string`): Newest first can go back as far as Hacker News goes. Most relevant first is Algolia's ranking and stops at 1,000 results per keyword. ## `dateFrom` (type: `string`): First day to include, YYYY-MM-DD (UTC), or a relative date such as 7 days (meaning 7 days ago). Leave empty for no lower limit. ## `dateTo` (type: `string`): Last day to include, YYYY-MM-DD (UTC), the whole day counts, or a relative date such as 1 day (meaning up to 1 day ago). Leave empty for up to now. ## `minPoints` (type: `integer`): Only stories with at least this many points, for example 100. Leave empty for no limit. ## `minComments` (type: `integer`): Only stories with at least this many comments. Leave empty for no limit. ## `usernames` (type: `array`): Hacker News usernames or profile links, one per line, for example pg or https://news.ycombinator.com/user?id=dang. Names are case sensitive. ## `userContent` (type: `string`): Which of the user's submissions to return, newest first. ## `itemIds` (type: `array`): Story or comment ids or links, one per line, for example 8863 or https://news.ycombinator.com/item?id=8863. Each gives its own row and the comments under it. ## `maxItems` (type: `integer`): The most stories (or comments, when you search comments) per list, keyword or user, 1 to 10,000. Top and New hold 500 stories, Best 200. Comments under stories do not count here; see Max comments per story. In monitor mode this is the window: the first this many items of each list or search are checked and only the new ones are returned. ## `includeComments` (type: `boolean`): On: each story is followed by its comments as separate rows, in the order the thread shows them, with depth, parent and story id. Always on in item mode. ## `maxCommentDepth` (type: `integer`): How many levels of replies to read. 1 = only direct replies to the story, 2 = those and the replies to them, and so on. ## `maxCommentsPerStory` (type: `integer`): The most comment rows under one story or item, 1 to 10,000. When a thread is larger, the higher levels are kept first. ## `monitorMode` (type: `boolean`): For scheduled runs: return only stories and comments that no earlier monitor run with the same mode and the same keywords, users or items delivered. Items skipped this way are not charged. Example: a keyword search for your product name every hour. ## Actor input object example ```json { "mode": "top", "keywords": [ "open source" ], "searchIn": "story", "sortBy": "date", "userContent": "stories", "maxItems": 50, "includeComments": false, "maxCommentDepth": 5, "maxCommentsPerStory": 100, "monitorMode": false } ``` # Actor output Schema ## `results` (type: `string`): No description ## `summary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "top", "keywords": [ "open source" ], "maxItems": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("themineworks/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "top", "keywords": ["open source"], "maxItems": 50, } # Run the Actor and wait for it to finish run = client.actor("themineworks/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "top", "keywords": [ "open source" ], "maxItems": 50 }' | apify call themineworks/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,themineworks/hacker-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/lP6YouUHhJ3EA0xCI/builds/STlabawQqfxPoziNQ/openapi.json