Hacker News Data Scraper

📰 Turn Hacker News into usable data

Hacker News Scraper is for developers, researchers, founders, and news curators who need public Hacker News stories and posts in structured dataset rows. Choose a feed, topic, user, date, or listing to collect titles, links, authors, scores, ranks, publication times, and optional top-level comments for analysis or reuse.

• Collect a ranked front-page snapshot with Hacker News top stories.
• Find stories about a topic with Hacker News search and review their titles, links, and scores.
• Read a known discussion with Hacker News comments when top-level comments matter.
• Review public hiring posts with Hacker News job listings.
• Open a known item with Hacker News scraper when you have its ID or URL.

📊 See the Hacker News fields you need

Each saved row represents a public Hacker News story or post found by the selected discovery method. The fields keep source values such as the item ID, title, publication time, author, score, and comment count. A topic search can also return the outbound URL and result rank. You can turn on top-level public comments when you need their author, text, posting time, and reply IDs.

▶️ Choose how to find Hacker News stories

Choose one discovery method for each run. Fill only the fields for that method because values in other method sections are ignored.

1. Use a Standard feed for the Front page, Top stories, Newest stories, Best stories, Ask HN, Show HN, or Jobs feed.
2. Use Topic search for a full-text topic or phrase. Add an order, date range, minimum score, or minimum comment count when needed.
3. Use User submissions to collect public stories submitted by one Hacker News username.
4. Use Historical content to collect public stories for a start date and an optional end date.
5. Use Specific listings to submit one or more Hacker News item IDs or public item URLs.

Run a small collection first when you want to check the fields. Set Maximum stories to a positive number to stop after that many stories. Leave it empty to collect all available results until the source is exhausted.

⚙️ Input

The public input form has one discovery method and matching options for each method.

Input fields

Field	Type	What it does
discoveryMethod	select	Chooses feed, search, user, date, or listing. Use one method per run.
feed	select	Chooses the public feed: front, top, new, best, ask, show, or jobs. Used only for feed.
searchQuery	string	Sets the full-text topic or phrase. Used only for search.
searchOrder	select	Orders topic results by relevance or newest.
searchStartDate	date string	Sets the first publication date for topic search in YYYY-MM-DD format. Leave it empty for no start-date filter.
searchEndDate	date string	Sets the last publication date for topic search in YYYY-MM-DD format. Leave it empty for no end-date filter.
minScore	integer	Keeps topic-search stories with at least this many points. Leave it empty for no score filter.
minComments	integer	Keeps topic-search stories with at least this many comments. Leave it empty for no comment filter.
userHandle	string	Sets one Hacker News username whose public story submissions should be collected. Used only for user.
historyStartDate	date string	Sets the first publication date for historical content in YYYY-MM-DD format. It is required for date.
historyEndDate	date string	Sets the last publication date for historical content in YYYY-MM-DD format. Use the same date as historyStartDate for one day.
listingTargets	array of strings	Adds one or more public Hacker News item IDs or item URLs. Used only for listing.
includeComments	boolean	Includes available top-level public comments with their author, text, posting time, and reply relationships.
maxItems	integer	Stops after this many stories. Leave empty to collect all available results until the source is exhausted.

Input example

This is the public input from a successful default-input QA run:

{
  "discoveryMethod": "feed",
  "feed": "front",
  "searchOrder": "relevance",
  "includeComments": false,
  "maxItems": 100
}

🧾 Output

The dataset output links to the collected Hacker News rows. Use the dataset after the run or retrieve it through the Apify API.

Run output

Field	Type	What it does
dataset	URL	Links to the collected Hacker News items in the dataset.

Standard Hacker News rows

Rows from feeds, topic search, user submissions, historical content, and specific listings use this shape. Fields marked optional are included when Hacker News supplies them for the item or selected method.

Field	Type	What it does
id	integer	Stable Hacker News item ID.
type	string	Source-provided item type, such as story or job.
title	string	Title supplied by Hacker News.
publishedAt	date-time string	Publication time supplied by Hacker News in ISO 8601 format.
text	string, optional	Public post text supplied by Hacker News when available. The source may format it as HTML.
url	URL, optional	Outbound URL supplied by Hacker News when available.
author	string, optional	Hacker News username supplied for the item when available.
score	integer, optional	Current score supplied by Hacker News when available.
commentCount	integer, optional	Current number of comments supplied by Hacker News when available.
rank	integer, optional	Position in the selected feed or topic-search results when available.

Example standard row

This is a genuine row from a successful current-beta front-page run.

{
  "id": 49554643,
  "type": "story",
  "title": "GPT-6 Astra",
  "publishedAt": "2026-09-03T18:41:05.000Z",
  "text": "System Card: <a href=\"https:&#x2F;&#x2F;deploymentsafety.openai.com&#x2F;gpt-6-astra\" rel=\"nofollow\">https:&#x2F;&#x2F;deploymentsafety.openai.com&#x2F;gpt-6-astra</a><p>Related ongoing threads:<p><i>OpenAI&#x27;s GPT-6 Astra on ARC-AGI-3</i> - <a href=\"https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49555691\">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49555691</a><p><i>GPT-6 Astra makes major gains in the Artificial Analysis Coding Agent Index</i> - <a href=\"https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49556147\">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49556147</a>",
  "url": "https://openai.com/index/gpt-6-astra/",
  "author": "kibae",
  "score": 665,
  "commentCount": 393,
  "rank": 1
}

Rows with top-level comments

When includeComments is on, the row keeps the standard fields and adds available top-level public comments. Each comment can include direct reply IDs. Replies are linked by ID and are not rewritten into a second row shape.

Field	Type	What it does
id	integer	Stable Hacker News item ID.
type	string	Source-provided item type, such as story or job.
title	string	Title supplied by Hacker News.
publishedAt	date-time string	Publication time supplied by Hacker News in ISO 8601 format.
text	string, optional	Public post text supplied by Hacker News when available. The source may format it as HTML.
url	URL, optional	Outbound URL supplied by Hacker News when available.
author	string, optional	Hacker News username supplied for the item when available.
score	integer, optional	Current score supplied by Hacker News when available.
commentCount	integer, optional	Current number of comments supplied by Hacker News when available.
rank	integer, optional	Position in the selected feed or topic-search results when available.
comments	array	Available top-level public comments when comment collection is enabled.
comments[].id	integer	Stable Hacker News comment ID.
comments[].author	string, optional	Hacker News username supplied for the comment when available.
comments[].text	string, optional	Public comment text supplied by Hacker News when available. The source may format it as HTML.
comments[].postedAt	date-time string, optional	Comment posting time supplied by Hacker News in ISO 8601 format when available.
comments[].replies	array of integers, optional	IDs of available direct replies to the comment.

Example row with comments

The comments list below is shortened. The first comment and all other shown values are genuine. The string "..." marks omitted comments.

{
  "id": 8863,
  "type": "story",
  "title": "My YC app: Dropbox - Throw away your USB drive",
  "publishedAt": "2007-04-04T19:16:40.000Z",
  "url": "http://www.getdropbox.com/u/2/screencast.html",
  "author": "dhouston",
  "score": 104,
  "commentCount": 71,
  "comments": [
    {
      "id": 9224,
      "author": "BrandonM",
      "text": "I have a few qualms with this app:<p>1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem.  From Windows or Mac, this FTP account could be accessed through built-in software.<p>2. It doesn't actually replace a USB drive.  Most people I know e-mail files to themselves or host them somewhere online to be able to perform presentations, but they still carry a USB drive in case there are connectivity problems.  This does not solve the connectivity issue.<p>3. It does not seem very \"viral\" or income-generating.  I know this is premature at this point, but without charging users for the service, is it reasonable to expect to make money off of this?",
      "postedAt": "2007-04-05T15:16:54.000Z",
      "replies": [
        9272
      ]
    },
    "..."
  ]
}

💳 Pricing

Saved-row charge

The Actor uses pay-per-event pricing. One event is charged for each successfully collected public story or post saved to the dataset. A run that saves no story or post does not create this item event. The current rate for your Apify plan is shown in the Pricing tab.

Use maxItems to bound the number of saved stories. Leave it empty to collect all available results until the source is exhausted.

🔌 Integrations

Dataset access

Open the dataset output after a run, or use the Apify API to retrieve the rows in your workflow.

❓ FAQ

Can I narrow a topic search by date, score, or comment count?

Yes. Use searchStartDate and searchEndDate for a publication-date range, or use minScore and minComments for topic-search thresholds. These fields apply only to Topic search.

Can I collect one known Hacker News item?

Yes. Choose Specific listings and add an item ID such as 8863 or a public item URL. You can add more than one target in the same list.

What does including comments add?

It adds available top-level public comments with their author, text, posting time, and direct reply IDs. The replies stay linked under their parent comment, and unavailable source values are not filled in.

What happens when an item has no score or comments?

Those fields are optional. When Hacker News does not provide a value for an item, the field is absent from that row rather than filled with a made-up value.

Can I use several discovery methods in one run?

No. Choose one method for each run. Values in the other option sections are ignored.

Do I need a Hacker News login or API key?

No. The Actor collects public Hacker News data without buyer-supplied source credentials.

Does the Actor scrape the websites linked from Hacker News?

No. It collects public Hacker News stories and posts. It does not fully extract content from outbound websites.

What does leaving Maximum stories empty do?

It returns all available results until the source is exhausted. Set a positive value when you want a bounded run.

📝 Changelog

v0.0 (03-09-2026)

• Initial release.

🆘 Support

For issues, questions, or feature requests, file a ticket and I'll fix or implement it in less than 24h 🫡

🔗 Related Actors

• Reddit Scraper collects public Reddit posts, comments, communities, and user data when you need a broader community source.
• Reddit Comments Search Scraper searches public discussion comments by keyword when comment text is your focus.
• Reddit Historical Archive Scraper for Posts and Comments finds dated public posts and comments when you need historical context.
• GitHub Discussions Scraper collects public developer discussions, replies, and reactions for product feedback.
• Product Hunt Scraper collects public product launches, topics, and engagement around new tools.

Made with ❤️ by Maxime Dupré