Hacker News Data Scraper

🚀 What is Hacker News Data Scraper?

Hacker News Data Scraper is an Apify Actor that collects stories from Hacker News by keyword or by browse mode. Each row holds the title, link, author, points, comment count and posting date, with optional comments and article descriptions. Use this Hacker News scraper instead of refreshing the front page by hand.

Tracking Hacker News by refreshing the front page misses the best threads. A search across all stories, or a browse mode on a schedule, keeps every post you care about in one dataset.

• 🔎 Keyword search: run a Hacker News search for any word or phrase and sort the results by relevance or by date.
• 🧭 Six browse modes: FRONTPAGE, NEWEST, ASK, SHOW, JOBS and BEST work without a keyword.
• 💬 Comments on demand: fetch comments for each story, or skip them for a faster run.
• 📰 Article descriptions: add a short description of each linked article to the text field when one is available.
• 📅 Date filters: use 7d, 30d, 6m, 1y or a custom start and end date.

Need a Hacker News web scraper without any setup? Open the Actor, enter a keyword and press Start. Looking for a Hacker News extractor that exports to spreadsheets? The dataset downloads as JSON, CSV or Excel.

📊 What data can you extract from Hacker News?

Each row in the dataset is one of the Hacker News stories you asked for, and the fields below are ready to export as JSON, CSV or Excel.

Field	Description	Example
id	Hacker News story ID.	45751400
type	story, ask_hn, show_hn, or job.	story
title	Story headline.	UV is the best thing to happen to Python
url	External link (null for self-posts).	https://example.com/article
domain	Extracted domain from the URL.	example.com
text	Post body text or article meta description.	A deep dive into...
author	HN username of the submitter.	todsacerdoti
points	Upvote score.	2214
num_comments	Total number of comments on HN.	1324
created_at	ISO 8601 submission timestamp.	2025-10-29T18:57:29Z
age_hours	Story age in decimal hours from scrape time.	8808.7
points_per_hour	Points divided by age_hours.	0.25
comments	Fetched comments (filled when maxCommentsPerPost is above 0).	[]

Hacker News points and num_comments show how much attention a story received. Hacker News comments are fetched only when maxCommentsPerPost is above 0, and each comment carries id, author, text, created_at and depth. The Actor caps the number of comments returned per story.

Hacker News posts without a link, such as Ask posts, have url set to null. Use the domain column to see Hacker News by domain.

The Overview view in the Output tab lists the main fields. The With Comments view adds the fetched comments.

💡 Use cases for Hacker News data

• Watch your product name: set up Hacker News keyword monitoring with queries, sortBy set to date and dateRange set to 7d, then read title, points and num_comments.
• Follow a technology: track Hacker News mentions of a framework or company by searching it on a schedule and comparing created_at and num_comments.
• Spot rising stories: points_per_hour and age_hours show which stories gain upvotes fastest, which helps when you check Hacker News trending topics.
• Study a topic over time: collect a year of stories with dateRange set to 1y for Hacker News trend analysis in your own spreadsheet or notebook.
• Keep an eye on rivals: search competitor names for Hacker News competitor monitoring and read title, domain and author.
• Build a prospect list: run SHOW or JOBS and collect author, url and domain as a starting point for Hacker News lead generation.
• Read the mood: set maxCommentsPerPost and export the comments text for Hacker News sentiment analysis in your own tools.
• Curate a digest: run BEST with dateRange set to 7d to shortlist stories for a Hacker News newsletter.
• Review hiring posts: JOBS mode returns Hacker News job postings with title, url and text.
• Feed a reader app: combine several queries and modes in one run to power a tech news aggregator of your own.
• Size up a market: search a product category for Hacker News market research and compare points across the results.
• Learn from launches: SHOW mode lists Hacker News project launches with author, domain and points.
• Find what resonates: read title and text from a keyword search for Hacker News content research on how people phrase a subject.
• Follow submitters: export author with points and created_at to study Hacker News author activity.
• Build a corpus: keep created_at, points and comments from many queries as a Hacker News research dataset for analysis.

🛠️ How to scrape Hacker News in six steps

1. Create a free Apify account and open Hacker News Data Scraper.
2. Enter one keyword or mode per line in Search Queries or Modes, for example python or FRONTPAGE.
3. Set Max Results per Query and keep it small for a first test.
4. Choose Sort Order, Date Range, Max Comments per Story and Fetch Article Description if you need them.
5. Click Start and wait for the run to finish.
6. Download your Hacker News data as JSON, CSV or Excel.

Hacker News date filter for recent stories

Enter a shortcut such as 7d, 30d, 6m or 1y in dateRange, or a custom range such as 2024-01-01,2024-12-31. Leave it empty to cover all time.

Hacker News to Excel

Open the finished dataset, choose Export and pick Excel. The same menu offers CSV and JSON, so Hacker News to CSV takes one more click.

💰 How much does it cost to scrape Hacker News?

Hacker News Data Scraper uses pay per event pricing: every result is one billed event, and the price per 1,000 results depends on your Apify plan.

Apify plan	Price per 1,000 results
Free	$1
Bronze	$0.9
Silver	$0.8
Gold, Platinum, Diamond	$0.7

For example, 1,000 results cost $1 in the Free row and $0.7 in the Gold, Platinum and Diamond row. Each run start also adds one small Actor Start event of $0.00005. See the Pricing tab for current rates.

⬇️ Input

Only queries is required: a list of keywords or browse modes, one entry per line. See the Input tab for every option.

Example input:

{
  "queries": ["python"],
  "maxResults": 10,
  "sortBy": "relevance",
  "dateRange": "",
  "maxCommentsPerPost": 0,
  "fetchArticleText": true
}

Field	Type	Default	Description
queries	array of strings	["python"] (pre-filled)	Keywords to search, or the mode names FRONTPAGE, NEWEST, ASK, SHOW, JOBS, BEST. Required.
maxResults	integer	10	Maximum stories per query or mode, from 0 to 50000. Set 0 for no limit.
sortBy	string	relevance	relevance or date (newest first). Ignored for browse modes.
dateRange	string	empty	7d, 30d, 6m, 1y or YYYY-MM-DD,YYYY-MM-DD. Empty means all time.
maxCommentsPerPost	integer	0	Comments to fetch per story (input range 0 to 5000; the Actor caps how many are returned per story). Set 0 to skip comments.
fetchArticleText	boolean	true	Add a short description from each linked article to text when available.

Browse modes

Mode	What it returns
FRONTPAGE	Stories on the Hacker News front page
NEWEST	The Hacker News newest submissions
ASK	Ask HN posts
SHOW	Show HN posts
JOBS	Hacker News jobs listings
BEST	Stories returned by the BEST mode

⬆️ Output

The scraped results are shown as a dataset in the Storage tab. The table view is for convenience and doesn't show all the fields.

Your Hacker News dataset can be downloaded as JSON, CSV or Excel. Use points_per_hour to compare Hacker News engagement rate across stories of different ages.

Example output (values taken from the dataset schema, not from a captured run):

{
  "id": 45751400,
  "type": "story",
  "title": "UV is the best thing to happen to Python",
  "url": "https://example.com/article",
  "domain": "example.com",
  "text": "A deep dive into...",
  "author": "todsacerdoti",
  "points": 2214,
  "num_comments": 1324,
  "created_at": "2025-10-29T18:57:29Z",
  "age_hours": 8808.7,
  "points_per_hour": 0.25,
  "comments": []
}

At the end of every run a RUN_SUMMARY record in the key-value store lists the total items, the queries, the status and the duration.

⚠️ Limitations and error handling

• Each query or mode returns at most the number set in maxResults, and 50000 is the highest value the input accepts.
• The Actor caps the number of comments returned per story, so a high maxCommentsPerPost does not guarantee that many comments.
• Only public stories and comments are collected. No login is used.
• Many fields can be empty (null), for example url on self-posts, so handle missing values in your own tools.
• An article description is added only when the linked page provides one. A story is still returned when it cannot be added.
• If queries is empty, the run stops with a message asking for at least one keyword or mode.
• If some queries cannot be completed, the run reports partial results in its status message and in RUN_SUMMARY.
• If you set a maximum spend per run, the Actor stops cleanly once that limit is reached.
• A query with no matches returns no rows.

🔌 Integrating Hacker News Data Scraper with other apps

Hacker News Data Scraper runs like any Apify Actor, so you can start it from the Apify API, a schedule or a webhook. See the API tab for ready-made code snippets.

• Schedules: use Apify schedules to repeat a run daily or weekly. Set sortBy to date and dateRange to 7d so each weekly run covers the latest week. Each run is independent and has no "only new stories" option.
• Webhooks: attach an Apify webhook to a scheduled run to build Hacker News alerts in the tools you already use.
• Integrations: send the dataset to spreadsheets and automation tools through the Apify integrations.
• API: if your code needs a Hacker News API, call the Actor through the Apify API: start a run, then read the dataset items as JSON.

🔗 More Apify Actors by crawl_craft

Actor	What it extracts	When to use it
Quora Scraper	Questions with view and answer counts, answers with text, upvotes and dates, author profiles, spaces and topics	Use it when you also want question-and-answer content on a topic, which Hacker News Data Scraper does not collect
Telegram Scraper	Messages, views, reactions, media, links, polls, comments and subscriber counts from public Telegram channels	Use it when your topic tracking also covers public Telegram channels
Facebook Posts Search Scraper	Public Facebook posts found by keyword or hashtag, with reactions, comments, shares and media	Use it to follow the same keyword on Facebook

⚖️ Is it legal to scrape Hacker News?

Hacker News Data Scraper collects only stories and comments that are publicly visible on Hacker News. It does not log in and does not collect private data such as email addresses. Results can still contain personal data, such as usernames in author, so collect them only with a legitimate reason and consider GDPR (Regulation (EU) 2016/679) and the site's terms. Consult a lawyer if you are unsure. For background, read Apify's posts on the legality of web scraping and ethical web scraping.

❓ FAQ

Do I need a Hacker News API key?

No. The Actor needs no key and no login. You only enter keywords or browse modes.

How much does Hacker News scraping cost?

Each result is billed as one event, at $1 per 1,000 results in the Free row and down to $0.7 per 1,000 in the top row of the table above. See the Pricing tab for current rates.

How many results can I get per query?

Set maxResults up to 50000 per query or mode, or enter 0 for no limit. Start with a small number to test your query.

How do I get Hacker News top stories?

Enter FRONTPAGE in queries for the stories on the front page. BEST returns the stories of the BEST mode, and dateRange narrows either one to recent weeks.

What is the difference between FRONTPAGE and NEWEST?

FRONTPAGE returns stories on the front page, while NEWEST returns the latest submissions. Both work without a keyword.

Can I collect comments?

Yes. Set maxCommentsPerPost above 0 and the comments field is filled for each story. Leave it at 0 to skip them.

Can I search by keyword instead of using a mode?

Yes. Any entry in queries that is not a mode name is searched as a keyword across stories, sorted by sortBy.

Can I scrape private data?

No. Only public stories and comments are collected.

💬 Your feedback

Found a bug or missing a field? Open a ticket in the Issues tab of this Actor. For a custom solution, such as extra fields or a tailored workflow, write to hello.crawlcraft@gmail.com.