🟠 Hacker News Scraper: Story, Comment & User Search by PunkRecordsData

Hacker News Scraper is an Apify actor that searches Hacker News stories, comments and user profiles by keyword and returns structured rows with points, comment counts, dates and direct links. Filter by type (stories, Ask HN, Show HN, jobs, polls, front page), minimum points and date range. It needs no login and no API key. Export to CSV, Excel, JSON or XML. Story rows cost $12 per 1,000 on the free tier.

A search for "web scraping" matches 2,083 stories and 9,585 comments in the Hacker News index (measured on 2026-10-04). This actor turns any topic into a ranked, dated dataset of what the Hacker News community said about it, using the public Hacker News Search API by Algolia.

🔎 What does the Hacker News Scraper do?

• Story search: title, external URL, author, points, comment count, post type, self text (when the post has one), creation date and exact Hacker News link.
• Type filters: stories, Ask HN, Show HN, jobs, polls, or the current front page.
• Signal filters: minimum points and a date range, so you only pay for stories that mattered.
• Comment search: full-text search across comments with author, parent story title, parent story ID and date.
• User profiles: karma, about text and account creation date for any list of usernames.
• Two sort modes: relevance (popularity-weighted) or newest first, for monitoring new mentions.

📊 What data does the Hacker News Scraper return?

Each row has a recordType of story, comment or user. Optional fields are left out when Hacker News has no value for them, so the dataset has no empty placeholder columns. Real output from a run with the query "web scraping":

{
  "recordType": "story",
  "hnId": "22180559",
  "title": "Congrats! Web scraping is legal! (US precedent)",
  "url": "https://parsers.me/us-court-fully-legalized-website-scraping-and-technically-prohibited-it/",
  "author": "ehurynovich",
  "points": 1057,
  "commentCount": 388,
  "storyType": "story",
  "createdAt": "2020-01-29T14:06:37Z",
  "hnUrl": "https://news.ycombinator.com/item?id=22180559",
  "scrapedAt": "2026-10-04T05:05:39.151Z"
}

A comment row from the same index:

{
  "recordType": "comment",
  "hnId": "9997773",
  "author": "smt88",
  "text": "If someone wants you to do webscraping, even for a good cause, I wouldn't waste my time if I were you. Most AWS IPs have been long-since blacklisted by the most interesting sites (e.g. craigslist).",
  "storyTitle": "Ask HN: $8k on AWS in August for you, any interesting computations?",
  "storyId": "9997643",
  "createdAt": "2015-08-03T17:05:26Z",
  "hnUrl": "https://news.ycombinator.com/item?id=9997773",
  "scrapedAt": "2026-10-04T05:05:55.572Z"
}

A user row:

{
  "recordType": "user",
  "username": "pg",
  "karma": 157316,
  "about": "Bug fixer.",
  "hnUrl": "https://news.ycombinator.com/user?id=pg",
  "scrapedAt": "2026-10-04T05:06:21.267Z"
}

Story rows from link posts have url; Ask HN and text posts have text instead. Download the dataset as JSON, CSV, Excel or XML.

💰 How much does the Hacker News Scraper cost?

Pricing is pay per event. You pay only for rows that are saved to your dataset.

Event	What you get	Price on the free tier
Story row	One story with points, comments, type and links	$0.012 ($12 per 1,000)
Comment row	One matched comment with author and parent story	$0.01 ($10 per 1,000)
User profile	One user with karma, about and account age	$0.01 ($10 per 1,000)

Paid Apify plans get lower story prices (down to $10 per 1,000 on the Silver tier and above). You are not charged for error rows, duplicates, empty results or a run that fails. A search with no matches finishes with a clear status message and a zero charge. The actor stops cleanly when your max charge limit is reached.

Example: 10 stories cost $0.12 on the free tier. 1,000 stories cost $12.

🚀 How do I scrape Hacker News in 3 steps?

1. Open the actor page on Apify and click Try for free.
2. Type a search query, choose a story type, sort order and optional filters (minimum points, dates, comments, usernames).
3. Click Start, then download the dataset as CSV, Excel, JSON or XML.

⚙️ Hacker News Scraper input

Field	Type	Default	What it does
searchQuery	string	web scraping	Full-text query across titles and story text. Empty returns everything matching the other filters.
storyType	select	story	story, ask_hn, show_hn, job, poll or front_page.
sortBy	select	relevance	relevance or date (newest first).
minPoints	integer	0	Only items with at least this many upvotes.
dateFrom / dateTo	string	empty	Date bounds in YYYY-MM-DD. An invalid date fails the run before anything is charged.
includeStories	boolean	true	Turn off to get only comments or profiles.
includeComments	boolean	false	Also search comments that match the query.
usernames	array	[]	Usernames to fetch profile rows for.
maxItems	integer	10	Maximum rows across all record types. Free Apify users are limited to 10.

Example input:

{
  "searchQuery": "postgres",
  "storyType": "story",
  "sortBy": "date",
  "minPoints": 100,
  "dateFrom": "2024-01-01",
  "maxItems": 100
}

📦 Hacker News Scraper output fields

Field	Appears on	Description
recordType	all	story, comment or user
hnId	story, comment	Hacker News item ID
title	story	Post title
url	story	External link, when the post has one
author	story, comment	Username of the poster
points	story, comment	Upvotes, when available
commentCount	story	Number of comments
storyType	story	story, ask_hn, show_hn, job or poll
text	story, comment	Plain text body, HTML removed
storyTitle, storyId	comment	The story the comment belongs to
username, karma, about	user	Profile data
createdAt	all	ISO 8601 creation time
hnUrl	all	Link to the item or profile on news.ycombinator.com
scrapedAt	all	When the row was collected (UTC)

⚖️ How does this Hacker News scraper compare to alternatives?

Prices below were read from the public Apify Store listings on 2026-10-04.

Actor	Price per 1,000 stories	Comments	User profiles
epctex/hackernews-scraper	about $0.30	not checked	not checked
gentle_cloud/hacker-news-scraper	about $0.20	not checked	not checked
automation-lab/hackernews-scraper	about $1.15	not checked	not checked
ryanclinton/hackernews-search	about $5	not checked	not checked
constructive_calm/hacker-news-scraper	about $0.40 (+ $0.01 per run start)	Yes, about $0.15 per 1,000	Yes, about $0.30 per 1,000
This actor	$12 (free tier), $10 (Silver and above)	Yes, $10 per 1,000	Yes, $10 per 1,000

This actor is more expensive per story, comment and user than every actor in that table. What the extra price buys: six post types including the live front page, points and date filters applied at the source so you do not pay for rows you would discard, and a strict no-charge rule for errors, duplicates and empty results. If all you need is a cheap bulk dump of stories, a cheaper actor will serve you better.

💼 What can I use Hacker News data for?

• Brand and product monitoring: every Hacker News mention of your product or category, scored and dated.
• Market and developer-sentiment research: what developers praised or criticized about a technology, year by year.
• Hiring intelligence: use the job type, or search comments in the monthly "Who is hiring?" threads.
• Content strategy: which headlines about your topic reached the front page and how many comments they drew.
• Academic datasets: tech discourse corpora with engagement signals.

🔌 Can I use the Hacker News Scraper through the API?

Yes. Run it with the Apify API, the Apify clients, or connect it to Zapier, Make, n8n, Airbyte, Slack or Google Sheets.

curl -X POST "https://api.apify.com/v2/acts/recordsdata~hackernews-search-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQuery":"web scraping","maxItems":10}'

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('recordsdata/hackernews-search-scraper').call({
    searchQuery: 'postgres', minPoints: 100, maxItems: 50,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

AI agents can call it through the Apify MCP server (mcp.apify.com), so an assistant can run the query and read the rows directly. Typical automations: hourly mention alerts to Slack, weekly sentiment exports, or archive syncs to a warehouse.

🛡️ Is it legal to scrape Hacker News?

The actor reads the public Hacker News Search API provided by Algolia and the public Hacker News user data. It does not log in, does not bypass any protection, and sends requests at a slow, polite rate. The data is public posts and comments; you remain responsible for using it in line with applicable law and the Hacker News terms, and for handling any personal data (usernames) appropriately.

❓ Frequently asked questions

Why did my Hacker News search return 0 results?

The query and filters matched nothing in the index. Try a shorter keyword, lower minPoints, widen the date range, or switch storyType. A run with no matches succeeds with a status message and costs nothing.

How far back does the Hacker News data go?

To the start of Hacker News in 2007. The item counter on the official Hacker News API read 49,950,816 on 2026-10-04. Use dateFrom and dateTo to slice any window.

What does sorting by relevance mean?

The search index blends text match with popularity. Choose date (newest first) for monitoring workflows.

Do comment rows include the story they belong to?

Yes. Each comment carries its parent storyTitle and storyId for joining.

Can I get the current Hacker News front page?

Yes. Set storyType to front_page.

Can I scrape Hacker News user profiles?

Yes. Add usernames to the usernames input. Unknown usernames return an uncharged error row.

Am I charged for failed or empty rows?

No. Error rows, duplicates, empty results and failed runs are not charged, and the run stops when your max charge limit is reached.

Can I try it for free?

Yes. Free Apify users get a 10-row preview. Paid plans unlock larger runs.

How do I get a CSV or Excel file?

Open the run's dataset and choose the export format, or add ?format=csv to the dataset items API URL.

🔗 Want more data actors?

• arXiv Research Papers Scraper
• Wikipedia Articles Scraper

📬 Support

Found a bug or a missing field? Open the Issues tab on this actor's page, or write to contact.punkrecordsdata@gmail.com. Custom solutions are available on request.

Last updated: 2026-10-03