Hacker News Scraper – Stories, Jobs & Comments API

Search Hacker News by keyword or export fresh Hacker News feeds into a clean Dataset. This Actor collects public stories, comments, Ask HN, Show HN, jobs, polls, and poll options from two public HN data surfaces:

• keyword search through the public Algolia-powered Hacker News Search API
• live feed and item collection through the official Firebase API

Use searchQuery when you want results for a phrase such as AI agents, vector database, or who is hiring. Leave searchQuery empty when you want the current top, new, best, ask, show, or job feeds.

What you can do with this Hacker News Scraper

1. Search Hacker News by keyword

Enter a keyword or phrase in searchQuery and the Actor returns matching Hacker News items as structured rows. This is useful for research, monitoring, trend discovery, competitive intelligence, and agent workflows that need HN search results without scraping HTML.

What you enter	Example
searchQuery	AI agents
recordTypes	story, comment, job
maxItems	25

Search mode returns Algolia HN Search hits directly. Rows include sourceSearchQuery and searchRank so you can trace which keyword produced each result.

2. Export the current Hacker News front page

Leave searchQuery empty and choose one or more official Firebase feeds such as top, new, or best. Each matching item becomes a normalized Dataset row with title, author, score, timestamp, URL, and direct HN link.

What you enter	Example
feeds	top, new, or best
recordTypes	story
maxItems	100

3. Collect Ask HN and Show HN discussions

Use the ask and show feeds to export questions, launches, demos, and the public replies beneath them. Turn on descendant collection when comments matter as much as the root post.

What you enter	Example
feeds	ask, show
recordTypes	story, comment
includeDescendants	true
maxDepth	1 or 2

The feed collector schedules roots fairly and walks descendants breadth-first, so a single large thread does not silently monopolize a small run.

4. Build a Hacker News jobs feed

Choose the official job feed to capture current HN job posts, or search for hiring-related phrases with searchQuery. Keep recordTypes set to job for a focused export.

5. Retrieve known Hacker News items by ID

If another workflow already has Hacker News item IDs, leave searchQuery empty and add them to itemIds. The Actor collects those explicit IDs before feed roots and deduplicates all items within the run.

6. Feed a dashboard, agent, or recurring workflow

Every row has stable itemId, hnUrl, source provenance, and a run timestamp. Export the Dataset to your data warehouse, spreadsheet, CRM enrichment step, or agent workflow without parsing Hacker News HTML.

How to use it — no code required

1. Click Try for free on this Actor page.
2. To search, enter a keyword in searchQuery.
3. To collect live feeds instead, leave searchQuery empty and select the Hacker News feeds you want.
4. Choose the item types you want in the Dataset.
5. Set maxItems and maxFetches, then click Start.

Download your results as Excel, CSV, JSON, or JSONL from the Dataset tab, or consume them through the Apify API.

What you get back

Each output row is a normalized public Hacker News item. The fields stay consistent across stories, comments, jobs, polls, and poll options while preserving what the source supplied.

Field	What it means
itemId	Stable Hacker News item ID.
type	story, comment, job, poll, or pollopt.
title, text, url	Public item content when present.
by, time, createdAt	Author and source timestamp.
score, descendants	Public engagement metadata when supplied by Hacker News or HN Search.
parentId, kids, parts	Thread and poll relationships when available.
hnUrl	Direct link to the item on Hacker News.
sourceFeeds, rootItemId, depth	Feed/item-ID provenance. Search rows use sourceFeeds: ["algolia-search"].
sourceSearchQuery, searchRank	Keyword and rank for search-mode rows.
deleted, dead	Source state flags when supplied.
source	hacker-news-algolia-search or hacker-news-firebase.
scrapedAt	When this Actor collected the row.

Malformed upstream records fail clearly before Dataset output instead of producing silently invalid rows.

Input reference

The Apify input form exposes the same controls used at runtime:

Parameter	Default	Description
searchQuery	empty	Optional keyword or phrase. When set, the Actor runs HN keyword search.
feeds	top, new, best, ask, show, job	Official Firebase feed roots to collect when searchQuery is empty.
recordTypes	all five supported types	Item types retained in the Dataset. Applies to both search and feed modes.
itemIds	[]	Optional explicit positive Hacker News item IDs for feed/item-ID mode.
includeDescendants	true	Include public comment children and poll parts in feed/item-ID mode.
maxDepth	2	Descendant levels below each root; root depth is 0. Not used in search mode.
maxItems	100	Maximum deduplicated Dataset records.
maxFetches	500	Maximum Firebase item detail requests in feed/item-ID mode; must be at least maxItems.

Keyword search behavior

When searchQuery is set, the Actor uses the public Algolia-powered HN Search API and writes matching search hits to the Dataset. Feed traversal, descendant traversal, and itemIds are not used in this mode. Unsupported keyword aliases such as query and keywords are rejected so runs do not silently change scope.

Data source and run boundaries

Keyword search uses the public Algolia Hacker News Search API. Feed and item-ID collection uses the public, unauthenticated official Hacker News Firebase API. Live feeds and search rankings change continuously, so each run is a timestamped snapshot—not a historical archive or a guaranteed complete feed mirror.

Network requests have bounded retries. The Actor also uses an overall collection deadline, positive integer limits, per-run deduplication, source-ID verification for Firebase items, and breadth-first descendant traversal for feed mode. If the source cannot be retrieved or its response does not match the expected public contract, the run writes an ERROR_SUMMARY and fails instead of reporting a misleading empty success.

Run artifacts

Besides the Dataset, the default key-value store contains useful, non-secret run artifacts:

Artifact	When it is written	What it contains
INPUT_ECHO	After valid input parsing	Normalized scope, keyword, and limits.
RUN_SUMMARY	After a successful run	Source, keyword, record counts, requested scope, and timestamps.
ERROR_SUMMARY	When a run fails	Error type, safe error message, and timestamps.

These artifacts make it straightforward to debug an automation without exposing credentials—this Actor does not require any.

Pricing

Pay per event. No subscription. Apify displays the effective event price before each run.

Event	Price	When it is charged
apify-actor-start	$0.005	Automatically once when a normal Actor run starts.
apify-default-dataset-item	$2.00 / 1,000	Automatically for each item written to the default Dataset.

For example, a run returning 100 Dataset rows costs $0.205 in Actor events: $0.005 to start plus $0.200 for results. Platform usage costs are charged to the user in addition to these Actor events, as shown by Apify before a run.

Both listed events are Apify synthetic events: the platform accounts for them automatically, so the Actor does not manually issue duplicate charges in code.

Live View and Standby endpoints

This Actor has an honest lightweight Live View service. Data collection stays in normal Actor runs; the standby service does not pretend to offer a separate scrape API.

Endpoint	Purpose
GET /	Readiness and service information.
GET /health	Fast health check.
GET /input-example	Low-cost input compatible with the Actor schema.
GET /openapi.json	The deployed OpenAPI 3.x contract.

The interactive Standby tab on Apify renders these endpoints from the same OpenAPI document served by the Actor.

Connect from code or an AI workflow

Use this Actor in Apify, Make, Zapier, n8n, or an agent workflow. The Actor ID is lentic_clockss/hacker-news-scraper.

FAQ

Does it search Hacker News by a phrase or brand name?

Yes. Set searchQuery to the phrase you want. The Actor returns public HN Search hits and records the keyword in sourceSearchQuery.

Does it use the official Hacker News API?

Feed and item-ID mode uses the official Hacker News Firebase API. Keyword search uses the public Algolia-powered HN Search API because Firebase does not provide keyword search.

Can I collect comments?

Yes. In search mode, include comment in recordTypes. In feed mode, keep recordTypes containing comment, set includeDescendants to true, and choose an appropriate maxDepth, maxItems, and maxFetches.

Why can a selected type have zero results?

Hacker News live feeds and keyword searches are not guaranteed to contain every item type at every moment. For example, polls can be uncommon in a particular feed snapshot or search result set.

What happens when a source returns malformed data or is unavailable?

The Actor retries bounded transient failures. Invalid source records are rejected before Dataset output, and a failed run writes ERROR_SUMMARY rather than claiming success with an untrustworthy result.

Can I use the Live View endpoints to start a collection?

No. Live View provides readiness, health, input guidance, and its OpenAPI document. Start data collection through a normal Actor run so output stays in the run Dataset and key-value store.

---

Explore more tools from lentic_clockss on Apify.