Pricing

from $1.10 / 1,000 results

Try for free

Go to Apify Store

Hacker News Scraper - Stories, Jobs, Ask HN & Show HN

Try for free

Export fresh Hacker News stories, comments, jobs, Ask HN, Show HN, and polls into structured datasets for research, hiring signals, and monitoring.

Pricing

from $1.10 / 1,000 results

Rating

0.0

(0)

Developer

kane liu

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

Hacker News Scraper – Stories, Jobs & Comments API

Search Hacker News by keyword or export fresh Hacker News feeds into a clean Dataset. This Actor collects public stories, comments, Ask HN, Show HN, jobs, polls, and poll options from two public HN data surfaces:

keyword search through the public Algolia-powered Hacker News Search API
live feed and item collection through the official Firebase API

Use searchQuery when you want results for a phrase such as AI agents, vector database, or who is hiring. Leave searchQuery empty when you want the current top, new, best, ask, show, or job feeds.

Architecture: this Apify Actor is a thin client. Collection runs on the hackernews-com Cloud Run worker over HTTPS + API key.

Worker URL is config, not code: the Actor reads WORKER_BASE_URL from Actor environment variables (plus optional per-run workerBaseUrl). Migrating the worker host means updating WORKER_BASE_URL (and WORKER_AUTH if the API key changes) in Apify Console — no Actor code change required.

What you can do with this Hacker News Scraper

1. Search Hacker News by keyword

Enter a keyword or phrase in searchQuery and the Actor returns matching Hacker News items as structured rows. This is useful for research, monitoring, trend discovery, competitive intelligence, and agent workflows that need HN search results without scraping HTML.

What you enter	Example
`searchQuery`	`AI agents`
`recordTypes`	`story`, `comment`, `job`
`maxItems`	`25`

Search mode returns Algolia HN Search hits directly. Rows include sourceSearchQuery and searchRank so you can trace which keyword produced each result.

2. Export the current Hacker News front page

Leave searchQuery empty and choose one or more official Firebase feeds such as top, new, or best. Each matching item becomes a normalized Dataset row with title, author, score, timestamp, URL, and direct HN link.

What you enter	Example
`feeds`	`top`, `new`, or `best`
`recordTypes`	`story`
`maxItems`	`100`

3. Collect Ask HN and Show HN discussions

Use the ask and show feeds to export questions, launches, demos, and the public replies beneath them. Turn on descendant collection when comments matter as much as the root post.

What you enter	Example
`feeds`	`ask`, `show`
`recordTypes`	`story`, `comment`
`includeDescendants`	`true`
`maxDepth`	`1` or `2`

The feed collector schedules roots fairly and walks descendants breadth-first, so a single large thread does not silently monopolize a small run.

4. Build a Hacker News jobs feed

Choose the official job feed to capture current HN job posts, or search for hiring-related phrases with searchQuery. Keep recordTypes set to job for a focused export.

5. Retrieve known Hacker News items by ID

If another workflow already has Hacker News item IDs, leave searchQuery empty and add them to itemIds. The Actor collects those explicit IDs before feed roots and deduplicates all items within the run.

6. Feed a dashboard, agent, or recurring workflow

Every row has stable itemId, hnUrl, source provenance, and a run timestamp. Export the Dataset to your data warehouse, spreadsheet, CRM enrichment step, or agent workflow without parsing Hacker News HTML.

More Actors like this

Need another social/content source or a different vertical? Use a dedicated Actor below for structured fields. Use this Actor for Hacker News stories, comments, jobs, Ask HN, Show HN, and polls.

Prefer another specialized scraper?

If your target has a specialized Actor below, use that first — usually cheaper and more structured than a generic scrape.

Jobs & Freelance

E-commerce

Real Estate

Travel & Stays

Ads Intelligence

Local & Maps

Google Maps Scraper

General Tools

→ See the full catalog in Related Actors below, or browse apify.com/lentic_clockss.

How to use it — no code required

Click Try for free on this Actor page.
To search, enter a keyword in searchQuery.
To collect live feeds instead, leave searchQuery empty and select the Hacker News feeds you want.
Choose the item types you want in the Dataset.
Set maxItems and maxFetches, then click Start.

Download your results as Excel, CSV, JSON, or JSONL from the Dataset tab, or consume them through the Apify API.

What you get back

Each output row is a normalized public Hacker News item. The fields stay consistent across stories, comments, jobs, polls, and poll options while preserving what the source supplied.

Field	What it means
`itemId`	Stable Hacker News item ID.
`type`	`story`, `comment`, `job`, `poll`, or `pollopt`.
`title`, `text`, `url`	Public item content when present.
`by`, `time`, `createdAt`	Author and source timestamp.
`score`, `descendants`	Public engagement metadata when supplied by Hacker News or HN Search.
`parentId`, `kids`, `parts`	Thread and poll relationships when available.
`hnUrl`	Direct link to the item on Hacker News.
`sourceFeeds`, `rootItemId`, `depth`	Feed/item-ID provenance. Search rows use `sourceFeeds: ["algolia-search"]`.
`sourceSearchQuery`, `searchRank`	Keyword and rank for search-mode rows.
`deleted`, `dead`	Source state flags when supplied.
`source`	`hacker-news-algolia-search` or `hacker-news-firebase`.
`scrapedAt`	When this Actor collected the row.

Malformed upstream records fail clearly before Dataset output instead of producing silently invalid rows.

Lead and contact fields

Every Dataset row includes the same lead shape, even when a particular Hacker News item has no outreach detail. Empty values remain explicit and leadAbsences explains why they were not populated.

Field	Type	Hacker News meaning and source
`phones`	`[{value, type, e164?}]`	Public phone numbers attributable to the item author or hiring organization. Phone extraction is currently unverified, so numeric item IDs, dates, versions, and UTC offsets are rejected rather than guessed as phone numbers.
`emails`	`[{value, type}]`	Public email addresses in lead-bearing comment text. The platform footer address is excluded because it belongs to Hacker News, not to the comment author or job entity.
`address`	`{street, unit, locality, region, postalCode, country, full}`	Public `Location:` text from comments. `unit` means the street number or door number. Locality, region, country, and full location may be present; `street`, `unit`, and postal code are currently unverified.
`geo`	`{lat, lng} \| null`	Structured coordinates in addition to an address. Hacker News does not provide coordinates on the measured public surfaces, so the Actor does not infer them from free text.
`person`	`{name, role?} \| null`	The attributable Hacker News username and an explicit public role or “seeking” statement when present.
`org`	`{name, id?} \| null`	A hiring organization parsed from an HN job title. HN job items do not currently provide a verified structured organization ID.
`web`	`{website, socials[]}`	Attributable websites and social profiles from comment text, a public user profile `about` field, or a job careers URL. A story/article URL is content provenance and is not automatically treated as the author’s website.
`leadSource`	`search \| detail \| entity \| contact \| reveal-xhr`	Surface that produced the lead. Current HN records primarily use `detail` for comments and `entity` for jobs; Algolia result leads use `search`.
`leadReveal`	`visible \| click \| xhr \| gated \| image \| absent`	How the lead became available. Current HN lead values are already public, so measured rows use `visible`; rows without a value use `absent`.
`leadAbsences`	object/map	Per-field reasons such as `未验证` when a value was not present or was not validated. This keeps missing phones, street numbers, roles, organizations, and profile links honest at row level.

The accepted detail sample contained public emails, comment locations, usernames, websites, and social links. HN job items can additionally provide an organization name and careers URL.

Reveal and enrichment cost

Hacker News has no measured “show phone”, “show email”, or “show address” control on these public surfaces. The Actor does not click a reveal control and does not issue a reveal XHR.

With enrichDetails=true (the default), the worker may make one additional public Firebase user-profile request per distinct candidate author before skipNoLeads filtering and final maxItems truncation. Profile responses are cached within the run, capped at 25 distinct candidate authors and bounded by a 20-second enrichment budget. A filtered run can therefore return fewer rows than the number of profiles attempted, especially when skipNoLeads=true; the RUN_SUMMARY profile counters show the actual attempts. This can add run time and public API requests, but it does not submit forms, follow submitted[], or turn article links into attributed contact data. If the profile budget or request fails after primary records were collected, the rows remain usable and the profile gap is recorded in leadAbsences.

Honest lead limitations

Public HN profiles are often sparse. Contact details in an attributable comment are the primary lead source; profile about enrichment is supplementary.
phones is unverified. The Actor deliberately avoids treating numeric HN IDs, dates, versions, model numbers, or time-zone offsets as phone numbers.
address.locality, address.region, address.country, and address.full can come from explicit public Location: text. Street, door/unit number, postal code, and structured coordinates were not verified during the measured research.
A username is a person identity, but person.role is populated only from explicit public wording. It is not inferred from unrelated story content.
Organization names are supported for organization-bearing job titles. A normal comment or story is not assigned an organization without explicit attributable evidence.
Story/article URLs remain content provenance. Only attributable comment/profile links and job careers URLs populate web.
The Hacker News platform footer email is excluded from entity rows.
Missing values vary by item. leadAbsences describes the row-level gap as unverified rather than claiming the field never exists site-wide.

Input reference

The Apify input form exposes the same controls used at runtime:

Parameter	Default	Description
`searchQuery`	empty	Optional keyword or phrase. When set, the Actor runs HN keyword search.
`feeds`	`top`, `new`, `best`, `ask`, `show`, `job`	Official Firebase feed roots to collect when `searchQuery` is empty.
`recordTypes`	all five supported types	Item types retained in the Dataset. Applies to both search and feed modes.
`itemIds`	`[]`	Optional explicit positive Hacker News item IDs for feed/item-ID mode.
`includeDescendants`	`true`	Include public comment children and poll parts in feed/item-ID mode.
`enrichDetails`	`true`	Fetch each distinct candidate author's public Firebase profile at most once per run before lead filtering/final truncation, then merge public website/social clues (maximum 25 candidate authors or 20 seconds).
`skipNoLeads`	`false`	When enabled, keep only records with an email, phone, location, organization, website, or social profile.
`maxDepth`	`2`	Descendant levels below each root; root depth is `0`. Not used in search mode.
`maxItems`	`100`	Maximum deduplicated Dataset records (1–2,000). Free Apify plan: also capped at 1,000/run and 10 total runs of this Actor.
`maxFetches`	`500`	Maximum Firebase item detail requests in feed/item-ID mode; must be at least `maxItems`.
`workerBaseUrl`	(env)	Optional per-run override of the Cloud Run worker origin. Default: Actor env `WORKER_BASE_URL`.

Keyword search behavior

When searchQuery is set, the Actor uses the public Algolia-powered HN Search API and writes matching search hits to the Dataset. Feed traversal, descendant traversal, and itemIds are not used in this mode. Unsupported keyword aliases such as query and keywords are rejected so runs do not silently change scope.

Data source and run boundaries

Keyword search uses the public Algolia Hacker News Search API. Feed and item-ID collection uses the public, unauthenticated official Hacker News Firebase API. Live feeds and search rankings change continuously, so each run is a timestamped snapshot—not a historical archive or a guaranteed complete feed mirror.

Network requests have bounded retries. The Actor also uses an overall collection deadline, positive integer limits, per-run deduplication, source-ID verification for Firebase items, and breadth-first descendant traversal for feed mode. If the source cannot be retrieved or its response does not match the expected public contract, the run writes an ERROR_SUMMARY and fails instead of reporting a misleading empty success.

Run artifacts

Besides the Dataset, the default key-value store contains useful, non-secret run artifacts:

Artifact	When it is written	What it contains
`INPUT_ECHO`	After valid input parsing	Normalized scope, keyword, and limits.
`RUN_SUMMARY`	After a successful run	Source, keyword, record counts, requested scope, and timestamps.
`ERROR_SUMMARY`	When a run fails	Error type, safe error message, and timestamps.

These artifacts make it straightforward to debug an automation without exposing credentials—this Actor does not require any.

Free Apify plan limits (set by this Actor's developer)

Limit	Free Apify plan	Paid Apify plan
Runs of this Actor	10 runs total	Unlimited
Results per run (`maxItems`)	Up to 1,000	Up to 2,000

When the run cap is hit, the run finishes with a clear status message — not an Apify platform error.

Pricing

Pay per event. No subscription. Apify displays the effective event price before each run.

Free Apify plan: up to 10 runs of this Actor and 1,000 results per run. Upgrade to a paid Apify plan for unlimited runs. (Limits set by the Actor developer.)

Event	Price	When it is charged
`apify-actor-start`	$0.005	Automatically once when a normal Actor run starts.
`apify-default-dataset-item`	$2.00 / 1,000	Automatically for each item written to the default Dataset.

For example, a run returning 100 Dataset rows costs $0.205 in Actor events: $0.005 to start plus $0.200 for results. Platform usage costs are charged to the user in addition to these Actor events, as shown by Apify before a run.

Both listed events are Apify synthetic events: the platform accounts for them automatically, so the Actor does not manually issue duplicate charges in code.

Live View and Standby endpoints

This Actor has an honest lightweight Live View service. Data collection stays in normal Actor runs; the standby service does not pretend to offer a separate scrape API.

Endpoint	Purpose
`GET /`	Readiness and service information.
`GET /health`	Fast health check.
`GET /input-example`	Low-cost input compatible with the Actor schema.
`GET /openapi.json`	The deployed OpenAPI 3.x contract.

The interactive Standby tab on Apify renders these endpoints from the same OpenAPI document served by the Actor.

Connect from code or an AI workflow

Use this Actor in Apify, Make, Zapier, n8n, or an agent workflow. The Actor ID is lentic_clockss/hacker-news-scraper.

FAQ

Does it search Hacker News by a phrase or brand name?

Yes. Set searchQuery to the phrase you want. The Actor returns public HN Search hits and records the keyword in sourceSearchQuery.

Does it use the official Hacker News API?

Feed and item-ID mode uses the official Hacker News Firebase API. Keyword search uses the public Algolia-powered HN Search API because Firebase does not provide keyword search.

Can I collect comments?

Yes. In search mode, include comment in recordTypes. In feed mode, keep recordTypes containing comment, set includeDescendants to true, and choose an appropriate maxDepth, maxItems, and maxFetches.

Why can a selected type have zero results?

Hacker News live feeds and keyword searches are not guaranteed to contain every item type at every moment. For example, polls can be uncommon in a particular feed snapshot or search result set.

What happens when a source returns malformed data or is unavailable?

The Actor retries bounded transient failures. Invalid source records are rejected before Dataset output, and a failed run writes ERROR_SUMMARY rather than claiming success with an untrustworthy result.

Can I use the Live View endpoints to start a collection?

No. Live View provides readiness, health, input guidance, and its OpenAPI document. Start data collection through a normal Actor run so output stays in the run Dataset and key-value store.

I'm on the free Apify plan — are there limits?

Yes. Free Apify plan users get 10 runs of this Actor and up to 1,000 results per run. These limits are set by the Actor developer (not Apify). When you hit the run cap, the run finishes with a clear status message — it is not a platform bug. Upgrade to a paid Apify plan for unlimited runs.

Explore more tools from lentic_clockss on Apify.

All 77 public Actors from [lentic_clockss]. Click a name to open the Store detail page.

Jobs & Freelance

Real Estate

E-commerce

Travel & Stays

Ads Intelligence

Local & Maps

Google Maps Scraper

General Tools

→ Browse the full profile: apify.com/lentic_clockss

Hacker News Search Scraper Stories, Comments, Show HN, Ask HN

seemuapps/hn-search-scraper

Search Hacker News stories, comments, Show HN, Ask HN, polls, and jobs by keyword, author, date range, points, and comment count. Full text and engagement metrics. No login.

Andrew

Hacker News Search Scraper

fetch_cat/hacker-news-search-scraper

Search Hacker News stories, comments, Ask HN, Show HN, polls, and author activity using public HN search data.

Hanna Nosova

Hacker News Scraper Pro — Stories, Jobs, Show HN, Ask HN

diverse_venture/hackernews-scraper

Comprehensive Hacker News scraper. Get top/new/best stories, Ask HN, Show HN, Who's Hiring jobs, comments, and search results. Uses the official HN Firebase API + Algolia search — no auth required. Export JSON, CSV, or Excel.

Chak Man Fung

Hacker News (HN) Stories, Show HN & Jobs Scraper

kravennagen/hacker-news-scraper

Fast, structured extraction of Hacker News stories, Show HN launches, Ask HN discussions, scores, and comment counts via official Firebase API.

Morgane Flamant

Hacker News Scraper - Stories, Points, Comments & Ask HN

feedforge/hacker-news-scraper

Scrape Hacker News stories, Show HN, Ask HN and jobs. Titles, points, authors, comment counts and URLs as JSON or CSV.

Dominik

Hacker News Scraper — Stories, Comments, Ask/Show HN

hipersoft/hacker-news-scraper

Scrape Hacker News stories, comments, Ask HN, Show HN, jobs and polls by keyword, points, comments and date range. Sort by relevance or newest. Title, URL, author, points, comment count, text and HN link. Fast clean HTTP via the HN Algolia API, no setup.

hiper soft

Show HN Scraper - Hacker News Stories, Ask & Show HN

flash_scraper/hn-scraper

Show HN scraper for Hacker News - Show HN launches, Ask HN threads, front-page stories and comments with points, authors and timestamps. Filter by minimum points or comments. No API key, no login. Export CSV, JSON or Excel.

Flash Scrape

Hacker News Scraper — Stories, Comments & Ask HN by Keyword

convertfleetdotonline/hacker-news-scraper

Search and export Hacker News stories, comments, Ask HN, Show HN and polls by keyword. Points, author, comment counts, URLs & timestamps. Official public API — no login, no key, no proxy. JSON/CSV export.

Hasnain Nisar