Hacker News Scraper - Stories, Jobs, Ask HN, Show HN

Get clean, structured Hacker News data on demand: front page stories, newest posts, Ask HN and Show HN threads, and Y Combinator job posts. Every run returns ready-to-use JSON you can drop straight into an AI/RAG pipeline, a trend dashboard, a job feed, or a BI warehouse. Powered by the official Hacker News Firebase API - no HTML scraping, no proxies, no login, no maintenance headaches.

Why not just scrape the website?

	Scraping news.ycombinator.com	This Actor
Data source	HTML pages	Official Hacker News Firebase API
Breaks on redesign	Yes, selectors rot	No, structured API fields
Proxies needed	Usually	None
Rate limiting / blocks	Common	Not an issue
Login or API key	Sometimes required	Never
Output	Messy, needs parsing	Clean JSON, typed fields
Setup	Custom selectors per page	Run with empty input {}
Speed	Slow page-by-page	Fast direct item fetches

What can it scrape?

Story type	storyType	What you get
Top	top	The current Hacker News front page ranking
New	new	The newest submissions, in posting order
Best	best	The highest scoring stories over a recent window
Ask HN	ask	Community questions and discussion threads
Show HN	show	Product, project, and startup launches
Jobs	job	Y Combinator company job posts

Example output

{
  "id": 123,
  "type": "story",
  "title": "Example story",
  "url": "https://example.com",
  "hnUrl": "https://news.ycombinator.com/item?id=123",
  "by": "pg",
  "score": 100,
  "time": 1710000000,
  "descendants": 42,
  "kids": [1, 2, 3]
}

The same records in table form (illustrative placeholder values):

id	type	title	url	by	score	descendants
123	story	Example story	https://example.com	pg	100	42
124	story	Another example story	https://example.org	user2	57	18
125	job	Example company is hiring	https://example.com/jobs	user3	1	0

Output fields

Field	Type	Description
id	integer	Unique Hacker News item ID
type	string	Item type as reported by the API, for example story or job
title	string	Title of the post
url	string	External link submitted with the post, when present
hnUrl	string	Permalink to the item on news.ycombinator.com
by	string	Username of the submitter
score	integer	Current points on the item
time	integer	Submission time as a Unix timestamp in seconds
descendants	integer	Total comment count on the item
kids	array of integers	IDs of the direct child comments

How to use it

1. Click Try for free and open the Actor input.
2. Pick a storyType (or leave everything empty - the Actor runs with empty input {} and returns 50 top stories).
3. Set maxResults (1-500) and turn on includeText if you want the HTML text body of Ask HN and self posts.
4. Run it, then export the dataset to CSV, JSON, Excel, XML, or pull it through the Apify API and webhooks into your own system.

Input

{
  "storyType": "top",
  "maxResults": 50,
  "includeText": false
}

Field	Type	Default	Description
storyType	string	top	One of top, new, best, ask, show, job
maxResults	integer	50	Number of stories to return, 1-500
includeText	boolean	false	Include HTML text body when available

Example 1 - latest Show HN launches

{
  "storyType": "show",
  "maxResults": 100,
  "includeText": true
}

Example 2 - job posts only

{
  "storyType": "job",
  "maxResults": 200,
  "includeText": false
}

Use cases

• AI and RAG datasets - build embeddings and knowledge bases from real technical discussion.
• Startup and tech trend monitoring - track what the developer community is upvoting day to day.
• Job monitoring - watch Y Combinator company hiring posts and feed them into a job board or alert.
• Market and competitor research - see which products launch on Show HN and how they land.
• News alerts - schedule runs and push new front page items into Slack or email.
• Sentiment and discussion analysis - use scores, comment counts, and comment IDs to measure engagement.

Integrations

Connect the Actor to Zapier, Make, n8n, Slack, Google Sheets, Airbyte, and LangChain, or call it directly through the Apify REST API and webhooks. Schedule it hourly or daily for continuous monitoring, and have each finished run push fresh data into your pipeline automatically.

Pricing

Pay Per Event:

• $0.002 per story returned
• $0.00005 per Actor start

You only pay for stories that are actually returned. No subscription, no proxy fees.

FAQ

Do I need a Hacker News account or an API key? No. The Hacker News Firebase API is public and requires no credentials.

Is this legal? It reads public data from the official Hacker News API. No login is bypassed and no private data is touched.

How fresh is the data? Real time. Every run queries the live API, so you get the current state of the story lists.

What is the maximum number of stories per run? 500. Set maxResults anywhere from 1 to 500.

Does it get comments? It returns the kids array of direct comment IDs plus the descendants total comment count, so you can resolve threads yourself.

Will it break when Hacker News changes its design? No. It uses the official structured API instead of HTML scraping, so front end redesigns do not affect it.

Can I run it without configuring anything? Yes. An empty input {} runs successfully and returns 50 top stories.

Troubleshooting

Problem	Fix
Empty dataset	Check that storyType is one of top, new, best, ask, show, job
Fewer results than maxResults	That story list is currently shorter than the number you requested
Need more than 500 stories	Run the Actor multiple times across different story types and merge the datasets
Missing url field on some items	Ask HN and text posts have no external link; use hnUrl instead
No text body in the output	Set includeText to true; text is only present on posts that have one