Hacker News Scraper

Hacker News is one of the best signals in tech, but the site gives you no easy way to pull it into a spreadsheet. You are stuck refreshing the front page, copying links by hand, and losing track of what scored well yesterday. This scraper reads any HN feed for you and hands back clean rows: the headline, the outbound link, the vote score, who submitted it, how many replies it drew, and a direct link to the thread. Point it at Top, New, Best, Ask, Show, or Jobs, or feed it your own list of HN page URLs, and export the lot as JSON, CSV, or Excel.

What you get

One row per post, with the same fields every time, so your columns line up whether you open the results in a sheet or load them into a database. Anything Hacker News does not show (vote scores and authors on job posts, for example) comes back as null rather than dropping out. Each row carries:

• The post — headline, linkUrl, sourceDomain, entryKind, and the numeric postId
• The signal — voteScore, repliesTotal, submitter, and postedAgo
• The links — threadUrl straight to the discussion, plus its feedPosition and a collectedAt timestamp

Quick start

1. Hit Try for free and open the input form.
2. Pick a Feed — Top, New, Best, Ask HN, Show HN, or Jobs.
3. Set a Results limit to control how many rows you bring back (each page holds 30).
4. Press Start, then export the results as JSON, CSV, Excel, or XML once the run finishes.

To crawl specific pages instead of a feed, paste their URLs into Seed URLs — those take over from the Feed setting.

Use cases

• Trend tracking — watch which stories climb the front page and which topics keep resurfacing
• Founder and job research — pull the Jobs and Show HN feeds to see who is hiring and what people are launching
• Newsletter curation — grab the day's top stories with their scores and links in one export
• Content and SEO research — find the domains and headlines that get traction with a technical audience
• Sentiment and discussion analysis — pair each post with its reply count and thread link for deeper reading
• Dataset building — snapshot a feed on a schedule and stack the rows into a historical record over time

Input

Field	Type	Required	Description
feedChoice	string	Yes	Which Hacker News feed to pull: top, new, best, ask, show, or jobs. Default top.
seedUrls	array of strings	No	Specific HN page URLs to crawl instead of a feed. When set, these override feedChoice.
resultsLimit	integer	No	Largest number of rows per run. Each feed page holds 30 entries. Default 25.
timeoutSeconds	integer	No	Seconds to wait on each request before giving up. Default 240; raise it if you hit timeouts.

Example input

{
    "feedChoice": "best",
    "seedUrls": [],
    "resultsLimit": 25,
    "timeoutSeconds": 240
}

Output

Every post on the feed becomes one row, and each field is always present — values Hacker News does not expose come back as null so the dataset stays rectangular.

Example output

{
    "postId": 48031684,
    "feedPosition": 1,
    "headline": "Agents can now create Cloudflare accounts, buy domains, and deploy products",
    "linkUrl": "https://blog.cloudflare.com/agents-stripe-projects/",
    "sourceDomain": "cloudflare.com",
    "voteScore": 200,
    "submitter": "rolph",
    "repliesTotal": 108,
    "threadUrl": "https://news.ycombinator.com/item?id=48031684",
    "postedAgo": "3 hours ago",
    "entryKind": "story",
    "collectedAt": "2026-06-29T12:00:00.000000+00:00"
}

Output fields

Field	Type	Description
postId	integer	Numeric Hacker News identifier for the entry
feedPosition	integer	Where the entry sits in the listing order
headline	string	Title text of the post or story
linkUrl	string	Outbound link the post points to; the HN thread link for Ask and Show entries
sourceDomain	string	Host portion of the linked URL
voteScore	integer	Upvotes on the entry; null for job posts
submitter	string	Account name of whoever posted it; null for job posts
repliesTotal	integer	Comment count on the thread; null for job posts
threadUrl	string	Link straight to the HN comment thread
postedAgo	string	Plain-language age of the entry
entryKind	string	Category: story, job, ask, show, or launch
collectedAt	string	ISO 8601 UTC timestamp of when the row was captured

Tips for best results

• Match the feed to the question. Use New for the freshest posts, Best for all-time winners, and Jobs or Show HN when you want a single post type.
• Keep test runs small. Set resultsLimit to 20–30 while you confirm the output fits your pipeline, then raise it for the full batch.
• Mind the 30-per-page rule. Each feed page holds 30 entries, so the scraper pages forward automatically until it reaches your limit.
• Expect null on job posts. Jobs carry no vote score, author, or reply count, so those fields stay empty by design — that is not an error.
• Raise timeoutSeconds if a run reports timeouts; slower network conditions sometimes need a longer window.

How can I use Hacker News data?

How can I use the Hacker News Scraper to track trending tech stories? Run the Top or Best feed on a schedule and each post comes back with its vote score, reply count, and outbound link. Sort by voteScore or repliesTotal to see what is climbing, and compare snapshots over time to spot which topics keep returning to the front page.

How can I scrape Hacker News jobs and Show HN posts? Set feedChoice to jobs or show, or drop the matching page URLs into seedUrls. The scraper returns each entry's headline, link, submitter, and thread URL, so you can build a feed of who is hiring or a list of fresh launches without scrolling the site by hand.

How can I export Hacker News data to a spreadsheet for research? Pick a feed, set a resultsLimit, and run it. Every post becomes one row with consistent columns, ready to download as JSON, CSV, Excel, or XML. From there you can pivot on domains, authors, or post types to study what resonates with a technical audience.

Is it legal to scrape data?

Our actors are ethical and do not extract any private user data, such as email addresses or private contact information. They only extract what the user has chosen to share publicly. We therefore believe that our actors, when used for ethical purposes by Apify users, are safe.

However, you should be aware that your results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers.

You can also read Apify's blog post on the legality of web scraping.

Support

Questions, feature requests, or a field you'd like added? Reach out at data.apify@proton.me and we'll get back to you.