Hacker News Stories & Comments Scraper avatar

Hacker News Stories & Comments Scraper

Pricing

from $1.19 / 1,000 results

Go to Apify Store
Hacker News Stories & Comments Scraper

Hacker News Stories & Comments Scraper

Hacker News Scraper SD - Hacker News Scraper is a news data tool that extracts Hacker News stories and comments from the front page, new, best, Ask HN, Show HN, jobs and Who is hiring threads, plus full-text search and comment trees via the official APIs.

Pricing

from $1.19 / 1,000 results

Rating

0.0

(0)

Developer

Neuro Scraper

Neuro Scraper

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

5 days ago

Last modified

Features

Limited permissions

Share

Hacker News Stories & Comments Scraper

The Hacker News Stories & Comments Scraper collects public Hacker news stories & comments and returns it as a clean dataset you can download as JSON, CSV or Excel, or read through the Apify API. No login, no cookies and no code: give the Hacker News Stories & Comments Scraper your input, start it and get structured Hacker data for research, monitoring, analytics or lead generation. You pay only for the results you receive.

⭐ Featured by Neuro Scraper: Facebook Reels Scraper - scrape Facebook Reels from any page, profile or reel link with exact views and plays, every reaction type (Like, Love, Care, Haha, Wow, Sad, Angry), comments, shares, captions, music and video links. No login and no cookies. Try the Facebook Reels Scraper →

Hacker News Scraper — stories, comments, front page and job posts

Hacker News Scraper collects Hacker News stories and comments through the two official public APIs: the HN Firebase API run by Y Combinator, and the Algolia HN Search API. It needs no account and no key, and it never loads the news.ycombinator.com pages.

It does four things, in any mix:

  • Lists → the live front page (top), new, best, Ask HN, Show HN and YC job posts, in Hacker News' own order (rank).
  • Who is hiring? → the top-level job posts of the latest monthly "Ask HN: Who is hiring?" thread.
  • Search → full-text search of stories, comments or both, by relevance or newest first, with date and points filters.
  • Story links → one story (or comment) with its whole comment tree, each comment with its depth.

Input

FieldTypeDefaultMeaning
listsmulti-selecttop (prefill)top, new, best, ask, show, job, whoishiring
searchTermslist–Search terms, one per line
searchTypeselectstoryStories, comments, or stories + comments + polls
searchSortselectrelevanceRelevance (max 1,000 results) or newest first (no cap)
fuzzyMatchingbooleanfalseAlgolia typo tolerance; off gives complete, stable results
dateFrom, dateToYYYY-MM-DD–Optional date window for searches
minPointsinteger–Optional minimum points (stories only)
storyUrlslist–https://news.ycombinator.com/item?id=... links or ids → item + comment tree
maxResultsinteger30Rows per list, search term or story link
maxItemsinteger–Optional cap on the whole run
proxyConfigurationproxyoffNot needed for the official APIs

Output (one row per story or comment)

FieldMeaning
typestory, comment, job or poll
id, hnUrlHacker News item id and link
title, url, domainStory title, linked URL and its site
textComment text, Ask HN / job text, as plain text (links kept)
author, points, commentCount, createdAtAuthor, score, number of comments, post time (UTC)
storyId, storyTitle, parentIdFor comments: the story and the parent item
depthStory links: 0 = the item, 1 = top-level comment, ...; Who is hiring: 1
rankLists: position in the live list (1 = top of the front page)
source, sourceInput, scrapedAtlist / whoishiring / search / item, your input, UTC time

Measured limits (2026-10-03, local runs)

  • Lists hold what Hacker News publishes: top 500, new 500, best 200, Ask HN 17, Show HN 148, jobs 31 ids at the time of testing.
  • List items are read 50 at a time from Algolia (points and comment counts matched Firebase exactly in our check); items Algolia does not have yet, and all job posts, come from Firebase one by one.
  • Relevance search stops at 1,000 results (Algolia's limit). Newest-first search walks back in time and has no cap: 1,500 results in 32 seconds in our test.
  • With fuzzy matching on, Algolia answers filtered searches non-exhaustively: the same query returned between 52 and 175 stories on repeated calls. Exact-word matching (the default) returned the same 50 every time.
  • Deleted and dead comments are not in the comment tree.
  • Speed: 781 rows (three lists + a 250-result search) in 51 seconds, about one request per second.

Example

{"lists": ["top", "whoishiring"], "searchTerms": ["web scraping"], "searchSort": "date", "maxResults": 100}

Hacker News content is written by its users: keep the author and hnUrl for attribution.

Why use the Hacker News Stories & Comments Scraper

The Hacker News Stories & Comments Scraper is a Hacker scraper and Hacker data extractor for anyone who needs public Hacker news stories & comments as clean data. The Hacker News Stories & Comments Scraper handles web scraping, proxy rotation and retries for you, so you get a ready dataset instead of raw pages. Use the Hacker News Stories & Comments Scraper for market research, competitor analysis, lead generation, analytics or monitoring, and export the results to JSON, CSV, Excel or Google Sheets.

  • No login and no code: the Hacker News Stories & Comments Scraper runs on Apify with one click or one API call.
  • Pay per result: the Hacker News Stories & Comments Scraper charges only for rows delivered - no monthly Hacker API fee.
  • Complete fields: the Hacker News Stories & Comments Scraper returns every public field, with a stable schema for automation.
  • Integrations: connect the Hacker News Stories & Comments Scraper to Zapier, Make, n8n, Google Sheets, a webhook or your CRM.

Hacker News Stories & Comments Scraper use cases

  • Market research: the Hacker News Stories & Comments Scraper collects Hacker news stories & comments at scale, so you can compare trends across weeks and niches.
  • Competitor analysis: run the Hacker News Stories & Comments Scraper on a schedule and export the dataset to Google Sheets or Excel to watch what competitors publish.
  • Lead generation and B2B prospecting: the Hacker News Stories & Comments Scraper gives a sales or agency team a clean list to load into a CRM.
  • Content and social media monitoring: the Hacker News Stories & Comments Scraper keeps a fresh copy of Hacker data through an Apify schedule and a webhook.
  • Analytics and dashboards: send the Hacker News Stories & Comments Scraper output as JSON through the Apify API into Python, Node.js or a BI tool.
  • Academic and data science projects: the Hacker News Stories & Comments Scraper returns a consistent schema, ready for pandas, R or a notebook.
  • Price, product and brand tracking: the Hacker News Stories & Comments Scraper turns public Hacker data into a dataset you can diff run by run.
  • Automation workflows: connect the Hacker News Stories & Comments Scraper to Zapier, Make or n8n so every new row starts your next step.

How to run it with the API, Python and Node.js

Start the Hacker News Stories & Comments Scraper from the Apify Console, or call it from code. The Hacker News Stories & Comments Scraper returns its dataset as JSON, CSV or Excel. With the Apify API you can run the Hacker News Stories & Comments Scraper from Python or Node.js, wait for it to finish and read the rows, which makes the Hacker News Stories & Comments Scraper easy to plug into an automation, a data pipeline or a scheduled export.

  1. Open the Hacker News Stories & Comments Scraper on Apify and fill in the input (or send the same input as JSON to the API).
  2. Start the Hacker News Stories & Comments Scraper. Follow the run log while the Hacker News Stories & Comments Scraper collects public Hacker data.
  3. Download the dataset as CSV, JSON, Excel or XLSX, or let a webhook send it on.
  4. Schedule the Hacker News Stories & Comments Scraper to keep your Hacker data fresh for monitoring and analytics.

Tips for better results

  • Start small: run the Hacker News Stories & Comments Scraper with a low maximum first, check the fields, then scale up the same input.
  • Split big jobs: several smaller Hacker News Stories & Comments Scraper runs (by keyword, URL list or date) finish faster and are easier to resume.
  • Keep a history: name a dataset or use a schedule so each Hacker News Stories & Comments Scraper run adds to your Hacker data over time.
  • Filter early: use the input filters of the Hacker News Stories & Comments Scraper so you only pay for the Hacker news stories & comments you really need.
  • Automate the hand-off: a webhook or the Zapier and Make integrations can move every Hacker News Stories & Comments Scraper result straight to Google Sheets, a CRM or a database.
  • Watch the run log: the Hacker News Stories & Comments Scraper writes what it is doing and why an item was skipped, so a surprising result is easy to explain.

Hacker News Stories & Comments Scraper FAQ

Why is a field empty in my results?

The Hacker News Stories & Comments Scraper fills every field Hacker publishes for that item. When Hacker does not show a value - for example a figure the owner hid - the field stays empty rather than being guessed.

Which formats can I download?

The Hacker News Stories & Comments Scraper dataset downloads as JSON, CSV, Excel (XLSX), XML, RSS or an HTML table, and the Apify API returns the same Hacker News Stories & Comments Scraper data page by page.

Can I schedule it?

Yes. Apify schedules run the Hacker News Stories & Comments Scraper hourly, daily or weekly, and a webhook can notify your system when new Hacker data is ready.

How is this different from other Hacker scrapers?

The Hacker News Stories & Comments Scraper combines complete public fields, no login, pay-per-result pricing and a stable output schema, so the Hacker News Stories & Comments Scraper fits both quick one-off exports and long-running automation.

Do you offer custom scrapers?

Yes. If you need a field or a filter the Hacker News Stories & Comments Scraper does not have yet, contact Neuro Scraper through the Apify Store page and we can extend the Hacker News Stories & Comments Scraper for you.

What does this Hacker scraper do?

The Hacker News Stories & Comments Scraper collects public Hacker news stories & comments and returns it as a structured dataset. It is web scraping without code: you set the input, start the Hacker News Stories & Comments Scraper and download JSON, CSV, Excel or XLSX.

The Hacker News Stories & Comments Scraper collects only publicly available data. If results contain personal data, process it under GDPR or the rules that apply to you, and only for a legitimate purpose.

What is the best Hacker scraper for news stories & comments?

The Hacker News Stories & Comments Scraper is built for exactly this job: public Hacker news stories & comments, complete fields, no login, pay per result and a clean export to JSON, CSV or Excel.

Can it replace the official Hacker API?

For public data, yes. The Hacker News Stories & Comments Scraper gives you Hacker data through the Apify API without an official Hacker API key, quota approval or developer account.

Can I export the results to Excel, CSV or Google Sheets?

Yes. Every Hacker News Stories & Comments Scraper run stores a dataset you can export as CSV, JSON, Excel (XLSX), XML or HTML, or send to Google Sheets with the Apify integration.

Does it work with proxies?

Yes. The Hacker News Stories & Comments Scraper uses Apify Proxy by default and you can pick the proxy group or country in the input.

What happens if a run is interrupted?

The Hacker News Stories & Comments Scraper saves its progress, so a resurrected or restarted run continues where it stopped instead of paying for the same rows again.

How do I get more results?

Raise the maximum items, add more inputs (keywords, URLs or profiles), or run the Hacker News Stories & Comments Scraper several times on a schedule. Results are de-duplicated.

Can I run it for many inputs at once?

Yes. Give the Hacker News Stories & Comments Scraper a list of inputs in one run; each row says which input it came from, so one Hacker News Stories & Comments Scraper run can cover a whole list.

How fast is it?

The Hacker News Stories & Comments Scraper runs requests in parallel and rotates proxy sessions automatically, so a typical run of a few hundred items finishes in minutes.

Can I send the data to my CRM?

Yes. Push Hacker News Stories & Comments Scraper rows to HubSpot, Salesforce, Pipedrive or any CRM through Zapier, Make or a webhook - a common setup for an agency or B2B sales team.

Do I need a Hacker account or login?

No. The Hacker News Stories & Comments Scraper works with no login and no cookies. It reads only public data, the same data any visitor can see.

More scrapers from Neuro Scraper

ActorWhat it extracts
⭐ Facebook Reels ScraperFacebook Reels with exact views, plays, every reaction type, comments and shares
Snapchat Profile ScraperSnapchat profile data with all public fields, exported to JSON, CSV or Excel
Amazon ScraperAmazon data with all public fields, exported to JSON, CSV or Excel
Instagram Reels ScraperInstagram reels with all public fields, exported to JSON, CSV or Excel
Facebook Video Details ScraperFacebook video details with all public fields, exported to JSON, CSV or Excel
Facebook Reels Scraper AdvancedFacebook reels advanced data with all public fields, exported to JSON, CSV or Excel
Facebook Reels DownloaderFacebook reels with all public fields, exported to JSON, CSV or Excel
Instagram Search ScraperInstagram search data with all public fields, exported to JSON, CSV or Excel
TikTok Video Details ScraperTikTok video details with all public fields, exported to JSON, CSV or Excel
Instagram Hashtag ScraperInstagram hashtag data with all public fields, exported to JSON, CSV or Excel
TikTok Profile ScraperTikTok profile data with all public fields, exported to JSON, CSV or Excel
Amazon Best Sellers ScraperAmazon best sellers with all public fields, exported to JSON, CSV or Excel
Facebook Photos ScraperFacebook photos with all public fields, exported to JSON, CSV or Excel