Hacker News Scraper — Extract Stories, Comments & Engagement Data

Scrape Hacker News stories, comments, and engagement metrics using the official Algolia Search API. Filter by keywords, content type, date range, and sort order to get exactly the data you need for trend analysis, content curation, market research, and competitive intelligence.

Use Cases

• Trend Analysis — Track what technologies, products, or topics are gaining traction on Hacker News.
• Content Curation — Discover high-performing stories in your niche for newsletter curation or social sharing.
• Market Research — Monitor discussions about your product, competitor, or industry to gauge sentiment and interest.
• Competitive Intelligence — Identify which stories get the most engagement and understand what resonates with the HN community.
• Academic Research — Collect large datasets of tech community discussions for NLP, sentiment analysis, or social science research.

Input

Field	Type	Required	Description
searchQueries	Array	No	Keywords to search for. Leave empty to fetch top/front page stories.
tags	String	No	Content type filter: story, comment, poll, show_hn, ask_hn, front_page (default: story)
timeRange	String	No	Filter by time: all, last24h, lastWeek, lastMonth, lastYear (default: all)
sortBy	String	No	Sort order: relevance, date, points (default: relevance)
maxItems	Number	No	Maximum results to return, 1–1000 (default: 100)
proxyConfiguration	Object	No	Optional proxy configuration for redundancy

Output

The Actor outputs a dataset with the following fields:

{
  "objectID": "21530860",
  "title": "John Carmack: I'm going to work on artificial general intelligence",
  "url": "https://www.facebook.com/100006735798590/posts/2547632585471243/",
  "author": "jbredeche",
  "points": 1574,
  "numComments": 889,
  "createdAt": "2019-11-13T23:17:23Z",
  "storyText": "",
  "tags": ["story", "author_jbredeche", "story_21530860"],
  "searchQuery": "artificial intelligence"
}

Pricing

Pay per event: $0.001 per story scraped.

No hidden fees. You only pay for the data you extract. The Algolia Search API is free and public, so there are no additional API costs.

Limitations

• Algolia HN Search API returns up to 1,000 hits per query.
• Very old stories may not be indexed.
• The API does not support sorting by points directly; sortBy: points falls back to relevance.
• Rate limits are generous but not officially documented; the Actor uses conservative concurrency to avoid issues.

FAQ

Q: Do I need an API key? A: No. The Hacker News Algolia Search API is completely free and requires no authentication.

Q: Can I scrape comments instead of stories? A: Yes. Set tags to comment to extract comments matching your search queries.

Q: How far back does the data go? A: The Algolia index covers the full history of Hacker News, but very old items may have incomplete metadata.

Q: What happens if I reach my spending limit? A: The Actor stops gracefully and saves all data collected up to that point.

Changelog

• v1.0.0 — Initial release with search, filtering, and pagination support.