Hacker News Scraper — Search HN Stories, Comments, Polls, Show HN & Ask HN (No API Key)

Bulk-scrape Hacker News — the homepage of the technical internet — by keyword, item type, points threshold and date. Pull stories, comments, polls, Show HN, Ask HN and front-page items through the official public HN Algolia Search API and get title, author, URL, body text, points, comment count, tags, created date and the HN permalink in clean, flat JSON. Tens of thousands of items per run thanks to date-windowed pagination that breaks past Algolia's 1,000-result cap. No API key, no proxy, no login, no browser.

🏆 Why this Hacker News scraper?

12 flat fields per item · tens of thousands of items per run · pure public REST (no browser, no proxy) · beats the Algolia 1,000-row cap via date windows · export to JSON / CSV / Excel. The unofficial Hacker News API alternative for tech-trend research, Show HN launch tracking, sentiment monitoring and LLM training corpora.

---

✨ What this Actor does / Key features

• 🟠 Every HN item type — story, comment, poll, show_hn, ask_hn and front_page, all in one Actor.
• 🔎 Free-text keyword search — track any topic, product, company, language, framework, person or technology across all of Hacker News history.
• ⭐ Points filter — return only items above a minimum upvote threshold to cut low-signal noise.
• 📈 Beyond the 1,000-row cap — date-windowed pagination automatically slices queries into smaller time ranges (year → month → week → day) to pull tens of thousands of items per run.
• 🏷️ Rich Algolia tag metadata — the tags array is preserved so you can filter downstream to specific accounts (author_<username>), story types (show_hn, ask_hn) or parent stories (story_<id>).
• 🔗 Full attribution — every row carries the stable HN item ID (objectId) plus a direct hnUrl permalink to news.ycombinator.com.
• ♾️ Unlimited mode — set maxItems to 0 to pull every matching item.
• 🧱 Flat, export-ready rows — 12 columns, no nested JSON, straight to CSV, Excel, JSON, XML or JSONL.
• ⚡ Direct public REST — uses only the official hn.algolia.com Search API; no HTML scraping, no headless browser, no anti-bot games.
• 🔓 No auth, no proxy — no HN account, no Algolia key, no residential proxy required. Data is real-time (Algolia indexes new HN items within seconds).

🚀 Quick start (3 steps)

1. Configure — set an itemType (e.g. show_hn) and optionally a query keyword and a minPoints threshold. Set maxItems to 0 for everything.
2. Run — click Start. The Actor queries the Algolia HN Search API and, when needed, date-windows the request to pull past the 1,000-result cap.
3. Get your data — open the Output tab and export to JSON, CSV, Excel, XML or JSONL, or pull it via the Apify API.

📥 Input

Everything is optional — with no input at all, the Actor scrapes recent story items. Add a query, tighten minPoints, or switch itemType to focus your run.

Example — track every AI front-page story

{
  "query": "AI",
  "itemType": "front_page",
  "minPoints": 50,
  "maxItems": 0
}

Example — mine Show HN launches above 100 points

{
  "itemType": "show_hn",
  "minPoints": 100,
  "maxItems": 5000
}

Example — build an Ask HN knowledge corpus

{
  "itemType": "ask_hn",
  "minPoints": 20,
  "maxItems": 0
}

Example — track every comment about Rust

{
  "query": "rust",
  "itemType": "comment",
  "minPoints": 10,
  "maxItems": 20000
}

Field	Type	Description
query	string	Free-text keyword searched across all of Hacker News (e.g. openai, rust, acquired, AGI). Leave empty to scrape everything matching the other filters.
itemType	string	What kind of HN item to scrape. One of story, comment, poll, show_hn, ask_hn or front_page. Default story.
minPoints	integer	Only return items with at least this many points. Use to cut noise. Default 0 (no filter).
maxItems	integer	Cap on items saved (after the minPoints filter). Set 0 to pull everything matching your query; date-windowed pagination breaks past Algolia's 1,000-row cap.

Story vs. front_page: story = any submitted HN story; front_page = only stories that actually reached the HN front page, which is a strong popularity filter on its own. show_hn and ask_hn target the Show HN and Ask HN threads specifically.

📤 Output

One flat row per HN item — 12 fields, exportable to JSON, CSV, Excel, XML or JSONL. Here is a sample record:

{
  "objectId": "44321987",
  "type": "story",
  "title": "Show HN: I built a self-hosted ChatGPT alternative in 200 lines of Go",
  "author": "indiehacker42",
  "url": "https://github.com/example/mini-llm",
  "text": null,
  "points": 487,
  "numComments": 132,
  "tags": ["story", "show_hn", "author_indiehacker42"],
  "createdAt": "2026-05-15T08:21:00.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=44321987",
  "scrapedAt": "2026-07-06T10:00:00.000Z"
}

The Overview table view surfaces title, author, points, numComments, createdAt and url for quick scanning; the full 12-field schema is always available in the raw JSON/CSV export.

⚙️ How it works

1. Parses input — keyword, item type, minimum points and max items.
2. Picks the endpoint — uses hn.algolia.com/api/v1/search (relevance) or search_by_date depending on the strategy needed to break past the 1,000-row cap.
3. Builds Algolia filters — combines tags, numericFilters (points>=N, created_at_i<timestamp) and query into a single request.
4. Date-windowed pagination — when more than ~1,000 matches exist, slices the time range into smaller windows (year → month → week → day) until each window fits under Algolia's hard cap, then concatenates.
5. Deduplicates by objectId so overlapping windows never produce duplicate rows.
6. Normalizes each hit into the flat 12-field schema, then streams rows into the Apify Dataset as they arrive — safe to interrupt and resume.

The Actor uses ONLY the official, publicly documented Algolia HN Search API — no HTML scraping, no headless browser, no proxy, no Cloudflare bypass.

💡 Use cases

• Tech & product trend research — chart which languages, frameworks and AI models keep hitting the front page over time by grouping on tags or query.
• Show HN launch tracker — scrape show_hn daily to maintain a product launch board and spot the next devtool, AI wrapper or indie SaaS early.
• Ask HN knowledge mining — pull years of ask_hn threads for a founder/engineer knowledge base, search corpus or fine-tuning dataset.
• Brand & competitor sentiment — track every HN mention of your company, competitor, framework or product; combine query with minPoints to surface high-signal discussion.
• AI / LLM training data — HN comments are dense, opinionated, technically literate and well-attributed; scrape topic corpora for fine-tuning code/tech assistants.
• Newsletter & content curation — schedule a daily pull of yesterday's top stories above a points threshold, filtered by tag (ai, rust, security) for niche editions.
• Hiring & talent intel — find who posts Show HN launches, answers Ask HN questions or writes widely-upvoted comments in your domain.
• Journalism & market research — build a longitudinal dataset on a topic (story counts per week, sentiment shifts, who's commenting) by backfilling years of history.

👥 Who uses it

Startup founders & indie hackers · growth marketers & competitive-intelligence teams · dev-tool & DevRel product teams · AI/LLM researchers & data engineers · technical journalists & data journalists · VC analysts, product managers and technical recruiters.

💰 Pricing

This Actor runs on a simple pay-per-result model — you only pay for the HN items actually saved (after the minPoints and maxItems filters); items that don't pass the filter aren't billed, and there are no separate Apify platform fees to calculate. Try it on the free tier first, then scale up. See the Pricing tab on this page for the current rate.

❓ Frequently Asked Questions

Is it legal to scrape Hacker News? The Actor reads only publicly available HN content through the official Algolia Search API endpoint that Hacker News itself provides for programmatic access. You are responsible for using the data in compliance with HN's terms and applicable law.

Does Hacker News have a public API? Yes — HN's search is powered by the public, unauthenticated Algolia HN Search API (hn.algolia.com/api/v1/search). This Actor wraps it and handles pagination, retries and the 1,000-result cap so you get a clean dataset without writing any API code.

Do I need an API key or a login? No. The Algolia HN Search endpoint is fully public — no HN account, no Algolia API key, no login and no proxy are required. Only an Apify account.

How much data can I get? Tens of thousands of items per run. Algolia caps any single query at 1,000 results, so the Actor splits your query into successive date windows (via created_at_i filters), each fitting under the cap, then merges and deduplicates by objectId — letting you backfill years of history.

How does the Actor break past the 1,000-result cap? Algolia limits one query to 1,000 results. The Actor slices the time range into smaller windows (year → month → week → day) until each window returns under 1,000 rows, then concatenates and deduplicates the output by objectId.

Is this a Hacker News API alternative?

Yes. It wraps the official public HN Algolia Search API and handles pagination, retries and the 1,000-result cap for you, so you get a clean, structured dataset without writing any API code yourself.

Can I scrape Hacker News without an API key or login?

Yes. The Algolia HN Search endpoint is fully public and unauthenticated — no HN account, no API key, no proxy and no login are needed. The Actor pulls publicly available items over direct HTTP.

How do I export Hacker News data to CSV or JSON?

Every run writes flat 12-column records to the Apify Dataset, which you can download directly as CSV, JSON, Excel, XML or JSONL — no post-processing needed — or pull via the Apify API.

Can I scrape both stories and comments in one run?

One run targets one itemType. Schedule two runs (one for story, one for comment) — they share the same 12-field dataset format and can be merged downstream. Comment rows include the full text and a story_<id> tag so you can join comments back to their parent stories.

How do I find the highest-scoring HN stories on a topic?

Set a minPoints threshold together with your query, and the Actor filters out low-signal noise so only widely upvoted HN discussions land in your dataset.

How fresh is the data?

Real-time. Algolia indexes new Hacker News items within seconds of posting, so a scheduled run always captures the latest stories, comments and launches.

What's the difference from the HN "Who is Hiring?" scraper?

This Actor is general-purpose HN search. The separate hacker-news-who-is-hiring-scraper is purpose-built for the monthly "Who is hiring?" thread — parsed by company, role, location, remote and tech stack.

🔗 More social & community scrapers by logiover

Building a cross-platform community-discussion dataset? Pair Hacker News with the rest of the social/dev/content suite:

Source	Actor
👽 Reddit	Reddit Search Scraper · Reddit Subreddit Scraper · Reddit Historical Archive Scraper
💬 Q&A	Stack Exchange Questions Scraper
👩‍💻 Dev content	Dev.to Articles Scraper · Substack Newsletter Scraper
🐦 Microblogs	X / Tweet Scraper · Threads Scraper
▶️ YouTube	YouTube Search Scraper
🎮 Communities	Steam Game Reviews Scraper · Product Hunt Daily Launches Scraper

👉 Browse all logiover scrapers on Apify Store — 180+ actors across real estate, jobs, crypto, social media & B2B data.

⏰ Scheduling & integration

Schedule this Actor on Apify to build a continuously growing HN dataset — every 15 minutes for real-time brand/competitor alerts, hourly for front-page + Show HN tracking, daily at 08:00 UTC for newsletter ingestion, or weekly for trend dashboards. Export results to JSON, CSV or Excel, sync to Google Sheets, or push to your database, BI tools and webhooks through the Apify API. Connect it to Make, n8n or Zapier to fan out new items into Slack, Discord, Notion or any HTTP endpoint the moment they appear.

⭐ Support & feedback

Found a bug or need an extra field? Open an issue on the Issues tab — response is usually fast. If this Actor saves you time, a ★★★★★ review on the Store page genuinely helps and is hugely appreciated. 🙏

⚖️ Legal

This Actor extracts only publicly available data through the official Algolia HN Search API and is intended for legitimate research, analytics and monitoring use. You are responsible for complying with Hacker News' terms of service, GDPR and any applicable local laws.

---

📝 Changelog

2026-07-06

• ✨ README overhaul: added a badge row, ready-to-run example scenarios, a collapsible full field reference, an expanded high-intent FAQ (API alternative, no-login export, data volume) and a curated social/community cross-promo grid.

2026-07-01

• Maintenance pass: re-verified end-to-end on live data and confirmed successful runs within the 5-minute quality window on the default input.
• Sharpened Store metadata (SEO title & description) and expanded the FAQ with high-intent, long-tail questions for easier discovery in Google and Apify Store search.
• Added ready-to-run example tasks that cover common real-world use cases.

2026-06-15

• Reliability pass: re-verified end-to-end on live data with real-world inputs. Routine maintenance build.

2026-06-07

• Docs: added coverage for using this as a Hacker News API alternative, exporting HN data to CSV/JSON, and scraping HN without login.

2026-06-05

• 🛡️ Reliability fix: results are no longer dropped by strict output validation — runs now complete cleanly even at high volume (thousands of results).
• ⚡ Stability & performance hardening; fresh rebuild.

2026-06-04

• Verified live & refreshed build — reliability/maintenance pass.

2026-06-01

• Maintenance & reliability pass: pulled the latest source and rebuilt the Actor on the current base image; build verified.

2026-05-25

• Maintenance & reliability pass: pulled the latest source and rebuilt the Actor on the current base image; build verified.

2026-05-20

• Maintenance pass: reviewed the input schema and default values for a smooth one-click start, and rebuilt the Actor on the latest base image.