Hacker News Scraper — Stories, Comments, Jobs & Users

Scrape Hacker News at scale: stories, Show HN, Ask HN, comments and user profiles — with title, URL, author, points, comment count and date. Search by keyword or tag, filter by points/comments, pull any story's comments (including monthly "Who's Hiring" job posts), and fetch user profiles. Clean JSON/CSV/Excel in seconds. Built for tech-trend tracking, research and sourcing.

---

What This Actor Does

Pull structured data from Hacker News:

• Stories — title, URL, author, points, comment count, date, self-post text, HN link
• Show HN / Ask HN / front page — by tag
• Comments — author, text, points, parent story — including the job posts inside "Ask HN: Who is hiring?" threads
• Users — karma, about, account age, submission count

Search by keyword, filter by minimum points or comments, sort by popularity or date, pull a specific story's comments, or fetch user profiles.

---

Why Use This

• Track tech trends. Pull top or newest stories on any topic with points and discussion counts.
• Mine "Who's Hiring". Extract every job post from the monthly hiring thread as structured rows.
• Research & monitoring. Follow Show HN launches, Ask HN questions and front-page movement.
• Open and free. Reads the public HN Search (Algolia) and official APIs — no key, no login, no anti-bot.

---

Quick Start

Run it in the console (no code)

1. Add search terms and a tag (story, show_hn, ask_hn…), or story IDs for comments, or usernames.
2. Optionally set min points / comments and sort order.
3. Set Max items, click Start, export as JSON, CSV, Excel or HTML.

Top stories on a topic (Python)

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("YOUR_USERNAME/hackernews-scraper").call(run_input={
    "searchTerms": ["rust"], "tags": "story", "minPoints": 100, "maxItems": 500,
})
for s in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(s["points"], "·", s["title"], "·", s["url"])

Extract "Who is hiring?" job posts (Python)

# Pass the monthly thread's story ID (find it by searching "Ask HN: Who is hiring?")
run = client.actor("YOUR_USERNAME/hackernews-scraper").call(run_input={
    "storyIds": ["22665398"], "maxItems": 1000,
})
for c in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(c["author"], "·", c["commentText"][:120])

---

Input Parameters

Field	Type	Description
searchTerms	array	Keywords to search stories/comments.
tags	string	story, show_hn, ask_hn, front_page, poll or comment.
sortBy	string	popularity (relevance) or date (newest).
minPoints	integer	Only items with ≥ this many points.
minComments	integer	Only items with ≥ this many comments.
storyIds	array	Fetch the comments of these story IDs (e.g. a Who's Hiring thread).
users	array	HN usernames to fetch profiles for.
maxItems	integer	Max items across all queries. 0 = no limit.
maxPagesPerQuery	integer	Pagination cap per query (100 per page).
proxyConfiguration	object	Apify Proxy. Datacenter is enough (public API).

---

Output

{
  "id": "22238335", "type": "story",
  "title": "Why Discord is switching from Go to Rust",
  "url": "https://blog.discord.com/...", "author": "Sikul",
  "points": 1582, "numComments": 642, "createdAt": "2020-02-04T17:30:40Z",
  "hnUrl": "https://news.ycombinator.com/item?id=22238335"
}

Comment and user records use type: "comment" / type: "user" with the relevant fields (comment text + parent story, or karma + about + submission count).

---

Use Cases

1. Tech trend tracking — top/newest stories on any topic, filtered by points.
2. "Who's Hiring" job extraction — turn the monthly thread into a structured job list.
3. Launch & sentiment monitoring — Show HN launches, Ask HN questions, front-page movement.
4. Community research — user profiles, karma and submission history.

---

Tips

• minPoints quickly filters to the signal (e.g. ≥100 points).
• sortBy: "date" gives newest-first for monitoring; popularity for best-of.
• For Who's Hiring, first search "Ask HN: Who is hiring?" to find the month's story ID, then pass it in storyIds.
• tags: "show_hn" or "ask_hn" targets those sections directly.
• Schedule it with Apify Schedules to track a topic or the monthly hiring thread.

---

Frequently Asked Questions

Do I need an account or key? No. The HN Search (Algolia) and official APIs are public and free — no login, key or anti-bot.

How do I get job posts from Who's Hiring? Each job is a top-level comment on the monthly "Ask HN: Who is hiring?" story. Pass that story's ID in storyIds and the Actor returns every comment.

Can I get the newest stories instead of the most popular? Yes — set sortBy to date.

How far back can I search? HN Search indexes the full history; use minPoints/minComments and pagination to scope results.

What export formats are supported? JSON, CSV, Excel, HTML, or via API — plus Google Sheets, webhooks, Make and Zapier.

---

Legal & Responsible Use

This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by Hacker News or Y Combinator. It reads only public data. Use it responsibly and in line with applicable terms and laws.