Hacker News Scraper & "Who Is Hiring?" Jobs Parser

Scrape Hacker News with the official HN API and HN Search by Algolia: the front page and the new, best, Ask HN, Show HN and job lists, full-text search with points and date filters, comment trees, and the monthly "Ask HN: Who is hiring?" thread parsed into structured jobs (company, roles, locations, remote, salary, visa sponsorship, technologies and links). Fast, cheap ($0.30 per 1,000 items) and with no browser.

• ✅ Story lists in live ranking order: top (front page), new, best, ask, show, job
• ✅ Search stories or comments by keyword, with minimum points or comments, a date range, and relevance or newest-first sorting
• ✅ Comments for every story, depth-first in thread order, with depth and parent_id
• ✅ Who is hiring parser: one clean record per job post, with salary ranges normalised to {min, max, currency, period}
• ✅ Filters: keywords, exclude words, remote only, locations, only posts with salary
• ✅ Only new items monitoring mode for keyword alerts and new hiring posts. You pay only for new items.

Input examples

Front page:

{
  "mode": "stories",
  "storyLists": [
    "top"
  ],
  "maxItems": 30
}

Remote jobs with salary from this month's "Who is hiring?" thread:

{
  "mode": "whoishiring",
  "hiringThread": "latest",
  "remoteOnly": true,
  "onlyWithSalary": true,
  "keywords": [
    "python",
    "rust"
  ],
  "maxItems": 500
}

A keyword alert (schedule it every hour):

{
  "mode": "search",
  "searchQuery": "postgres",
  "searchSort": "date",
  "minPoints": 10,
  "maxItems": 50,
  "onlyNewItems": true
}

Output example (real run)

A story:

{
  "record_type": "story",
  "list": "top",
  "rank": 1,
  "id": 50180873,
  "title": "PHP 8.2.35",
  "url": "https://github.com/php/php-src/releases/tag/php-8.2.35",
  "domain": "github.com",
  "author": "tuyuri",
  "points": 852,
  "num_comments": 479,
  "created_at": "2026-10-21T13:45:33Z",
  "text": null,
  "hn_url": "https://news.ycombinator.com/item?id=50180873",
  "parent_id": null,
  "story_id": null,
  "dead": null,
  "scraped_at": "2026-10-21T19:03:26Z"
}

A parsed "Who is hiring?" post (record_type: hiring_post, text shortened):

{
  "record_type": "hiring_post",
  "id": 49922609,
  "thread_id": 49922569,
  "thread_title": "Ask HN: Who is hiring? (October 2026)",
  "thread_month": "October 2026",
  "author": "mcintyem",
  "posted_at": "2026-10-01T15:03:36Z",
  "company": "Sidekick Labs",
  "roles": [
    "Senior SWE, R&D Engineer, Intelligence Engineer"
  ],
  "locations": [
    "REMOTE (US Timezones)"
  ],
  "remote": true,
  "workplace_types": [
    "remote"
  ],
  "employment_types": [
    "full-time"
  ],
  "salary": {
    "min": 170000,
    "max": 230000,
    "currency": "USD",
    "period": "year",
    "text": "$170k to $230k"
  },
  "equity": true,
  "visa_sponsorship": null,
  "technologies": [
    "Python",
    "Go"
  ],
  "urls": [
    "https://www.linkedin.com/in/three-two-one",
    "https://sidekicklabs.com"
  ],
  "header": "Sidekick Labs | Senior SWE, R&D Engineer, Intelligence Engineer | REMOTE (US Timezones) | Full-time | $170k to $230k Base + Equity + Health",
  "text": "Sidekick Labs | Senior SWE, R&D Engineer, Intelligence Engineer | REMOTE (US Timezones) | Full-time | $170k to $230k Base + Equity + Health\n\nSidekick builds AI security software for enterprises. Our sensors observe AI…",
  "hn_url": "https://news.ycombinator.com/item?id=49922609",
  "replies": 0,
  "scraped_at": "2026-10-21T19:06:23Z"
}

Use the Stories & comments or Who is hiring: jobs dataset views, or export JSON, CSV or Excel. The OUTPUT record has per-task counts and errors.

Pricing

Pay per event, no subscription:

Event	Price
Item result (story, comment, job or poll)	$0.0003 ($0.30 per 1,000)
Parsed hiring post	$0.002 ($2 per 1,000)
Actor start (per run)	$0.002

The default run (front page, 30 stories) costs about $0.011. A whole monthly "Who is hiring?" thread (about 250–400 posts) costs about $0.50–0.80. Filtered-out items are never charged, and you can set a maximum cost per run.

Use cases

• Tech job boards and newsletters: structured, filterable jobs from the monthly HN hiring thread.
• Recruiting and sales intelligence: which startups are hiring, where, for which stack, and at what salary.
• Brand and keyword monitoring: get every new story or comment that mentions your product.
• Trend research: top stories, domains and discussions over time.
• AI and RAG datasets: clean HN discussions with thread structure.

FAQ

Which APIs are used? The official Hacker News API (hacker-news.firebaseio.com) for story lists and items, and HN Search by Algolia (hn.algolia.com/api) for search, comment trees and hiring threads. Both are public, read-only and need no login. Requests are rate-limited politely.

How accurate is the hiring parser? HN posts are free text. Most follow the Company | Role | Location | REMOTE | Salary header convention, and the parser reads that line and then scans the body for salary, visa and technologies. Fields it can't find are left null or empty, and the full text and the hn_url are always included so you can check.

Can I get older hiring threads? Yes. Set hiringThread to a month such as September 2026, a thread URL or ID, or use latest with hiringMonths: 6 to parse the last six threads.

How does "Only new items" work? Delivered item IDs are remembered per mode in a named key-value store (stateStoreName), and later runs skip them. Run the hiring mode daily and you only get, and pay for, posts added since the last run.

Related actors

More Actors by the same author:

• Substack Scraper (sysco32/substack-newsletter-scraper): posts, likes, comments and full text of free posts from any public Substack newsletter.
• Remote Jobs Scraper (sysco32/remote-jobs-feed): remote jobs from Jobicy, Remote OK, Arbeitnow and company career pages.
• ATS Jobs Scraper (sysco32/ats-jobs-feed): jobs from 9 applicant tracking systems in one schema, with ATS auto-detection and only-new-job alerts.
• RSS Feed Reader & Monitor (sysco32/rss-feed-reader): RSS, Atom and JSON Feed items as clean JSON, with only-new-item monitoring.

Need a custom scraper or data pipeline?

If you need data from another community or news site, extra fields, or the results delivered to Google Sheets, a database or an n8n workflow on a schedule, I take custom projects on Fiverr: Python scraping scripts and n8n automation workflows. Public data only; message me with the details first and I'll confirm whether it's doable.