Hacker News Scraper: Search, Mentions Hacker News Scraper: Search, Front Page, Mentions & Who Is Hiring Jobs Who Is Hiring Jobs

Get Hacker News data in three modes, through the official Hacker News APIs (no login, no browser):

1. Search stories, comments, Ask HN, Show HN, polls and job stories by keyword, by linked domain, by date range, points and comment count. With "only new" mode it becomes a mention monitor: schedule it and get only what was posted since the last run.
2. Ranked lists: the front page, new, best, Ask HN, Show HN and jobs, in their current order, with rank.
3. Who is hiring jobs: the monthly "Ask HN: Who is hiring?" thread as structured jobs: company, remote / onsite / hybrid, salary, employment type, visa, tech stack and apply link. Older months too (tested back to 2015).

• ✅ Comments of every story, flattened with their depth in the thread
• ✅ No 1,000-result cap when sorted by date (the search API alone stops at 1,000)
• ✅ Domain search: every story that links to a website, for example your own
• ✅ Clean text: HTML entities decoded, links kept in full
• ✅ No usernames: rows contain content, not people
• ✅ Fair billing: per result (story, comment or job post)

Input examples

Monitor mentions of a product (schedule it daily):

{
    "mode": "search",
    "queries": ["my-product", "myproduct.com"],
    "type": "all",
    "onlyNew": true,
    "maxResults": 500
}

Top stories about a topic, with the first 5 comments of each:

{
    "mode": "search",
    "queries": ["postgres"],
    "minPoints": 100,
    "maxResults": 30,
    "includeComments": true,
    "maxCommentsPerStory": 5
}

Remote jobs from the latest "Who is hiring?" thread that mention Rust:

{
    "mode": "whoIsHiring",
    "remoteOnly": true,
    "keywords": ["rust"]
}

Output examples

Real output from test runs. A story:

{
    "id": 44073588,
    "type": "story",
    "title": "Postgres IDE in VS Code",
    "url": "https://techcommunity.microsoft.com/blog/adforpostgresql/announcing-a-new-ide-for-postgresql-in-vs-code-from-microsoft/4414648",
    "domain": "techcommunity.microsoft.com",
    "points": 1040,
    "commentCount": 407,
    "text": null,
    "tags": [],
    "createdAt": "2025-05-23T15:12:44Z",
    "hnUrl": "https://news.ycombinator.com/item?id=44073588",
    "query": "postgres"
}

A comment (text shortened):

{
    "id": 18717600,
    "type": "comment",
    "text": "As a reminder, PostgreSQL is distributable under a license that's basically BSD/MIT: https://www.postgresql.org/about/licence/ …",
    "storyId": 18717168,
    "parentId": 18717168,
    "depth": 1,
    "createdAt": "2018-12-19T17:48:09.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=18717600"
}

A job post from "Who is hiring?" (text shortened):

{
    "postId": 49922616,
    "month": "2026-10",
    "threadTitle": "Ask HN: Who is hiring? (October 2026)",
    "postedAt": "2026-10-01T15:04:11.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=49922616",
    "company": "PAGNOS",
    "header": "PAGNOS | Cloud Engineer, AWS Serverless (TypeScript/Node.js) | REMOTE (Germany) | Full-time | €75k–80k | business-fluent German required",
    "headerParts": ["PAGNOS", "Cloud Engineer, AWS Serverless (TypeScript/Node.js)", "REMOTE (Germany)", "Full-time", "€75k–80k", "business-fluent German required"],
    "remote": true,
    "onsite": false,
    "hybrid": false,
    "salary": "€75k–80k",
    "employmentTypes": ["full-time"],
    "visaSponsorship": true,
    "technologies": ["TypeScript", "React", "Next.js", "Node.js", "AWS"],
    "applyUrl": "https://germantechjobs.de/jobs/PAGNOS-GmbH-Cloud-AI-Engineer--AWS-Serverless-mwd",
    "text": "PAGNOS | Cloud Engineer, AWS Serverless … We build serverless applications on AWS for clients and for our own SaaS products. …"
}

In our test, the October 2026 thread gave 259 job posts; 140 of them remote, 88 with a salary.

Pricing

A fixed price per result: one story, one comment or one job post. Runs are fast: 2,500 search results took 3 seconds, a whole "Who is hiring?" thread 3 seconds.

Paid Apify plans pay less: 10% off on Starter, 20% on Scale and 30% on Business and higher plans.

FAQ

How does "only new" work? The Actor remembers the newest item of each search in your key-value store hacker-news-monitor-state and next time asks only for newer items. The first run returns the latest items. If more new items exist than "Max results", the oldest of them are skipped, so set it high enough for your schedule.

Why at most 1,000 results with relevance sorting? That is a limit of the Hacker News search API. Sort by date and there is no limit.

How exact are salary, remote flag and tech stack? They are read from the post text with rules: remote, onsite and hybrid from the first line, the first salary-like amount, and a list of about 80 common technologies. The full post text is always included, so you can check or refine.

Why are there no usernames? The Actor returns content only. The "Who wants to be hired?" thread (posts by people) and the "seeking work" posts of the freelancer thread are left out for the same reason.

More tools from Tidelane Data

• RSS Feed Reader: news, blogs and podcasts to JSON
• Research Papers: search PubMed, arXiv and Crossref at once
• Remote Jobs Feed API: remote jobs by where you may work from
• Career Site Job Feed API: 2.5M+ jobs from 50,000+ career sites at once

All our tools: apify.com/tidelane

---

Impressum

Levi Griesinger, Tidelane Data
Besigheimer Straße 18
74369, Löchgau, Deutschland
E-Mail: LeviGriesinger@gmail.com