Hacker News Scraper: Search, Brand Monitoring & Who Is Hiring

Search Hacker News stories and comments by keyword, domain or brand with points and date filters, grab the front page, or extract every job post from the monthly 'Who is hiring?' thread with company, remote flag and parsed header fields. Uses the official HN Algolia API.

Why this actor

• Three use cases in one actor (search, front page, Who is hiring) at a mid price; parsed hiring posts.
• Pay only for results: $1.00 per 1,000 items, no start fee, failed lookups are free.
• Robust: automatic retries with exponential backoff, respects Retry-After, per-item error rows instead of crashed runs.
• Clean, flat JSON/CSV/Excel output ready for Google Sheets, Make, Zapier, n8n, Clay, or the Apify API.

Use cases

• Brand / launch monitoring for dev-tool companies
• Recruiters mining monthly 'Who is hiring?' posts
• Trend research for founders and VCs
• Dataset building for ML

How to use Hacker News Scraper: Search, Brand Monitoring & Who Is Hiring

1. Configure mode, queries, tags in the Input tab (or leave the prefilled sample values to test immediately).
2. Optionally set maxItems to cap the number of results and control spend.
3. Click Start and export structured results as JSON, CSV, Excel, or connect directly via API / MCP / n8n / Make.

Input

Field	Type	Description
mode	string	What to extract.
queries	array	Keywords, brand or domain names to find in stories/comments.
tags	string	Search stories, comments, both, Show HN or Ask HN.
sortBy	string	Newest first, or most relevant.
minPoints	integer	Only items with at least this many upvotes.
sinceDays	integer	Only items from the last N days (great for daily alerts).
maxResultsPerQuery	integer	Cap per query.
hiringThread	string	Which monthly thread.
keywords	array	Keep job posts mentioning any of these (python, remote, NYC...).
maxItems	integer	0 = no limit.

Example input:

{
  "queries": [
    "apify"
  ]
}

Output

Each dataset item looks like this (real sample from a test run, long fields trimmed):

{
  "type": "story",
  "id": "50016426",
  "title": "Show HN:APIFlow Studio \u2013 Open-source, offline-first Postman alternative",
  "url": "https://apiflowstudio.onrender.com/",
  "hnUrl": "https://news.ycombinator.com/item?id=50016426",
  "author": "yashpateldev",
  "points": 3,
  "numComments": 1,
  "createdAt": "2026-10-09T05:40:43Z",
  "text": null,
  "storyId": 50016426,
  "query": "apify",
  "scrapedAt": "2026-10-14T05:54:48+00:00"
}

Rows that could not be processed contain an error field and are not charged.

Extracted output fields

Every dataset row includes structured fields ready for spreadsheets, CRMs, and AI pipelines:

• Fields: type, id, title, url, hnUrl, author, points, numComments, createdAt, text, storyId, query, scrapedAt

Comparison: Hacker News Scraper: Search, Brand Monitoring & Who Is Hiring vs alternatives

Feature	This Actor (shawndunn/hacker-news-scraper)	Typical alternatives
Pricing model	$1 / 1,000 results, $0.00 start fee	$20–$49/mo subscription or per-run start fees
Failed / error lookups	$0.00 (never charged)	Often billed by compute time even on failures
API keys / proxies required	None (works out of the box)	Often requires paid third-party keys or residential proxies
AI Agent & MCP ready	Yes (mcp.apify.com compatible, optional defaults)	Strict required schemas that reject agent calls

Pricing

Pay-per-event: $0.001 per item (= $1.00 / 1,000). Use Max results to cap spend. Apify's free plan credit covers thousands of results per month.

Integrations & API

Run on a schedule in Apify Console, call via the Apify API / Python & JS clients, or connect to Make, Zapier, n8n, Google Sheets, Slack and webhooks. Works as a tool for AI agents via Apify MCP.

Use with AI agents (MCP)

Hacker News Scraper: Search, Brand Monitoring & Who Is Hiring is configured for zero-config AI agent tool use via the Apify MCP Server (no required-field blocking on defaulted inputs).

Streamable HTTP endpoint:

https://mcp.apify.com/?tools=shawndunn/hacker-news-scraper

Claude Desktop / Cursor / Windsurf mcpServers config:

{
  "mcpServers": {
    "apify-hacker-news-scraper": {
      "url": "https://mcp.apify.com/?tools=shawndunn/hacker-news-scraper",
      "headers": {
        "Authorization": "Bearer <YOUR_APIFY_TOKEN>"
      }
    }
  }
}

Direct synchronous API call:

POST https://api.apify.com/v2/acts/shawndunn~hacker-news-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>

FAQ

Is it legal? The actor only reads publicly available data from public endpoints, without logins, and does not bypass paywalls or access controls. You are responsible for using the data in line with applicable laws (e.g. GDPR) and the source sites' terms.

Something broke? Open an issue in the Issues tab; we typically respond within 24 hours and ship fixes fast.

More tools by shawndunn

• ATS Jobs Scraper — Scrape jobs from Greenhouse, Lever, Ashby, Workable, SmartRecruiters & Recruitee boards.
• RSS & Atom Feed Reader with Auto-Discovery — Turn any website or feed into clean JSON items: title, link, date, author, summary, categories, enclosures ($1/1k, no start fee).
• Job Posting Extractor — Extract structured job data from any careers page or job board that publishes Google-for-Jobs (schema.
• RSS Feed Finder — Find RSS, Atom and JSON feeds for any list of websites via link tags and common feed paths, with item count and latest post date to ($1/1k, no start fee).
• Remote Jobs Aggregator — Remote jobs from 5 public job feeds in one normalized schema with keyword filtering: title, company, location restrictions, tags ($1/1k, no start fee).