📰 Hacker News Intelligence — Tech Trends, Startups & Jobs with AI Engagement Scoring

Extract stories, discussions and job posts from Hacker News — Top, New, Best, Ask HN, Show HN and Who's Hiring jobs — as clean, structured data with an AI engagement score on every item. It's the fastest way to turn the front page into a dataset: spot trending tools before they go mainstream, discover new startup launches, track what developers are talking about, and source tech talent. Export to JSON/CSV/Excel, run on a schedule, call via API, or connect to Make, Zapier or n8n.

🧠 What is the Hacker News Intelligence Scraper?

It turns any Hacker News feed into a structured, filterable dataset. Pick which story types to pull (Top, New, Best, Ask, Show, Job), set thresholds for upvotes, comments, time range or keywords, and it returns every matching story with its score, comment count, author, timestamp and a 0–100 engagement score that ranks the hottest discussions first — so trend analysts, investors, recruiters and content curators see what matters without refreshing the homepage all day. It reads the official Hacker News API, so the data is fresh and reliable.

What data does it extract?

• Title and the story URL (plus the Hacker News discussion link)
• Story type — story, Ask HN, Show HN or job post
• Score (upvotes) and comment count (descendants)
• Author and timestamp (raw + ISO-formatted)
• Text/body for Ask HN, Show HN and job posts
• Top 5 comments per story (optional)
• AI engagement score (0–100) ranking discussion heat

⬇️ Input

Choose which feeds to pull and how to filter them:

Field	Description
storyTypes	Any of top, new, best, ask, show, job
maxResults	Maximum number of stories to return (1–500)
minScore	Only stories with at least this many upvotes
minComments	Only stories with at least this many comments
timeRange	all, 24h, week, or month
keywords	Only stories whose title or text contains these terms
includeEngagementScore	Toggle the 0–100 AI engagement score
fetchComments	Include the top 5 comments for each story

Example input

{
  "storyTypes": ["top", "show"],
  "maxResults": 100,
  "minScore": 50,
  "minComments": 10,
  "timeRange": "week",
  "keywords": ["AI", "rust", "startup"],
  "includeEngagementScore": true,
  "fetchComments": false
}

⬆️ Output

Every story is one clean row (view as a table, or export JSON / CSV / Excel):

{
  "id": 38765432,
  "title": "Show HN: I built an open-source alternative to Notion",
  "url": "https://github.com/example/project",
  "hn_url": "https://news.ycombinator.com/item?id=38765432",
  "type": "story",
  "by": "janedev",
  "author": "janedev",
  "score": 412,
  "descendants": 168,
  "comment_count": 168,
  "time": 1716800000,
  "time_formatted": "2026-06-26T15:30:00",
  "text": "",
  "kids": [38765500, 38765512, 38765540],
  "dead": false,
  "deleted": false,
  "engagement_score": 88
}

💡 Use cases

• 📈 Tech trend monitoring: track what's climbing the front page — filter by keywords like AI or rust and sort by engagement to catch tools before they go mainstream.
• 🚀 Startup & investor discovery: pull Show HN launches and high-engagement stories to find new products and founders early for sourcing or competitive intelligence.
• 🧑‍💻 Developer recruiting: scrape the job feed and Who's Hiring threads to surface hiring companies and roles, then build a targeted outreach list.
• 📨 Content curation & newsletters: grab the day's or week's highest-scoring stories with one run and feed them straight into a tech newsletter or dashboard.

🧮 How the engagement score works

A transparent 0–100 score combining score/upvotes (up to 40 points, with 500+ counting as viral), comment count (up to 30 points), recency (up to 20 points — fresher stories rank higher) and story type (jobs and stories weighted highest). Use it to sort feeds and surface the discussions with the most momentum right now.

❓ FAQ

How do I scrape Hacker News stories? Choose your storyTypes (e.g. top and show), set optional filters for score, comments, time range or keywords, then Run. You get structured stories with score, comments, author, timestamp and an engagement score.

Can I get Show HN and Ask HN posts? Yes — add show and/or ask to storyTypes. The body text of those posts is included in the text field.

Can I find tech jobs and Who's Hiring posts? Yes — add job to storyTypes to pull the Hacker News jobs feed, ideal for recruiting and sourcing.

Can I filter by keyword or topic? Yes — pass keywords (e.g. ["AI", "rust"]) and only stories whose title or text contains one of them are returned.

Can I get the comments too? Yes — enable fetchComments to include the top 5 comments (author, text, timestamp) for each story.

How do I get only the trending stories? Set a minScore and/or minComments threshold, choose a timeRange like 24h or week, and sort the output by engagement_score.

Do I need an API key? No. It reads the public Hacker News (Firebase) API — no key, token or login required.

Can I run it on a schedule or via API? Yes — schedule recurring runs in Apify, call it via the API/SDK, or connect it to Make, Zapier or n8n to push fresh stories into your stack automatically.

Is scraping Hacker News legal? It reads publicly available data from the official Hacker News API. Use it responsibly for research and monitoring, and follow applicable laws and Y Combinator's terms.

🔗 You might also like

• Dev.to Articles Intelligence — trending developer articles
• GitHub Repository Intelligence — trending repos and project data

---

Keywords: Hacker News scraper, Hacker News API, tech news scraper, Show HN, Ask HN, Who's Hiring, startup discovery, tech trends monitoring, developer recruiting, YC startups, trending tools, engagement score, content curation, tech newsletter, competitive intelligence, HN stories, upvotes and comments data.