Hacker News Intelligence - AI-Powered Tech Stories & Jobs Scraper

Extract trending tech stories, job postings, startup launches, and discussions from Hacker News with AI-powered engagement scoring.

Features

• Multiple Story Types: Top, New, Best, Ask HN, Show HN, Job postings
• AI Engagement Score: 0-100 scoring based on upvotes, comments, recency, and story type
• Advanced Filtering: Filter by minimum score, comments, time range, and keywords
• Job Postings: Access tech job listings from Hacker News
• Startup Discovery: Track Show HN posts for new products and startups
• Discussions: Monitor Ask HN threads for community insights
• Optional Comments: Fetch top 5 comments for each story
• Real-time Data: Direct access to Hacker News official API

Use Cases

🎯 Tech Recruiting

• Find companies hiring developers from job postings
• Track "Who's Hiring?" monthly threads
• Source top tech talent from engaged commenters

🚀 Startup Intelligence

• Discover new startups via Show HN posts
• Monitor product launches and reception
• Track YC company mentions and discussions

📈 Trend Analysis

• Monitor emerging tech trends (AI, blockchain, etc.)
• Track sentiment around technologies
• Analyze viral tech discussions

📰 Content Curation

• Aggregate top tech news for newsletters
• Find high-engagement content for sharing
• Track specific topics via keyword filters

💼 Market Research

• Competitive intelligence from community discussions
• Developer pain points from Ask HN
• Technology adoption trends

Input Configuration

Field	Type	Description	Default
Story Types	Array	Types to scrape: top, new, best, ask, show, job	["top"]
Max Results	Integer	Maximum stories to return (1-500)	100
Min Score	Integer	Minimum upvotes filter	0
Min Comments	Integer	Minimum comment count filter	0
Time Range	String	Filter by time: all, 24h, week, month	all
Keywords	Array	Filter by keywords in title/text	[]
Include Engagement Score	Boolean	Calculate AI engagement score	true
Fetch Comments	Boolean	Include top 5 comments per story	false

Output Format

Each story includes:

{
  "id": 45947810,
  "title": "Open-source Zig book",
  "url": "https://www.zigbook.net",
  "hn_url": "https://news.ycombinator.com/item?id=45947810",
  "type": "story",
  "by": "rudedogg",
  "score": 604,
  "descendants": 245,
  "comment_count": 245,
  "time": 1763322267,
  "time_formatted": "2025-01-17T10:24:27",
  "text": "",
  "author": "rudedogg",
  "engagement_score": 95,
  "dead": false,
  "deleted": false,
  "top_comments": []
}

AI Engagement Scoring Algorithm

The engagement score (0-100) is calculated using 4 factors:

1. Score/Upvotes (40 points max)

• 500+ upvotes: 40 points (viral)
• 200-499: 30 points (high)
• 50-199: 20 points (medium)
• 10-49: 10 points (low)

2. Comment Count (30 points max)

• 200+ comments: 30 points
• 100-199: 25 points
• 30-99: 15 points
• 10-29: 5 points

3. Recency (20 points max)

• <6 hours: 20 points (very fresh)
• <24 hours: 15 points (fresh)
• <72 hours: 10 points (recent)
• <7 days: 5 points (this week)

4. Type Criticality (10 points max)

• Job: 10 points
• Story: 8 points
• Poll: 6 points
• Comment: 4 points

Example Use Cases

Find Tech Jobs

{
  "storyTypes": ["job"],
  "maxResults": 50,
  "timeRange": "week"
}

Discover Trending Startups

{
  "storyTypes": ["show"],
  "minScore": 20,
  "maxResults": 100
}

Monitor AI/ML Discussions

{
  "storyTypes": ["top", "ask"],
  "keywords": ["AI", "machine learning", "LLM", "GPT"],
  "minComments": 10
}

High-Engagement Stories Only

{
  "storyTypes": ["top"],
  "minScore": 100,
  "minComments": 50,
  "timeRange": "24h"
}

Pricing

• Free Tier: First 100 stories = $0.00
• Paid Tier: $0.02 per story after 100

Example Costs:

• 100 stories = $0.00
• 500 stories = $8.00 (100 free + 400 × $0.02)
• 1,000 stories = $18.00 (100 free + 900 × $0.02)

Data Source

This actor uses the official Hacker News API:

• API: https://hacker-news.firebaseio.com/v0
• No authentication required
• No rate limits
• Free government data

Technical Details

• Language: Python 3.11
• Framework: Apify SDK
• HTTP Client: aiohttp (async)
• Rate Handling: Graceful timeout handling
• Data Format: JSON

Support

For issues or questions, contact the actor maintainer through Apify platform.

Attribution

Data provided by Hacker News (YCombinator).