Pricing

Pay per usage

Hacker News Deep Scraper

Extract stories, points, authors, and links from Hacker News with full metadata. Track trends, monitor tech discussions, and export structured data.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Josh Pinkerton

Actor stats

Bookmarked

Total users

Monthly active users

5 hours ago

Last modified

Hacker News Deep Scraper

Extract stories, points, authors, comment counts, and timestamps from Hacker News with full metadata. Track tech trends, monitor competitor mentions, or export structured HN data for analysis.

Why use this scraper?

Export HN front-page and paginated stories as structured JSON
Track technology trends over time by re-running on a schedule
Monitor mentions of specific companies, products, or keywords
Feed downstream research pipelines and data warehouses

How to use

Open the Hacker News Deep Scraper in Apify Console
Enter one or more start URLs (default: https://news.ycombinator.com)
Set maxPages for how many paginated pages to crawl
Click Run and watch the Output tab populate in real time

Input

Field	Type	Description	Default
`startUrls`	array	HN pages or sections to scrape	`https://news.ycombinator.com`
`maxPages`	integer	Max paginated pages to follow	`10`

Output

{
  "type": "story",
  "title": "Show HN: My new project",
  "url": "https://example.com",
  "score": "142",
  "author": "joshua",
  "num_comments": "56",
  "created": "2 hours ago",
  "page": 1,
  "source": "https://news.ycombinator.com",
  "scraped_at": "2026-07-05T17:00:00.000Z"
}

You can download the dataset in JSON, CSV, Excel, or XML from the Apify Console.

Pricing

Pay-per-event: $0.01 per extracted result. A typical front-page crawl costs less than $0.20.

Tips

Start with maxPages: 3 to validate selectors before scaling
Combine with the Apify Scheduler to run hourly for trend tracking
Use the title and url fields as keys for deduplication across runs

FAQ

Is it okay to scrape Hacker News? HN is generally scraper-friendly, but respect rate limits and do not hammer the site with concurrent requests.

Where does the data go? Results are stored in an Apify dataset you can export or connect to Make/Zapier/Google Sheets.

Hacker News Search — Stories, Comments & Developer Sentiment

ryanclinton/hackernews-search

Search and extract stories, comments, polls, Show HN, and Ask HN posts from Hacker News. This actor uses the Algolia HN Search API to find content by keyword, filter by author, date range, minimum points, and comment count -- then returns clean, structured JSON ready for analysis, monitoring, or ...

Ryan Clinton

Identity Risk Analyzer

rexreus/Identity-Risk-Analyzer

Map anyone's digital footprint across 15+ platforms in seconds. Collects public profiles from social media, forums & dev platforms — runs identity clustering, risk scoring, breach detection, and generates an interactive graph. For security research, HR & fraud prevention.

REXREUS D.O

5.0

(2)

Jobs & ATS MCP Server - Greenhouse, Lever, Ashby, HN Hiring

fetchcraft/recruiting-jobs-mcp

MCP server giving AI agents 4 recruiting tools: pull live jobs from Greenhouse/Lever/Ashby/Workable, deep Greenhouse boards, Hacker News Who-is-Hiring, and buying-signal hiring detection. Streamable HTTP, works with Claude/Cursor/Cline. $0.10 per tool call.

Emily Ward

🗄️ Reddit Archive Scraper - Years of Posts & Comments

benthepythondev/reddit-archive-scraper

Reddit Archive Scraper to extract years of historical Reddit posts and comments from the PullPush archive. Reddit's API caps subreddits at ~1000 posts; this Actor pulls months or years from many subreddits by date range and keyword. For historical backfill, research and AI datasets.

ben

Hacker News Scraper

constructive_calm/hacker-news-scraper

Scrapes Hacker News stories, comments, jobs, polls, and user profiles via the official Firebase and Algolia APIs. Supports full-text search, Who's Hiring thread extraction, author karma snapshots, and deep comment trees.

Omar Eldeeb

5.0

(1)

GitHub Repository Scraper - Stars, Topics, Trending

logiover/github-repository-scraper

Scrape GitHub repos by search query and export stars, topics, forks & license to CSV/JSON. GitHub data export without an API key - trending repos scraper.

Logiover

Dark Funnel Scraper

lissome_dancer/dark-funnel-scraper

Find B2B buyers before they fill a form. Monitors Reddit, G2, GitHub, HackerNews and LinkedIn for competitor switching signals and buying intent. Outputs CRM-ready leads with intent scores and outreach angles. Pay only for high-intent leads delivered.

Rohith S

1.0

(1)

Hacker News Scraper

crawlerbros/hackernews-scraper

Scrape Hacker News stories, comments, jobs, and user profiles. Modes: top/new/best/ask/show/jobs/past/item/user/search. Filters: minScore, domainFilter, dateRange, commentMinScore. No proxy, no auth.

Crawler Bros

Hacker News Stories & Comments Scraper

taroyamada/hacker-news-intelligence

Extract trending tech discussions, nested comment hierarchies, and post scores from Hacker News directly into structured JSON for custom RAG pipelines.

naoki anzai

Indeed Job Scraper

junipr/indeed-job-scraper

Scrape Indeed jobs with title, company, location, salary, type, description, posting date. Filter by date, type, experience, remote. Anti-detection with fingerprint rotation and residential proxy. Click-based pagination.