Hacker News Trend Aggregator

Scrape Hacker News stories and comments at scale — keyword search, date/points filters, and live top/best/new/Ask/Show/Jobs rankings — through YC's official Algolia + Firebase APIs. No account, no API key, no proxy spend.

The fastest way to turn Hacker News into structured, analysis-ready data for trend monitoring, market research, and AI pipelines. One input, one clean schema, pay only for the rows you keep.

What it does

• Keyword search — query stories and/or comments by relevance or date, with a since/until date range and minPoints/minComments thresholds.
• Live rankings — pull the current Top, Best, New, Ask HN, Show HN, or Jobs lists straight from HN's Firebase feed.
• Real depth on date search — Algolia caps any single query at 1000 hits; this actor walks the time window automatically to pull well beyond 1000 results per term in date mode (verified live).
• Unified output — stories and comments in one dataset with consistent field names, ready for a database, a dashboard, or a RAG index.

Why use this actor

Alternatives	Their gap	This actor
Search-only HN scrapers	No live front-page / ranked lists	Keyword search and Top/Best/New/Ask/Show/Jobs
Generic multi-source scrapers	No HN-native fields	Native points, num_comments, and date filtering
Roll-your-own API calls	Algolia's 1000-hit wall, item hydration, retries, and charge logic to build and maintain	Time-window pagination, bounded-concurrency hydration, and pay-per-result — done

Hacker News is the canonical early signal for startups, AI, and developer tooling. What trends here often trends everywhere weeks later — which makes it a high-value input for trend dashboards, competitive research, and AI training/retrieval sets.

Built to never break your pipeline: every upstream call is wrapped with retry-and-backoff on transient errors and graceful degradation on hard errors. If HN's API hiccups, the run keeps the results it already has and finishes cleanly — it does not crash to an empty, failed run.

Output fields

Field	Type	Description
type	string	story, comment, job, or poll
object_id	string	HN item ID
url	string	HN permalink (news.ycombinator.com/item?id=…)
external_url	string	Linked article URL (stories)
title	string	Story title (or parent story title for comments)
text	string	Self-text (Ask/Show) or comment body
author	string	HN username
author_url	string	HN user profile link
points	integer	Score
num_comments	integer	Comment count
story_id	string	Parent story ID (comments)
story_title	string	Parent story title (comments)
created_at	string	ISO 8601 timestamp
created_at_i	integer	Unix timestamp
tags	array	HN / Algolia tags

Usage examples

High-signal AI stories from 2025, newest first (uses time-window depth to exceed 1000):

{
  "mode": "search",
  "searchTerms": ["AI agents", "LLM"],
  "itemType": "story",
  "sortBy": "date",
  "minPoints": 50,
  "since": "2025-01-01T00:00:00Z",
  "maxResults": 3000
}

Live front-page snapshot for a daily dashboard:

{
  "mode": "top",
  "maxResults": 100
}

Comment mining on a topic (sentiment / discourse analysis):

{
  "mode": "search",
  "searchTerms": ["kubernetes"],
  "itemType": "comment",
  "sortBy": "date",
  "maxResults": 2000
}

Pricing

Pay per result: $0.001 per item (story or comment) pushed to your dataset.

Example: 10,000 items ≈ $10. You are charged only for rows delivered, and the run stops automatically when your spending limit is reached — no overrun.

Technical notes

• Uses the official hn.algolia.com search API and hacker-news.firebaseio.com Firebase API — both free, public, and keyless.
• Zero proxy cost — official public APIs, no anti-bot bypass needed.
• Depth: sortBy: "date" walks the time window to pull far beyond Algolia's 1000-hit-per-query ceiling. sortBy: "relevance" returns up to Algolia's 1000 most-relevant hits per term (relevance order is not time-monotonic, so it cannot be windowed — use date for large pulls).
• Ranked lists (Top/Best/New/…) are hydrated with bounded concurrency for fast, low-latency runs.
• Resilient by design: transient failures (429 / 5xx / network) retry with exponential backoff; deterministic errors degrade gracefully and preserve partial results. Runs finish clean instead of failing.

Common use cases

• AI & startup intelligence — track what the HN community is building and debating.
• RAG pipelines — feed fresh, high-signal tech discourse into retrieval-augmented systems.
• Trend monitoring — schedule daily pulls of Top/Best lists into a dashboard.
• Developer-tooling & market research — mine Show HN launches and Ask HN demand signals.
• Sentiment & discourse analysis — collect full comment threads on any topic.

Data compliance

This actor accesses only public Hacker News data through YC's official, publicly documented APIs. No login credentials are used. Hacker News content is intentionally public; the data is suitable for research and business use.