📰 Hacker News Intelligence — Stories, Comments & Trends

🔗 View on Apify Store | 🇺🇸 English | 🇧🇷 Português

Scrape Hacker News stories and comments using the public Algolia HN API and Firebase API. Search by keyword, browse the front page, get top stories, or explore Show HN and Ask HN submissions. No API key needed.

✨ Features

• 🔍 5 scraping modes — Search, Front Page, Top Stories, Show HN, Ask HN
• 🔑 Keyword search — Find stories by topic with relevance or date sorting
• 🕐 Time range filters — Past 24h, week, month, year, or all time
• 💬 Comment extraction — Fetch top comments per story (optional)
• 📊 Full metrics — Points, comment count, author, timestamps
• 🔗 Direct links — Original URL + HN discussion URL
• 🛡️ Anti-placeholder guardrails — Every result validated as real data
• ⚡ API-based — Fast and reliable via Algolia HN Search + Firebase

📥 Input

Parameter	Type	Required	Default	Description
mode	string	✅	"search"	Scraping mode: search, front_page, top_stories, show_hn, ask_hn
searchQueries	string[]	❌	—	Keywords to search (required in "search" mode, up to 10)
sortBy	string	❌	"relevance"	Sort: relevance, date
timeRange	string	❌	"all"	Time filter: all, 24h, week, month, year
maxItems	integer	❌	50	Max stories to scrape (1-500)
includeComments	boolean	❌	false	Fetch top comments for each story
maxCommentsPerStory	integer	❌	5	Max comments per story (1-20)

Input Example

{
  "mode": "search",
  "searchQueries": [
    "AI startup",
    "machine learning",
    "open source LLM"
  ],
  "sortBy": "date",
  "timeRange": "week",
  "maxItems": 100,
  "includeComments": true,
  "maxCommentsPerStory": 5
}

📤 Output

Every story includes these fields:

Field	Type	Description
storyId	string	Hacker News story ID
title	string	Story title
url	string	Original link URL
author	string	Submitter username
points	number	Upvote count
numComments	number	Total comment count
createdAt	string	Submission timestamp (ISO 8601)
storyText	string	Story body text (for Ask HN / Show HN)
tags	string[]	HN tags (story, show_hn, ask_hn, etc.)
hnUrl	string	Hacker News discussion URL
platform	string	Always "hackernews"
scrapedAt	string	ISO 8601 timestamp
topComments	object[]	Top comments (when includeComments is enabled)

Comment Fields

Field	Type	Description
commentId	string	Comment ID
author	string	Commenter username
text	string	Comment text (HTML stripped)
createdAt	string	Comment timestamp
points	number	Comment points

Output Example

{
  "storyId": "39847261",
  "title": "Show HN: We built an open-source alternative to Notion with local-first sync",
  "url": "https://github.com/AppFlowy-IO/AppFlowy",
  "author": "annieflowy",
  "points": 847,
  "numComments": 213,
  "createdAt": "2026-03-04T16:42:31.000Z",
  "storyText": "",
  "tags": ["story", "show_hn", "front_page"],
  "hnUrl": "https://news.ycombinator.com/item?id=39847261",
  "platform": "hackernews",
  "scrapedAt": "2026-03-06T14:15:22.108Z",
  "topComments": [
    {
      "commentId": "39848102",
      "author": "tptacek",
      "text": "This is impressive work. The offline-first approach with CRDT sync is the right way to build collaborative tools.",
      "createdAt": "2026-03-04T17:08:14.000Z",
      "points": 142
    }
  ]
}

📋 Use Cases

• Tech Trend Detection — Track which technologies and startups are gaining traction
• Content Marketing — Find trending topics in your niche for blog inspiration
• Competitive Intelligence — Monitor mentions of competitors or products
• Developer Relations — Track Show HN launches and community feedback
• Research — Analyze discussion patterns and sentiment in tech communities
• News Aggregation — Build curated feeds of top stories by topic

✅ Capabilities & Limits

Stated up front, so you do not pay a run to find out.

Built here: in search mode maxItems applies twice - once per keyword while fetching, then again as a global cap on the pushed dataset. Every query pulls up to maxItems stories, they are merged in query order, deduplicated, and truncated to maxItems total. Five keywords at maxItems: 50 still yield 50 rows, and the last keywords can be cut entirely. One keyword per run is the only way to get 50 each.

Input / feature	Supported	Notes
Mode	✅	mode — required; search, front page or trending
Keyword search	✅	searchQueries — search mode only
Sort and time range	✅	sortBy, timeRange
Top comments	✅	includeComments with maxCommentsPerStory — slower, more data
Full comment threads	❌	top-level comments only, not the whole nested tree
User profiles / karma history	❌	stories and comments, not accounts
Dead or flagged posts	❌	what the public HN interface shows
Result volume	⚠️	maxItems

🚦 What a run with zero stories does

The actor tells you why it found nothing, and the reason comes from what the HN APIs actually returned — the HTTP status code and the record container inside the response body. No guessing from page text.

What the origin did	Run status	Dataset	Charged
Answered HTTP 200 and returned an empty result set — no HN story matches your keyword	SUCCEEDED	1 diagnostic row, _dataQuality: "diagnostic"	No PPE
Blocked every request (HTTP 403, HTTP 429, timeout) so no record ever arrived	SUCCEEDED	1 diagnostic row with _blockReason	No PPE
Returned stories but the actor parsed none of them	FAILED, exit code 1	empty	No PPE
Answered normally with an empty set while timeRange was narrowing it	SUCCEEDED, diagnostic row names timeRange as the likely cause	1 diagnostic row	No PPE
Never answered, or changed the response shape	FAILED, exit code 1	empty	No PPE

Parser drift fails loudly on purpose. A run that returns nothing because the HN API renamed a field would otherwise look identical to "your keyword has no matches", and you would pay a run start to find out months later.

The diagnostic row from the first line looks like this:

{
  "setup_status": "DIAGNOSTIC_GUIDE",
  "_dataQuality": "diagnostic",
  "_blockReason": "origin-confirmed-empty (HTTP 200, empty result set)",
  "message": "[DIAGNÓSTICO] Zero stories (mode=search) — Hacker News answered and has no matching story.",
  "potential_causes": [
    "The HN API answered normally and returned an empty result set for this query",
    "No Hacker News story matches [\"quantum yodelling compiler\"]"
  ],
  "remediations": [
    "Use a broader or differently spelled keyword (HN search matches title text, not synonyms)",
    "Try mode=\"front_page\" or mode=\"top_stories\" to confirm the actor reaches HN normally"
  ],
  "mode": "search",
  "searchQueries": ["quantum yodelling compiler"],
  "timeRange": "all",
  "sortBy": "relevance",
  "maxItems": 50,
  "recordsReturnedByOrigin": 0,
  "rawStoriesCollected": 0,
  "uniqueAfterDedup": 0,
  "dataSource": "hn-algolia + hn-firebase",
  "scrapedAt": "2026-08-27T14:15:22.108Z"
}

Filter your dataset on _dataQuality != "diagnostic" and diagnostic rows never reach your pipeline.

❓ FAQ

Q: Does this actor need an API key? A: No. It uses the public Algolia HN Search API and Firebase API, both freely accessible without authentication.

Q: What is the difference between "front_page" and "top_stories" modes? A: "front_page" returns the stories currently displayed on the Hacker News homepage (around 30 items). "top_stories" uses Firebase's canonical top stories list, which can return up to 500 of the highest-ranked active stories.

Q: Will enabling comments slow down the scraper? A: Yes. Each story with comments requires an additional API call. For large runs (100+ stories), expect significantly longer execution times. Use maxCommentsPerStory to control the volume.

Q: Can I search for multiple keywords in one run? A: Yes. The searchQueries parameter accepts up to 10 keywords. Results from all queries are deduplicated by story ID.

Q: Why did the actor return fewer items than maxItems? A: Fewer stories matched your criteria. The actor deduplicates by story ID and drops titles shorter than 3 characters.

Q: What happens on a run that finds zero stories? A: The ending depends on what Hacker News answered, and the run never charges PPE. An empty result set confirmed by the API — including one narrowed by your timeRange, which is sent to the HN API as a created_at filter — ends SUCCEEDED with one diagnostic row; so does a block (403/429/timeout). The failures are the cases where nothing confirms the emptiness: the API returned stories and the parser extracted none, the response had an unexpected shape, or the API never answered. Those exit with code 1. See the table above.

💰 Pricing

This actor uses Pay Per Event (PPE) pricing:

Metric	Cost
Per story scraped	$0.03

🔗 Related Actors

• Stack Overflow Intelligence — SO questions & answers
• GitHub Trending Scraper — Trending repositories
• Craigslist Scraper — Classifieds & listings
• YouTube Fast Scraper — YouTube video metrics

📝 Changelog

v1.0 (Current)

• ✅ 5 scraping modes (search, front_page, top_stories, show_hn, ask_hn)
• ✅ Keyword search with relevance and date sorting
• ✅ Time range filtering (24h, week, month, year, all)
• ✅ Optional top comment extraction per story
• ✅ Dual API support (Algolia HN Search + Firebase)
• ✅ Deduplication by story ID
• ✅ Anti-placeholder guardrails
• ✅ Retry logic with rate limit handling
• ✅ Zero-story runs classified from observed HTTP status and response containers: confirmed-empty and confirmed-block end SUCCEEDED with a diagnostic row; parser drift, a silent API and user-emptied filters fail with exit code 1

---

📰 Hacker News Intelligence — Stories, Comentários & Tendências

🇺🇸 English | 🇧🇷 Português

Extraia stories e comentários do Hacker News usando a API pública Algolia HN e Firebase. Busque por palavra-chave, navegue pela front page, obtenha top stories, ou explore Show HN e Ask HN. Sem necessidade de API key.

✨ Funcionalidades

• 🔍 5 modos de scraping — Search, Front Page, Top Stories, Show HN, Ask HN
• 🔑 Busca por palavra-chave — Encontre stories por tópico com ordenação por relevância ou data
• 🕐 Filtros de período — Últimas 24h, semana, mês, ano ou todos os tempos
• 💬 Extração de comentários — Busque top comentários por story (opcional)
• 📊 Métricas completas — Pontos, contagem de comentários, autor, timestamps
• 🔗 Links diretos — URL original + URL da discussão no HN
• 🛡️ Guardrails anti-placeholder — Todo resultado validado como dado real
• ⚡ Baseado em API — Rápido e confiável via Algolia HN Search + Firebase

📥 Entrada

Parâmetro	Tipo	Obrigatório	Padrão	Descrição
mode	string	✅	"search"	Modo: search, front_page, top_stories, show_hn, ask_hn
searchQueries	string[]	❌	—	Palavras-chave (obrigatório no modo "search", até 10)
sortBy	string	❌	"relevance"	Ordenar: relevance, date
timeRange	string	❌	"all"	Período: all, 24h, week, month, year
maxItems	inteiro	❌	50	Máx stories para extrair (1-500)
includeComments	boolean	❌	false	Buscar top comentários por story
maxCommentsPerStory	inteiro	❌	5	Máx comentários por story (1-20)

Exemplo de Entrada

{
  "mode": "search",
  "searchQueries": [
    "AI startup",
    "machine learning",
    "open source LLM"
  ],
  "sortBy": "date",
  "timeRange": "week",
  "maxItems": 100,
  "includeComments": true,
  "maxCommentsPerStory": 5
}

📤 Saída

Cada story inclui estes campos:

Campo	Tipo	Descrição
storyId	string	ID da story no Hacker News
title	string	Título da story
url	string	URL do link original
author	string	Username do autor
points	número	Contagem de upvotes
numComments	número	Total de comentários
createdAt	string	Timestamp da submissão (ISO 8601)
storyText	string	Texto da story (para Ask HN / Show HN)
tags	string[]	Tags do HN (story, show_hn, ask_hn, etc.)
hnUrl	string	URL da discussão no Hacker News
platform	string	Sempre "hackernews"
scrapedAt	string	Timestamp ISO 8601
topComments	objeto[]	Top comentários (quando includeComments habilitado)

Campos do Comentário

Campo	Tipo	Descrição
commentId	string	ID do comentário
author	string	Username do comentarista
text	string	Texto do comentário (HTML removido)
createdAt	string	Timestamp do comentário
points	número	Pontos do comentário

Exemplo de Saída

{
  "storyId": "39847261",
  "title": "Show HN: We built an open-source alternative to Notion with local-first sync",
  "url": "https://github.com/AppFlowy-IO/AppFlowy",
  "author": "annieflowy",
  "points": 847,
  "numComments": 213,
  "createdAt": "2026-03-04T16:42:31.000Z",
  "storyText": "",
  "tags": ["story", "show_hn", "front_page"],
  "hnUrl": "https://news.ycombinator.com/item?id=39847261",
  "platform": "hackernews",
  "scrapedAt": "2026-03-06T14:15:22.108Z",
  "topComments": [
    {
      "commentId": "39848102",
      "author": "tptacek",
      "text": "This is impressive work. The offline-first approach with CRDT sync is the right way to build collaborative tools.",
      "createdAt": "2026-03-04T17:08:14.000Z",
      "points": 142
    }
  ]
}

📋 Casos de Uso

• Detecção de Tendências Tech — Acompanhe quais tecnologias e startups estão ganhando tração
• Marketing de Conteúdo — Encontre tópicos em alta no seu nicho para inspiração de blog
• Inteligência Competitiva — Monitore menções de concorrentes ou produtos
• Developer Relations — Acompanhe lançamentos Show HN e feedback da comunidade
• Pesquisa — Analise padrões de discussão e sentimento em comunidades tech
• Agregação de Notícias — Monte feeds curados de top stories por tópico

🚦 O que acontece quando a execução não encontra nenhuma story

O actor informa por que não achou nada, e o motivo vem do que as APIs do HN de fato responderam — o código HTTP e o contêiner de registros dentro do corpo da resposta. Nada é deduzido do texto da página.

O que a origem fez	Status da execução	Dataset	Cobrança
Respondeu HTTP 200 com conjunto de resultados vazio — nenhuma story do HN casa com sua palavra-chave	SUCCEEDED	1 linha de diagnóstico, _dataQuality: "diagnostic"	Sem PPE
Bloqueou todas as requisições (HTTP 403, HTTP 429, timeout) e nenhum registro chegou	SUCCEEDED	1 linha de diagnóstico com _blockReason	Sem PPE
Devolveu stories e o actor não extraiu nenhuma	FAILED, exit code 1	vazio	Sem PPE
Respondeu normalmente com conjunto vazio enquanto o timeRange estreitava a busca	SUCCEEDED, linha de diagnóstico aponta timeRange como causa provável	1 linha de diagnóstico	Sem PPE
Nunca respondeu, ou mudou o formato da resposta	FAILED, exit code 1	vazio	Sem PPE

Drift de parser falha alto de propósito. Uma execução que volta vazia porque a API do HN renomeou um campo seria idêntica a "sua palavra-chave não tem resultado", e você pagaria o start da execução para descobrir isso meses depois.

A linha de diagnóstico da primeira situação sai assim:

{
  "setup_status": "DIAGNOSTIC_GUIDE",
  "_dataQuality": "diagnostic",
  "_blockReason": "origin-confirmed-empty (HTTP 200, empty result set)",
  "message": "[DIAGNÓSTICO] Zero stories (mode=search) — Hacker News answered and has no matching story.",
  "potential_causes": [
    "The HN API answered normally and returned an empty result set for this query",
    "No Hacker News story matches [\"quantum yodelling compiler\"]"
  ],
  "remediations": [
    "Use a broader or differently spelled keyword (HN search matches title text, not synonyms)",
    "Try mode=\"front_page\" or mode=\"top_stories\" to confirm the actor reaches HN normally"
  ],
  "mode": "search",
  "searchQueries": ["quantum yodelling compiler"],
  "timeRange": "all",
  "sortBy": "relevance",
  "maxItems": 50,
  "recordsReturnedByOrigin": 0,
  "rawStoriesCollected": 0,
  "uniqueAfterDedup": 0,
  "dataSource": "hn-algolia + hn-firebase",
  "scrapedAt": "2026-08-27T14:15:22.108Z"
}

Filtre o dataset por _dataQuality != "diagnostic" e nenhuma linha de diagnóstico entra no seu pipeline.

❓ Perguntas Frequentes

P: Este actor precisa de uma API key? R: Não. Ele usa a API pública Algolia HN Search e a API Firebase, ambas acessíveis livremente sem autenticação.

P: Qual é a diferença entre os modos "front_page" e "top_stories"? R: "front_page" retorna as stories atualmente exibidas na página inicial do Hacker News (cerca de 30 itens). "top_stories" usa a lista canônica de top stories do Firebase, que pode retornar até 500 das stories ativas com melhor classificação.

P: Habilitar comentários vai deixar o scraper mais lento? R: Sim. Cada story com comentários requer uma chamada API adicional. Para execuções grandes (100+ stories), espere tempos de execução significativamente maiores. Use maxCommentsPerStory para controlar o volume.

P: Posso buscar várias palavras-chave em uma execução? R: Sim. O parâmetro searchQueries aceita até 10 palavras-chave. Resultados de todas as queries são deduplicados por story ID.

P: Por que o actor retornou menos itens que maxItems? R: Menos stories corresponderam aos seus critérios. O actor deduplica por story ID e descarta títulos com menos de 3 caracteres.

P: O que acontece numa execução que não encontra nenhuma story? R: O desfecho depende do que o Hacker News respondeu, e a execução nunca cobra PPE. Conjunto vazio confirmado pela API — inclusive quando o seu timeRange estreitou a busca, já que ele é enviado à API do HN como filtro created_at — termina SUCCEEDED com uma linha de diagnóstico; bloqueio (403/429/timeout) também. Falham as situações em que nada confirma o vazio: a API devolveu stories e o parser não extraiu nenhuma, a resposta veio com formato inesperado, ou a API nunca respondeu. Essas saem com exit code 1. Veja a tabela acima.

💰 Preços

Este actor usa precificação Pay Per Event (PPE):

Métrica	Custo
Por story extraída	$0.03

🔗 Actors Relacionados

• Stack Overflow Intelligence — Perguntas e respostas do SO
• GitHub Trending Scraper — Repositórios em alta
• Craigslist Scraper — Classificados & anúncios
• YouTube Fast Scraper — Métricas do YouTube

📝 Changelog

v1.0 (Atual)

• ✅ 5 modos de scraping (search, front_page, top_stories, show_hn, ask_hn)
• ✅ Busca por palavra-chave com ordenação por relevância e data
• ✅ Filtragem por período (24h, semana, mês, ano, todos)
• ✅ Extração opcional de top comentários por story
• ✅ Suporte dual API (Algolia HN Search + Firebase)
• ✅ Deduplicação por story ID
• ✅ Guardrails anti-placeholder
• ✅ Lógica de retry com tratamento de rate limit
• ✅ Execução com zero stories classificada pelo status HTTP e pelo contêiner de registros observados: vazio-confirmado e bloqueio-confirmado terminam SUCCEEDED com linha de diagnóstico; drift de parser, API muda e filtro do usuário falham com exit code 1