# Hacker News Scraper (`nogards95/hacker-news-scraper`) Actor
Scrape Hacker News stories, comments, jobs, Ask HN, and Show HN using Algolia Search API and HN Firebase API. Supports full-text search, date/points filters, and live feeds.
- **URL**: https://apify.com/nogards95/hacker-news-scraper.md
- **Developed by:** [Igli](https://apify.com/nogards95) (community)
- **Categories:** News, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 50.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet
## Pricing
from $1.50 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Hacker News Scraper
Extract stories, comments, jobs, Ask HN, and Show HN from Hacker News — fast, cheap, no browser required.
### Features
- **Full-text search** via [Algolia HN Search API](https://hn.algolia.com/api) — filter by keyword, date, points, comments, author
- **Live feeds** via [HN Firebase API](https://hacker-news.firebaseio.com) — top, new, best, ask, show, job
- **Comment fetching** — get all comments for individual stories or include comments in a search run
- **128 MB / ~2–30s** — no browser, no proxies, no auth needed
- **Clean output** — HTML stripped, `hnUrl` always present, domain extracted
---
### Input
| Field | Type | Default | Description |
|---|---|---|---|
| `mode` | `search` \| `feed` | `search` | Search via Algolia or fetch a live Firebase feed |
| `query` | string | — | Keyword to search (required for `mode=search`) |
| `type` | `story` \| `comment` \| `ask_hn` \| `show_hn` \| `job` \| `all` | `story` | Content type to retrieve |
| `sortBy` | `relevance` \| `date` | `relevance` | Sort order for search results |
| `feedType` | `top` \| `new` \| `best` \| `ask` \| `show` \| `job` | `top` | Feed to fetch (only for `mode=feed`) |
| `maxItems` | integer | `100` | Maximum results (1–1000) |
| `minPoints` | integer | — | Minimum story points |
| `minComments` | integer | — | Minimum comment count |
| `dateFrom` | string (ISO) | — | Filter items after this date (e.g. `2024-01-01`) |
| `dateTo` | string (ISO) | — | Filter items before this date |
| `author` | string | — | Filter by HN username |
| `storyId` | string | — | Fetch all comments for a specific story ID |
| `includeComments` | boolean | `false` | Also fetch comments for each story found |
#### Example inputs
**Search AI stories with 100+ points:**
```json
{
"mode": "search",
"query": "artificial intelligence",
"type": "story",
"sortBy": "date",
"minPoints": 100,
"maxItems": 50
}
````
**Fetch current top 30 stories:**
```json
{
"mode": "feed",
"feedType": "top",
"maxItems": 30
}
```
**Get all Ask HN posts:**
```json
{
"mode": "search",
"query": "who is hiring",
"type": "ask_hn",
"maxItems": 20
}
```
**Get all comments for a specific story:**
```json
{
"mode": "search",
"query": "",
"type": "comment",
"storyId": "39930210",
"maxItems": 500
}
```
***
### Output
#### Story
```json
{
"id": "39930210",
"type": "story",
"title": "Show HN: I built a HN scraper",
"url": "https://github.com/example/hn-scraper",
"hnUrl": "https://news.ycombinator.com/item?id=39930210",
"domain": "github.com",
"author": "pg",
"points": 342,
"commentCount": 87,
"text": null,
"tags": ["story", "author_pg"],
"publishedAt": "2024-03-15T10:30:00.000Z",
"scrapedAt": "2026-05-21T12:00:00.000Z"
}
```
#### Comment
```json
{
"id": "39930300",
"type": "comment",
"text": "Great work! How do you handle rate limits?",
"textHtml": "Great work! How do you handle rate limits?
",
"author": "dang",
"points": null,
"storyId": "39930210",
"storyTitle": "Show HN: I built a HN scraper",
"parentId": "39930210",
"hnUrl": "https://news.ycombinator.com/item?id=39930300",
"publishedAt": "2024-03-15T11:00:00.000Z",
"scrapedAt": "2026-05-21T12:00:00.000Z"
}
```
***
### APIs used
| API | Use case | Auth |
|---|---|---|
| [Algolia HN Search](https://hn.algolia.com/api) | Search, filtering, comments | None |
| [HN Firebase API](https://hacker-news.firebaseio.com) | Live feeds (top/new/best/ask/show/job) | None |
***
### Performance
- **128 MB memory** — smallest Apify tier
- **~2–5s** for a simple keyword search
- **~10–30s** for a 100-item feed run (Firebase requires one request per item)
- No proxies, no headless browsers, no authentication needed
***
### Keywords
Hacker News API, HN scraper, tech trends, Ask HN, Show HN, jobs board, startup news, tech community, Hacker News stories, Hacker News comments
# Actor input Schema
## `mode` (type: `string`):
Search mode: 'search' uses Algolia full-text search, 'feed' fetches live HN feeds via Firebase.
## `query` (type: `string`):
Keyword to search for. Required when mode is 'search'.
## `type` (type: `string`):
Type of content to retrieve.
## `sortBy` (type: `string`):
Sort search results by relevance or date (most recent first). Only applies to mode=search.
## `feedType` (type: `string`):
Which HN live feed to fetch. Only used when mode is 'feed'.
## `maxItems` (type: `integer`):
Maximum number of results to return.
## `minPoints` (type: `integer`):
Filter stories with at least this many points. Only applies to mode=search.
## `minComments` (type: `integer`):
Filter items with at least this many comments. Only applies to mode=search.
## `dateFrom` (type: `string`):
Filter items published after this date. ISO 8601 format (e.g. 2024-01-01). Only applies to mode=search.
## `dateTo` (type: `string`):
Filter items published before this date. ISO 8601 format (e.g. 2024-12-31). Only applies to mode=search.
## `author` (type: `string`):
Filter by HN username. Only applies to mode=search.
## `storyId` (type: `string`):
Fetch all comments for a specific story by its HN item ID. When provided with type=comment, overrides the query.
## `includeComments` (type: `boolean`):
Fetch comments for each story found. Increases run time and item count. Only applies to mode=search with type=story.
## Actor input object example
```json
{
"mode": "search",
"query": "javascript",
"type": "story",
"sortBy": "relevance",
"feedType": "top",
"maxItems": 100,
"includeComments": false
}
```
# Actor output Schema
## `results` (type: `string`):
No description
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {};
// Run the Actor and wait for it to finish
const run = await client.actor("nogards95/hacker-news-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {}
# Run the Actor and wait for it to finish
run = client.actor("nogards95/hacker-news-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{}' |
apify call nogards95/hacker-news-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=nogards95/hacker-news-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Hacker News Scraper",
"description": "Scrape Hacker News stories, comments, jobs, Ask HN, and Show HN using Algolia Search API and HN Firebase API. Supports full-text search, date/points filters, and live feeds.",
"version": "1.0",
"x-build-id": "x3MYBz3YLWn4i6K6I"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/nogards95~hacker-news-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-nogards95-hacker-news-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/nogards95~hacker-news-scraper/runs": {
"post": {
"operationId": "runs-sync-nogards95-hacker-news-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/nogards95~hacker-news-scraper/run-sync": {
"post": {
"operationId": "run-sync-nogards95-hacker-news-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"mode"
],
"properties": {
"mode": {
"title": "Mode",
"enum": [
"search",
"feed"
],
"type": "string",
"description": "Search mode: 'search' uses Algolia full-text search, 'feed' fetches live HN feeds via Firebase.",
"default": "search"
},
"query": {
"title": "Search Query",
"type": "string",
"description": "Keyword to search for. Required when mode is 'search'.",
"default": "javascript"
},
"type": {
"title": "Content Type",
"enum": [
"story",
"comment",
"ask_hn",
"show_hn",
"job",
"all"
],
"type": "string",
"description": "Type of content to retrieve.",
"default": "story"
},
"sortBy": {
"title": "Sort By",
"enum": [
"relevance",
"date"
],
"type": "string",
"description": "Sort search results by relevance or date (most recent first). Only applies to mode=search.",
"default": "relevance"
},
"feedType": {
"title": "Feed Type",
"enum": [
"top",
"new",
"best",
"ask",
"show",
"job"
],
"type": "string",
"description": "Which HN live feed to fetch. Only used when mode is 'feed'.",
"default": "top"
},
"maxItems": {
"title": "Max Items",
"minimum": 1,
"maximum": 1000,
"type": "integer",
"description": "Maximum number of results to return.",
"default": 100
},
"minPoints": {
"title": "Minimum Points",
"minimum": 0,
"type": "integer",
"description": "Filter stories with at least this many points. Only applies to mode=search."
},
"minComments": {
"title": "Minimum Comments",
"minimum": 0,
"type": "integer",
"description": "Filter items with at least this many comments. Only applies to mode=search."
},
"dateFrom": {
"title": "Date From",
"pattern": "^\\d{4}-\\d{2}-\\d{2}(T.*)?$",
"type": "string",
"description": "Filter items published after this date. ISO 8601 format (e.g. 2024-01-01). Only applies to mode=search."
},
"dateTo": {
"title": "Date To",
"pattern": "^\\d{4}-\\d{2}-\\d{2}(T.*)?$",
"type": "string",
"description": "Filter items published before this date. ISO 8601 format (e.g. 2024-12-31). Only applies to mode=search."
},
"author": {
"title": "Author",
"type": "string",
"description": "Filter by HN username. Only applies to mode=search."
},
"storyId": {
"title": "Story ID",
"type": "string",
"description": "Fetch all comments for a specific story by its HN item ID. When provided with type=comment, overrides the query."
},
"includeComments": {
"title": "Include Comments",
"type": "boolean",
"description": "Fetch comments for each story found. Increases run time and item count. Only applies to mode=search with type=story.",
"default": false
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```