# Hacker News Scraper (`superslowsloth/hacker-news-scraper`) Actor

Search Hacker News stories and comments with filters for tag, minimum score and date range. Returns title, URL, author, points, comment count, timestamps and full comment text. Sort by relevance or date, filter to front page, Show HN or Ask HN, and page up to Algolia's 1,000-hit window per query.

- **URL**: https://apify.com/superslowsloth/hacker-news-scraper.md
- **Developed by:** [Superslow Sloth](https://apify.com/superslowsloth) (community)
- **Categories:** News, Developer tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 story or comments

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Hacker News Scraper

Searches Hacker News and returns stories and comments as structured rows. It
covers keyword search, browsing by item type (front page, Show HN, Ask HN, polls,
jobs), a minimum score, and an arbitrary date range. No login and no proxy are
required.

The data comes from the official Algolia-powered Hacker News Search API, the same
index behind the search box on `news.ycombinator.com`.

### Input

| Field | Type | Notes |
|---|---|---|
| `queries` | array | Search terms. Each is searched separately and results are merged and de-duplicated by item ID. May be left empty when a tag alone is the search, for example `front_page`. |
| `tags` | array | `story`, `comment`, `front_page`, `poll`, `pollopt`, `job`, `show_hn`, `ask_hn`. Several tags are combined with AND. Every one of these was verified against the live API; an unrecognised tag is rejected up front, because the API answers it with an empty result set that looks exactly like a broken scraper. |
| `sortBy` | string | `relevance` (default) or `date`. These are two different indices, not a sort flag: `relevance` uses `/search`, `date` uses `/search_by_date`. |
| `minPoints` | integer | Minimum score. Applied as `points>=N` in the API's `numericFilters`. |
| `startDate` / `endDate` | string | `YYYY-MM-DD`, a full ISO 8601 timestamp, or a Unix timestamp. Converted into `created_at_i` bounds. A bare end date includes the whole of that day. |
| `maxItems` | integer | Total item budget across all queries. Default 100. |
| `proxyConfiguration` | object | Optional, and off by default. The API is open. |

### Output

Two shapes share the dataset, distinguished by `type`.

A story (`type: "story"` — also covers Ask HN, Show HN, polls and job posts):

```json
{
  "type": "story",
  "object_id": "45751400",
  "title": "Uv is the best thing to happen to the Python ecosystem in a decade",
  "url": "https://emily.space/posts/251023-uv",
  "author": "todsacerdoti",
  "points": 1049,
  "num_comments": 605,
  "story_text": null,
  "created_at": "2025-10-29T18:57:29Z",
  "created_at_i": 1761764249,
  "hn_url": "https://news.ycombinator.com/item?id=45751400",
  "tags": ["story", "author_todsacerdoti", "story_45751400"]
}
```

A comment (`type: "comment"`):

```json
{
  "type": "comment",
  "object_id": "9999987",
  "author": "pbaehr",
  "comment_text": "ABI Research | Oyster Bay, New York<p>We are looking for ...",
  "parent_id": 9996333,
  "story_id": 9996333,
  "story_title": "Ask HN: Who is hiring? (August 2015)",
  "points": null,
  "created_at": "2015-08-03T21:30:28Z",
  "created_at_i": 1438637428,
  "hn_url": "https://news.ycombinator.com/item?id=9999987",
  "tags": ["comment", "author_pbaehr", "story_9996333"]
}
```

### Notes on the data

- **Comment text contains HTML markup.** Hacker News stores comment and story
  bodies as HTML fragments: `<p>` between paragraphs, `<a href="...">` around
  links, `<i>` for italics, `<pre><code>` for code blocks. HTML entities are
  unescaped for you (`&#x2F;` becomes `/`, `&quot;` becomes `"`), but the tags
  themselves are left in place, because stripping them would destroy the
  paragraph breaks. Strip them yourself if you want plain text.
- **A missing field is `null`, never `0` or `""`.** Hacker News does not score
  individual comments, so `points` on a comment is always `null` — a zero there
  would read as a measurement that was never made. Likewise a text post has no
  `url`, and a link post has no `story_text`.
- **A single query reaches at most 1000 items.** The search API pages through
  the first 1000 hits of any one query and then returns nothing, however the
  pages are sliced; `hitsPerPage` above 1000 is silently clamped. `maxItems`
  above 1000 therefore needs several narrower queries, or a date range walked
  in slices, to go further back. Both limits were measured on 2026-08-24.
- **Items are de-duplicated by `objectID`** before they are delivered, so the
  same story matching two of your queries is emitted and billed once.
- No rate limiting was observed in testing (40 requests back to back all
  returned HTTP 200), but the actor still treats HTTP 429, 403 and 5xx as
  transient and retries them with backoff. A malformed filter, which the API
  answers with HTTP 400, is treated as permanent and is not retried.

### Billing

One event per delivered item, charged after the item is written to the dataset:
`post-scraped` for stories and `comment-scraped` for comments. The event is
chosen from what each item actually is, not from what was requested, so a mixed
search bills each row correctly. A small `actor-start` event covers the fixed
cost of a run that legitimately finds nothing.

# Actor input Schema

## `queries` (type: `array`):

Keywords to search Hacker News for. Each query is searched separately and the results are merged and de-duplicated. Leave empty to browse by tag alone, for example the current front page.

## `tags` (type: `array`):

Restrict results to these kinds of item. Several tags are combined with AND, so 'story' plus 'show\_hn' means Show HN submissions only. Leave empty to return both stories and comments.

## `sortBy` (type: `string`):

Relevance ranks by how well an item matches the query, which is what the Hacker News search box does by default. Date returns the newest items first and ignores relevance entirely.

## `minPoints` (type: `integer`):

Only return items with at least this many points. Hacker News does not score individual comments, so setting this while searching comments returns nothing.

## `startDate` (type: `string`):

Only return items created on or after this date. Accepts YYYY-MM-DD, a full ISO 8601 timestamp, or a Unix timestamp in seconds.

## `endDate` (type: `string`):

Only return items created on or before this date. A bare YYYY-MM-DD date includes the whole of that day. Accepts the same formats as the start date.

## `maxItems` (type: `integer`):

Stop after this many items in total, across all queries. The search API only pages through the first 1000 hits of any single query, so use several narrower queries or a date range to reach further back.

## `proxyConfiguration` (type: `object`):

Proxy used for the search requests. The Hacker News search API needs no proxy and no login, so this is only worth enabling if your runs are being rate limited.

## Actor input object example

```json
{
  "queries": [
    "python"
  ],
  "tags": [
    "story"
  ],
  "sortBy": "relevance",
  "maxItems": 100,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `items` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "python"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("superslowsloth/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["python"] }

# Run the Actor and wait for it to finish
run = client.actor("superslowsloth/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "python"
  ]
}' |
apify call superslowsloth/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,superslowsloth/hacker-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Nj40ZDUD2WuDSFdZ6/builds/wJVGHta1m14fiemTB/openapi.json
