# Hacker News Scraper — Stories, Ask/Show HN, Comments & Discu (`spotless_studios/hacker-news-scraper`) Actor

Scrape Hacker News stories, Ask HN questions, Show HN product launches, job postings, comments, points, and sentiment. Full-text search and real-time monitoring.

- **URL**: https://apify.com/spotless_studios/hacker-news-scraper.md
- **Developed by:** [Spotless Studios](https://apify.com/spotless_studios) (community)
- **Categories:** Social media, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 hacker news story extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Stories, Ask/Show HN & Comments Scraper

Fast, structured data extraction from Hacker News via the official Algolia search infrastructure. Monitor product launches, tech sentiment, developer discussions, and brand mentions without blocks or timeouts.

### Features

- **Comprehensive HN Data**: Extracts story title, external URL, permanent HN discussion link, author, points, comments count, item type (`story`, `show_hn`, `ask_hn`, `job`, `comment`), and full text.
- **Precision Filtering**: Filter by category (`show_hn`, `ask_hn`, `job`, `story`), minimum karma points, minimum comments, specific author, and time window (`last24h`, `pastWeek`, `pastMonth`, `allTime`).
- **Flexible Sorting**: Sort by overall community popularity or by real-time date order (`search_by_date`).
- **Scheduled Monitoring**: Enable `onlyNewItems` with a `stateKey` to automatically get pinged whenever new discussions matching your product or keywords appear on Hacker News.
- **Fast & Resilient**: Lightweight JSON API requests consuming minimal memory (128 MB), zero browser overhead.

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `searchQuery` | String | `"Show HN"` | Search query or topic |
| `storyType` | String | `"story"` | `all`, `story`, `show_hn`, `ask_hn`, `job`, `comment` |
| `sortBy` | String | `"popularity"` | `popularity` or `date` |
| `minPoints` | Integer | `0` | Minimum upvote threshold |
| `minComments` | Integer | `0` | Minimum comments threshold |
| `timeRange` | String | `"allTime"` | `last24h`, `pastWeek`, `pastMonth`, `allTime` |
| `author` | String | `""` | Filter by submitter username |
| `maxResults` | Integer | `100` | Maximum items to extract |
| `onlyNewItems` | Boolean | `false` | Monitoring mode: only emit newly posted stories |
| `outputPreset` | String | `"full"` | `full`, `lean`, or `contacts` |

### Output Example

```json
{
  "storyId": "41982736",
  "title": "Show HN: Spotless AI — Resilient web intelligence engine",
  "url": "https://spotless.dev",
  "hnUrl": "https://news.ycombinator.com/item?id=41982736",
  "author": "spotless_founder",
  "points": 342,
  "numComments": 88,
  "itemType": "show_hn",
  "storyText": "Hey HN! We built a high-speed proxy-escalating crawler...",
  "tags": ["story", "author_spotless_founder", "show_hn"],
  "createdAt": "2026-10-27T10:15:00.000Z",
  "scrapedAt": "2026-10-27T12:00:00.000Z"
}
```

***

### More from Spotless Studios

**[All actors on Apify](https://apify.com/spotless_studios)** · Support: spotless.studios.info@gmail.com

| Actor | Best for |
|-------|----------|
| [SLED Government Bids](https://apify.com/spotless_studios/sled-procurement-scraper) | Daily US gov RFP / bid alerts |
| [Webflow Experts](https://apify.com/spotless_studios/webflow-experts-scraper) | Webflow agency & freelancer leads |
| [Stripe Partners](https://apify.com/spotless_studios/stripe-partners-scraper) | Stripe implementation partner lists |
| [UK Planning Applications](https://apify.com/spotless_studios/uk-planning-applications-scraper) | UK planning leads + agent contacts |
| [Product Hunt](https://apify.com/spotless_studios/product-hunt-scraper) | Every PH launch + makers / websites |
| [Duunitori Jobs](https://apify.com/spotless_studios/duunitori-jobs-scraper) | Finland job listings |
| [Clutch Agencies](https://apify.com/spotless_studios/clutch-agencies-scraper) | Clutch agency prospecting |
| [Skool Communities](https://apify.com/spotless_studios/skool-communities-scraper) | Skool community & creator leads |

# Actor input Schema

## `searchQuery` (type: `string`):

Search keywords (e.g. 'open source llm', 'Show HN', 'Stripe', 'Postgres'). Leave empty for top stories.

## `searchQueries` (type: `array`):

Optional list of multiple search queries to run sequentially in a single job.

## `storyType` (type: `string`):

Filter by Hacker News item type.

## `sortBy` (type: `string`):

Sort by popularity (points/relevance) or newest date.

## `domain` (type: `string`):

Only keep stories linking to this domain (e.g. 'github.com', 'arxiv.org', 'substack.com').

## `minPoints` (type: `integer`):

Only return items with at least this many points (e.g. 50, 100, 500).

## `maxPoints` (type: `integer`):

Maximum points threshold.

## `minComments` (type: `integer`):

Only return items with at least this many comments.

## `timeRange` (type: `string`):

Filter items posted within a specific time window.

## `postedAfter` (type: `string`):

Custom start date (YYYY-MM-DD, ISO, or relative like '7 days' / '48 hours'). Overrides timeRange.

## `postedBefore` (type: `string`):

Custom end date (YYYY-MM-DD, ISO, or relative).

## `author` (type: `string`):

Filter posts submitted by a specific Hacker News user (e.g. 'pg', 'tptacek', 'dang').

## `excludeAuthors` (type: `array`):

Skip items submitted by these usernames.

## `itemIds` (type: `array`):

Scrape specific Hacker News item IDs or URLs directly (e.g. '39501234', 'https://news.ycombinator.com/item?id=39501234').

## `includeComments` (type: `boolean`):

Fetch the item thread to attach the top community comments (topComments) to each story.

## `maxCommentsPerStory` (type: `integer`):

Maximum number of top-level comments to include when includeComments is enabled.

## `includeKeywords` (type: `array`):

Keep only items matching these keywords or /regex/ in title, text, domain, or comments.

## `excludeKeywords` (type: `array`):

Skip items matching any of these keywords or /regex/.

## `keywordMatch` (type: `string`):

Require any or all includeKeywords to match.

## `onlyNewItems` (type: `boolean`):

Remember seen story IDs across runs and emit only newly discovered items.

## `stateKey` (type: `string`):

Separate state key per saved search or schedule.

## `maxResults` (type: `integer`):

Maximum number of Hacker News items to save.

## `outputPreset` (type: `string`):

Choose full record, lean summary, or contacts & links only.

## `outputFields` (type: `array`):

Keep only these specific fields in each saved record.

## `excludeFields` (type: `array`):

Remove specific fields from each saved record.

## `includeEmptyFields` (type: `boolean`):

When disabled, strips null and empty values from each saved record.

## `requiredFields` (type: `array`):

Only save records where every listed field has a non-empty value (e.g. url, storyText).

## `maxPages` (type: `integer`):

Maximum search pages per query (100 items per page).

## `maxConcurrency` (type: `integer`):

Parallel requests when fetching direct item IDs.

## `maxRequestRetries` (type: `integer`):

Retries per failed HTTP request.

## `requestTimeoutSecs` (type: `integer`):

Timeout per HTTP request.

## `proxyConfiguration` (type: `object`):

Optional proxy settings.

## `proxyEscalation` (type: `boolean`):

Automatically step up from direct to Apify datacenter/residential proxies if blocked.

## Actor input object example

```json
{
  "searchQuery": "AI agent",
  "storyType": "story",
  "sortBy": "popularity",
  "timeRange": "allTime",
  "includeComments": false,
  "maxCommentsPerStory": 5,
  "keywordMatch": "any",
  "onlyNewItems": false,
  "stateKey": "default",
  "maxResults": 50,
  "outputPreset": "full",
  "includeEmptyFields": true,
  "maxPages": 20,
  "maxConcurrency": 5,
  "maxRequestRetries": 3,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "proxyEscalation": true
}
```

# Actor output Schema

## `items` (type: `string`):

Scrape Hacker News stories, Ask HN questions, Show HN product launches, job postings, comments, points, and sentiment. Full-text search and real-time monitoring.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "AI agent",
    "maxResults": 50,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("spotless_studios/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQuery": "AI agent",
    "maxResults": 50,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("spotless_studios/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": "AI agent",
  "maxResults": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call spotless_studios/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,spotless_studios/hacker-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/n9dK7EYeyIEVOP7Xf/builds/D6JjMNHa8lR0zV9p5/openapi.json