# Hacker News Scraper \[$0.002💰] | Trends | Show HN | Data (`ahmed_jasarevic/news-ycombinator-scraper`) Actor

Extract Hacker News stories, comments, and user profiles for trend analysis and tech intelligence

- **URL**: https://apify.com/ahmed\_jasarevic/news-ycombinator-scraper.md
- **Developed by:** [Ahmed Jasarevic](https://apify.com/ahmed_jasarevic) (community)
- **Categories:** News, Developer tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.90 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Hacker News Scraper

Scrape Hacker News stories, jobs, discussions, and user profiles from all major sections. Extract front page posts, Show HN launches, Ask HN discussions, and job listings with comments, scores, and full metadata.

### Features

- Scrape **front page**, **Ask HN**, **Show HN**, and **Hacker News Jobs** sections
- Retrieve user profiles, karma scores, and submission history
- Pull top-level comments for any story
- Flat or nested output formats for flexible data processing
- Configurable item limits per section and total
- Timestamp conversion to human-readable or ISO format
- Filter by specific item IDs or usernames

### Why Use This Actor?

Hacker News is one of the most influential tech communities, shaping trends across startups, open source, and developer tools. This scraper gives you structured access to that data for competitive intelligence, trend analysis, lead generation, and market research.

### Use Cases

- **Trend Monitoring** — Track what's trending on the front page daily
- **Startup Intelligence** — Monitor Show HN launches and competitor activity
- **Job Market Analysis** — Scrape startup job postings and hiring trends
- **Developer Community Research** — Analyze user profiles and submission patterns
- **Content Curation** — Aggregate top tech stories for newsletters or dashboards
- **AI/ML News Tracking** — Filter AI and machine learning discussions on HN

### Pricing

This is a pay-per-result Actor at **$0.0019 per result**. You only pay for the items actually scraped.

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `sections` | `string[]` | `["front"]` | Sections to scrape: `front`, `ask`, `show`, `jobs` |
| `maxItemsPerSection` | `integer` | `50` | Max items to fetch per section |
| `maxItemsTotal` | `integer` | — | Global cap across all sections |
| `itemIds` | `string[]` | — | Scrape specific HN item IDs directly |
| `usernames` | `string[]` | — | Fetch specific HN user profiles |
| `includeCommentCount` | `boolean` | `false` | Include total comment count per story |
| `includeTopLevelComments` | `boolean` | `false` | Fetch top-level comments for stories |
| `convertTimestamps` | `boolean` | `false` | Convert Unix timestamps to ISO 8601 |
| `outputFormat` | `string` | `"flat"` | `flat` (one row per item) or `nested` (with replies) |

### Output

The Actor produces a dataset with items in the selected format. Each story item includes:

- `id` — HN item ID
- `title` — Story title
- `url` — Link URL (if external)
- `score` — Points/votes
- `by` — Author username
- `time` — Unix timestamp (converted if enabled)
- `descendants` — Total comment count (if enabled)
- `kids` — Array of child item IDs
- `text` — Text content (for Ask HN, comments)
- `section` — Source section (`front`, `ask`, `show`, `jobs`)

### Dataset Views

This Actor exposes two dataset views:

- **stories** — Flat list of all scraped stories with metadata
- **users** — User profiles and karma data

### Tips

- Use `maxItemsPerSection: 100` to get a broader view of current trends
- Set `convertTimestamps: true` for easier date filtering in downstream tools
- Combine `includeTopLevelComments: true` with `outputFormat: "nested"` for full discussion threads
- Use `itemIds` to re-scrape specific stories for updated comment counts

### Related Actors

Verified related actors on Apify that pair well with this one. All links point to real, publicly available actors.

- [Reddit Scraper](https://apify.com/trudax/reddit-scraper-lite)
- [YouTube Scraper](https://apify.com/streamers/youtube-scraper)
- [Twitter Scraper](https://apify.com/apidojo/tweet-scraper)

# Actor input Schema

## `sections` (type: `array`):

Which HN sections to scrape. Leave empty for all sections (front, ask, show, jobs).

## `maxItemsPerSection` (type: `integer`):

Maximum number of stories to fetch per section (max 500).

## `maxItemsTotal` (type: `integer`):

Hard cap on total items scraped across all sections (0 = unlimited by this cap, limited by maxItemsPerSection only).

## `itemIds` (type: `array`):

Fetch specific HN item IDs directly (overrides sections). Example: 41804059, 41804060

## `usernames` (type: `array`):

Fetch HN user profiles by username (overrides sections and itemIds).

## `includeCommentCount` (type: `boolean`):

Include the number of comments (descendants) for each story.

## `includeTopLevelComments` (type: `boolean`):

Include the number of top-level comments (kids array length) for each story.

## `convertTimestamps` (type: `boolean`):

Convert Unix timestamps to ISO 8601 date strings.

## `outputFormat` (type: `string`):

How to structure the output data.

## Actor input object example

```json
{
  "sections": [
    "front",
    "ask",
    "show",
    "jobs"
  ],
  "maxItemsPerSection": 50,
  "maxItemsTotal": 0,
  "itemIds": [],
  "usernames": [],
  "includeCommentCount": true,
  "includeTopLevelComments": false,
  "convertTimestamps": true,
  "outputFormat": "flat"
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("ahmed_jasarevic/news-ycombinator-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("ahmed_jasarevic/news-ycombinator-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call ahmed_jasarevic/news-ycombinator-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ahmed_jasarevic/news-ycombinator-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/zpOyZObnuDBLU8Zec/builds/R9W3dOOQZcAw9mEJY/openapi.json