Hacker News Story Scraper

Collect stories from Hacker News straight from its free, official Firebase API. There is nothing to configure — no API key, no login, and no proxy — the feed is public and rock-solid.

Point it at a feed (or a handful of item IDs), pick how many stories you want, and get back one tidy JSON row per story.

Features

• Reads any standard Hacker News feed: Top, New, Best, Ask HN, Show HN or Jobs.
• Optionally targets a set of specific item IDs instead of a feed.
• Fetches item details concurrently in small batches to stay fast and polite.
• Emits one flat, ready-to-use JSON record per story.
• Skips gracefully over deleted or missing items — they are recorded with an error field so a single bad item never stops the run.

Input

Field	Type	Default	Description
list	select	topstories	Which feed to scrape: topstories, newstories, beststories, askstories, showstories, jobstories.
maxItems	integer	50	Maximum number of stories to fetch.
itemIds	array	—	Optional. Fetch these specific item IDs instead of a feed. When set, list is ignored.

Example input

{
  "list": "topstories",
  "maxItems": 10
}

Or fetch specific items:

{
  "itemIds": ["8863", "121003"]
}

Output

Each dataset record looks like this:

{
  "id": 8863,
  "type": "story",
  "title": "My YC app: Dropbox - Throw away your USB drive",
  "url": "http://www.getdropbox.com/u/2/screencast.html",
  "hnUrl": "https://news.ycombinator.com/item?id=8863",
  "by": "dhouston",
  "score": 111,
  "comments": 71,
  "time": "2007-04-04T19:16:40.000Z",
  "text": null
}

Field	Description
id	Hacker News item ID.
type	Item type (story, job, poll, etc.).
title	Story title.
url	The story's external URL (null for Ask HN / text posts).
hnUrl	Link to the discussion on Hacker News.
by	Author username.
score	Points/upvotes.
comments	Comment count (descendants).
time	Submission time as an ISO 8601 timestamp.
text	Body text (for Ask HN and text posts), as raw HTML.

If an item can't be fetched (e.g. it was deleted), the row is { "id": <id>, "error": "<message>" } instead.

Pricing

This actor uses the pay-per-event model: you are charged once per successfully scraped story (the story-scraped event). Deleted or unreachable items produce an error row and are never charged.

Data source

The actor talks to the official Hacker News API hosted on Firebase:

• Base URL: https://hacker-news.firebaseio.com/v0
• Lists: /topstories.json, /newstories.json, /beststories.json, /askstories.json, /showstories.json, /jobstories.json
• Items: /item/{id}.json

The API is free and requires no authentication.

License

MIT © Assia Fadli