# Hacker News Scraper (`exuberant_volley/hackernews-scraper`) Actor Scrape Hacker News stories — top, new or best — with title, url, score, comment count and timestamps, via the official HN Firebase API. Clean JSON, no author usernames, no personal data. - **URL**: https://apify.com/exuberant\_volley/hackernews-scraper.md - **Developed by:** [ScrapeForge](https://apify.com/exuberant_volley) (community) - **Categories:** News, Developer tools - **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hacker News Scraper — Top, New & Best Stories as Clean JSON **Launch pricing: this actor currently adds no fee — run it on your Apify free-plan credits. Try it, bookmark it, wire it into your stack.** A **Hacker News scraper** on the official Firebase API: **scrape Hacker News** Top, New or Best lists and get title, link, score, comment count and timestamps as clean JSON. It's an **HN API scraper** — fast, keyless, unblockable — and works as a **Hacker News top stories scraper** for daily digests, dashboards and **tech trend monitoring** pipelines. > The tech zeitgeist, structured: 50 stories in seconds, no usernames, no HTML parsing. ### Who uses this - **Content curators & newsletter writers** — the morning Top-30 with scores and comment counts, ready to summarize. - **Founders, PR & marketers** — track when your product, category or competitors hit the front page. - **Trend analysts** — score + comment velocity over scheduled runs = what the dev world cares about this week. - **LLM pipelines** — clean JSON for summarization and classification agents. Want the code layer under the discussion layer? The [GitHub Repositories Scraper](https://apify.com/exuberant_volley/github-repos-scraper) tracks the repos those threads are about. ### How it works The actor fetches the official HN Firebase list you choose (`topstories`, `newstories`, `beststories`), then loads each story item with bounded concurrency (16 parallel requests). Author usernames (`by`) and comment id trees (`kids`) are intentionally excluded. A failed list fetch, all-items-failed, or zero stories pushed **fails the run loudly instead of succeeding empty**. ### Output One record per story: ```json { "id": 48915709, "type": "story", "title": "Jurassic Park computers in excruciating detail", "url": "https://fabiensanglard.net/jurrasic_park_computers/index.html", "score": 425, "commentCount": 102, "time": 1784084267, "timeIso": "2026-07-15T02:57:47.000Z", "scrapedAt": "2026-07-15T09:34:39.233Z" } ```` - `url` is the external link; Ask/Show HN posts may have `url: null` — link them as `https://news.ycombinator.com/item?id=`. - `score` and `commentCount` are point-in-time — schedule runs to capture velocity. ### Input | Field | Default | Description | |---|---|---| | `listType` | `topstories` | `topstories` (Top), `newstories` (New) or `beststories` (Best); short forms `top`/`new`/`best` work too | | `maxItems` | `50` | Stories to fetch from the top of the list (HN exposes up to 500) | **The default run needs no configuration** — an empty input fetches the current Top 50. ### Reliability & limits - **Official HN Firebase API** — no scraping of news.ycombinator.com, nothing to block or break. - Deleted/dead items are skipped automatically; the run fails loudly if literally nothing could be fetched. - Scores change minute-to-minute; treat each run as a snapshot and schedule for time series. ### Paste this output into… - **Google Sheets** — `=IMPORTDATA("https://api.apify.com/v2/acts/exuberant_volley~hackernews-scraper/runs/last/dataset/items?format=csv&clean=1&token=YOUR_TOKEN")` for a self-refreshing front-page sheet. - **Make / n8n** — schedule at 07:00, fetch `.../runs/last/dataset/items`, filter `score > 100`, and post to your team's Slack channel. - **Morning digest (curators)** — pipe the Top-30 JSON into an LLM step ("group by theme, 1-line summary each, keep links") and email it to yourself — a zero-cost personal briefing. ### ScrapeForge free data suite One publisher, fourteen plug-and-play datasets — all currently free to run: | Actor | What it delivers | | |---|---|---| | [Executive Changes Tracker](https://apify.com/exuberant_volley/executive-changes-scraper) | New CEO/CFO/board moves from SEC 8-Ks + official newswires — source-cited B2B trigger leads | [run it next →](https://apify.com/exuberant_volley/executive-changes-scraper) | | [IKEA Product Scraper](https://apify.com/exuberant_volley/ikea-product-scraper) | Names, prices, ratings and images from any IKEA search or category page | [run it next →](https://apify.com/exuberant_volley/ikea-product-scraper) | | [Shopify Store Scraper](https://apify.com/exuberant_volley/shopify-store-scraper) | Any Shopify store's full catalog — variants, SKUs, prices — via products.json | [run it next →](https://apify.com/exuberant_volley/shopify-store-scraper) | | [App Store Apps Scraper](https://apify.com/exuberant_volley/app-store-apps-scraper) | App rankings, ratings, prices and metadata by keyword, any country | [run it next →](https://apify.com/exuberant_volley/app-store-apps-scraper) | | [Remote Jobs Scraper](https://apify.com/exuberant_volley/remote-jobs-scraper) | Live remote listings from RemoteOK + Remotive, keyword-filtered | [run it next →](https://apify.com/exuberant_volley/remote-jobs-scraper) | | [CoinGecko Market Scraper](https://apify.com/exuberant_volley/coingecko-market-scraper) | Top-coin prices, market caps and 24h moves — keyless | [run it next →](https://apify.com/exuberant_volley/coingecko-market-scraper) | | [GitHub Repositories Scraper](https://apify.com/exuberant_volley/github-repos-scraper) | Repo search with stars, forks, topics and licenses | [run it next →](https://apify.com/exuberant_volley/github-repos-scraper) | | [Hacker News Scraper](https://apify.com/exuberant_volley/hackernews-scraper) | Top/New/Best HN stories with scores and comment counts | [run it next →](https://apify.com/exuberant_volley/hackernews-scraper) | | [Wikipedia Scraper](https://apify.com/exuberant_volley/wikipedia-scraper) | Article summaries, images and URLs in any language — RAG-ready | [run it next →](https://apify.com/exuberant_volley/wikipedia-scraper) | | [SEC EDGAR Filings Scraper](https://apify.com/exuberant_volley/sec-edgar-filings-scraper) | Any US public company's filing history by ticker | [run it next →](https://apify.com/exuberant_volley/sec-edgar-filings-scraper) | | [Open-Meteo Weather Scraper](https://apify.com/exuberant_volley/open-meteo-weather-scraper) | 16-day daily forecast for any coordinates — temps, rain, wind — keyless | [run it next →](https://apify.com/exuberant_volley/open-meteo-weather-scraper) | | [arXiv Papers Scraper](https://apify.com/exuberant_volley/arxiv-papers-scraper) | Research papers by topic — abstracts, categories, PDF links, newest-first sort | [run it next →](https://apify.com/exuberant_volley/arxiv-papers-scraper) | | [NVD CVE Scraper](https://apify.com/exuberant_volley/nvd-cve-scraper) | Fresh CVEs with CVSS severity, CWE ids and CISA known-exploited flags | [run it next →](https://apify.com/exuberant_volley/nvd-cve-scraper) | | [npm Registry Scraper](https://apify.com/exuberant_volley/npm-registry-scraper) | npm package vitals — latest version, license, weekly downloads, dependency count | [run it next →](https://apify.com/exuberant_volley/npm-registry-scraper) | ### Compliance - **No personal data** — the author field (`by`) and comment id list (`kids`) are intentionally excluded; story catalog data only, GDPR-safe. - **Official API underneath** — the public HN Firebase API; no HTML parsing. - Records live only in your run's dataset; the actor keeps nothing beyond it. # Actor input Schema ## `listType` (type: `string`): Which Hacker News list to scrape. ## `maxItems` (type: `integer`): How many stories to fetch from the top of the list (HN returns up to 500 ids). ## Actor input object example ```json { "listType": "topstories", "maxItems": 50 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "listType": "topstories", "maxItems": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("exuberant_volley/hackernews-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "listType": "topstories", "maxItems": 50, } # Run the Actor and wait for it to finish run = client.actor("exuberant_volley/hackernews-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "listType": "topstories", "maxItems": 50 }' | apify call exuberant_volley/hackernews-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=exuberant_volley/hackernews-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hacker News Scraper", "description": "Scrape Hacker News stories — top, new or best — with title, url, score, comment count and timestamps, via the official HN Firebase API. Clean JSON, no author usernames, no personal data.", "version": "0.1", "x-build-id": "9A9BlZRCPPToVrj7e" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/exuberant_volley~hackernews-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-exuberant_volley-hackernews-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/exuberant_volley~hackernews-scraper/runs": { "post": { "operationId": "runs-sync-exuberant_volley-hackernews-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/exuberant_volley~hackernews-scraper/run-sync": { "post": { "operationId": "run-sync-exuberant_volley-hackernews-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "listType": { "title": "Story list", "enum": [ "topstories", "newstories", "beststories" ], "type": "string", "description": "Which Hacker News list to scrape.", "default": "topstories" }, "maxItems": { "title": "Max stories", "minimum": 1, "maximum": 500, "type": "integer", "description": "How many stories to fetch from the top of the list (HN returns up to 500 ids).", "default": 50 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```