# Hacker News Scraper API: Top, New, Best, Ask, Show Stories (`rowcraft/hacker-news-scraper`) Actor Read Hacker News stories from the official Firebase API: top, new, best, Ask HN, Show HN and job lists with score, comments, domain and age. Filter by keyword, score and domain. No proxy, no usernames, you pay only for stories delivered. - **URL**: https://apify.com/rowcraft/hacker-news-scraper.md - **Developed by:** [Rowcraft](https://apify.com/rowcraft) (community) - **Categories:** News, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.12 / 1,000 hacker news stories This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Hacker News Scraper API: Top, New, Best, Ask, Show Stories Get the stories of **Hacker News** as clean rows: the front page (top), the newest stories, the best ones, Ask HN, Show HN and the job posts, each with its rank, title, link, domain, points, comment count and age, as JSON, CSV or Excel. The actor reads the official Hacker News API, not the website, so there is no page parsing to break, no proxy and no login. Filter by keyword, minimum score, minimum comments, domain and age before anything is charged, and run it on a schedule to receive only the stories you have not seen yet. It is built for monitoring and research, and it leaves people out on purpose: no usernames and no comments are ever read, only the number of comments. ![Input form of Hacker News Scraper API: Top, New, Best, Ask, Show Stories](https://api.apify.com/v2/key-value-stores/7hbQFQdGq61v2OZ2h/records/hacker-news-scraper-input.png) ### What you get One row per story, with 18 fields: - **Identity:** the Hacker News item `id`, the `type` (story, job or poll) and `hnUrl`, the link to the discussion on Hacker News - **Where it was found:** the `list` (top, new, best, ask, show or job), the `rank` in that list (1 = first) and `alsoIn`, the other lists of your selection that contain the same story - **The story itself:** `title`, `url` (null for Ask HN and other text posts) and `domain`, the host of the link without `www.` - **Reception:** `score` (points) and `commentCount`, the number of comments - **Time:** `postedAt` (ISO 8601, UTC) and `ageHours` at the moment the lists were read - **Kind:** `isAsk`, `isShow` and `isJob`, so you can split Ask HN, Show HN and job posts without parsing titles - **Text:** `text`, the body of Ask HN, Show HN and job posts as plain text, with e-mail addresses and phone numbers masked - **Run time:** `scrapedAt`, when the lists were read Empty values are `null`, never missing, so CSV and Excel columns stay stable from run to run. The run also writes a `COVERAGE` record with one line per list you asked for (how many ids it holds, how many were read, delivered, filtered, deleted or already delivered) and a `RUN_SUMMARY` with the totals, the filters applied and why the run stopped. ### Why this actor - **Official API only.** Every request goes to the public Hacker News API on Firebase: six lists of story ids and one item per id. The website is never loaded, so a change in its layout cannot break the actor. - **Filters before charging.** Keywords, exclusions, domains, minimum points, minimum comments and maximum age are applied before a story is delivered. Stories your filters drop cost nothing. - **A story once.** A story that is in several lists (a Show HN that is also on the front page, for example) is delivered once, under the first list in your order, with the others in `alsoIn`. - **Fast and polite.** Up to 10 requests are in flight and a new one starts every 100 ms by default, so a list of 500 stories takes about a minute. The pace is yours to change with `requestIntervalMs`. - **Count first.** Tick `countOnly` to see how many stories match on each list and what they would cost, for free. - **Ready for schedules.** `onlyNew` delivers, and charges, only stories you have not received before, and `notifyUrl` sends a message to Slack, Discord, Make, n8n or Zapier when the run ends. - **No people.** Author names are not read, comments are not read, user pages are not requested. ### How to use it 1. Open the Actor and fill in **Hacker News lists** (the other fields are optional filters). 2. Set **Maximum stories** to a small number for a first try (the default is a safe start). 3. Optional: tick **Count only** and start once. You get the number of matching rows and the estimated cost, for free (only the start event is charged). Untick it to get the data. 4. Click **Start**. The status line shows progress; the run stops cleanly at your limit or at the charge limit you set in the run options. 5. Open the **Dataset** tab and download JSON, CSV, Excel, XML or HTML, or send the rows to Google Sheets, Slack, Make, n8n or Zapier with the Integrations tab. 6. To run it on a schedule, use **Schedules** in Apify Console and turn on **Only new rows since the last run** below: every run then costs only the new rows, and a webhook URL tells you when something new came in. A few habits that save money and time: - The lists are read in rank order, so `maxItems` of 30 on **Top** gives the 30 highest-ranked stories that pass your filters, not a random sample. - Combine lists freely. `["top", "best"]` returns the stories of both without repeats. - Keywords match whole words, not parts of words: `ai` finds "AI-powered" and "AI/ML", but not "said" or "maintain". - For a daily digest, read the **Top** list with `onlyNew` on and a `minScore` that suits you, and put a Slack webhook in `notifyUrl`. ### Input | Field | What it does | |---|---| | `lists` | Which lists to read: `top`, `new`, `best`, `ask`, `show`, `job`. Default `top`. The order decides which list a story is delivered under. | | `maxItems` | Stop after this many delivered stories. Default 100, 0 = every story of the chosen lists. | | `keywords` | Keep stories whose title or text contains any of these words or phrases (whole words, not case-sensitive). | | `excludeKeywords` | Drop stories whose title or text contains any of these. | | `domains` | Keep only stories linking to these sites. Sub-domains match, so `github.com` also keeps `gist.github.com`. Ask HN posts have no link and are dropped when this is set. | | `excludeDomains` | Drop stories linking to these sites. | | `minScore`, `minComments` | Keep stories with at least this many points or comments. | | `postedWithinHours` | Keep stories posted in the last N hours. 0 = any age. | | `includeText` | Deliver the text of Ask HN, Show HN and job posts. Default on; keywords search the text either way. | | `countOnly` | Count the matching stories per list and estimate the cost; nothing is delivered. | | `onlyNew`, `stateName` | Deliver a story only if you have not received it in an earlier run with the same state name. | | `notifyUrl`, `notifyWhenEmpty` | A webhook that receives one message when the run ends; see **Automation** below. | | `requestIntervalMs` | Minimum time between two requests, default 100 ms, minimum 20 ms. | Example: the 30 highest-ranked front-page stories with at least 100 points. ```json { "lists": ["top"], "maxItems": 30, "minScore": 100 } ``` Example: Show HN posts of the last two days about AI, with the text. ```json { "lists": ["show"], "keywords": ["AI", "LLM", "GPT", "agent"], "postedWithinHours": 48 } ``` Example: every story in the best list that links to GitHub. ```json { "lists": ["best"], "domains": ["github.com"], "maxItems": 0 } ``` Example: an hourly watch on new stories, with a Slack message when something about your product turns up. ```json { "lists": ["new", "top"], "keywords": ["postgres", "supabase"], "onlyNew": true, "stateName": "database-watch", "notifyUrl": "https://hooks.slack.com/services/..." } ``` ### Output 18 fields per row. Empty values are `null`, never missing, so CSV and Excel columns are stable. | Field | Type | Example | |---|---|---| | `id` | integer | 50141909 | | `type` | string | story | | `list` | string | top | | `alsoIn` | string | best, show | | `rank` | integer | 3 | | `title` | string | Show HN: I built a tiny database in Rust | | `url` | string | `https://example.org/blog/tiny-database` | | `domain` | string | example.org | | `hnUrl` | string | `https://news.ycombinator.com/item?id=50141909` | | `score` | integer | 212 | | `commentCount` | integer | 87 | | `postedAt` | string | 2026-10-18T15:26:06.000Z | | `ageHours` | number | 1.7 | | `isAsk` | boolean | False | | `isShow` | boolean | True | | `isJob` | boolean | False | | `text` | string | We built it because... | | `scrapedAt` | string | 2026-10-18T17:05:25.789Z | #### Example rows Two real rows from a run (long texts shortened here, complete in the dataset): ```json [ { "id": 50138531, "type": "story", "list": "top", "alsoIn": null, "rank": 6, "title": "DeepSeek search and fetch – $0.75 per 1000 successful requests", "url": "https://api-docs.deepseek.com/guides/search/", "domain": "api-docs.deepseek.com", "hnUrl": "https://news.ycombinator.com/item?id=50138531", "score": 419, "commentCount": 206, "postedAt": "2026-10-18T06:50:36.000Z", "ageHours": 10.4, "isAsk": false, "isShow": false, "isJob": false, "text": null, "scrapedAt": "2026-10-18T17:14:52.627Z" }, { "id": 50137950, "type": "story", "list": "show", "alsoIn": null, "rank": 2, "title": "Show HN: Sphinx – A hook that quizzes you on your AI's code before shipping", "url": "https://sphinx-six-theta.vercel.app/", "domain": "sphinx-six-theta.vercel.app", "hnUrl": "https://news.ycombinator.com/item?id=50137950", "score": 210, "commentCount": 71, "postedAt": "2026-10-18T04:33:49.000Z", "ageHours": 12.7, "isAsk": false, "isShow": true, "isJob": false, "text": "FOSS, Basically it plugs into your existing coding agent workflow and every time you push to prod it'll throw up a questionnaire that che…", "scrapedAt": "2026-10-18T17:15:02.140Z" } ] ``` Results can be downloaded as JSON, CSV, Excel, XML or HTML from the Dataset tab, or read through the Apify API. ![Results of Hacker News Scraper API: Top, New, Best, Ask, Show Stories in the Apify dataset view](https://api.apify.com/v2/key-value-stores/7hbQFQdGq61v2OZ2h/records/hacker-news-scraper-results.png) ### Pricing **Pay per result.** You pay for each story delivered to your dataset, nothing else: no subscription, no minimum, no charge for rows your filters drop or for the requests behind them. | Your Apify plan | Price per 1,000 rows | Price per row | |---|---|---| | Free plan | **$0.18** | $0.00018 | | Starter plan | **$0.16** | $0.00016 | | Scale plan | **$0.14** | $0.00014 | | Business plan and above | **$0.12** | $0.00012 | Example: 1,000 stories cost $0.18 on the Free plan and $0.12 on the Business plan. Platform usage of the run (compute, storage) is included in that price and is not billed to you separately (pay per event: the actor covers it). A start event of $0.00005 is charged once per run (Apify default). **Cap your spend:** set *Maximum total charge* in the run options. The actor stops cleanly when the cap is reached and you keep everything delivered until then. The run summary (`RUN_SUMMARY` in the key-value store) says why a run stopped. **Check before you pay:** turn on **Count only** to see how many rows match and what they would cost at your tier. That run delivers nothing and costs only the start event. What that means in practice: a full read of the Top list (500 stories) costs about $0.09 on the Free plan, and all six lists together (1,113 distinct stories on 18 October 2026, because many stories are in more than one list) cost about $0.20. With filters the bill shrinks, because only the stories that pass are charged. ### Automation: only new stories, notifications, schedules **Only new rows since the last run.** Turn on `onlyNew` and the actor remembers every story it delivered to you (in a key-value store named `rowcraft-hacker-news-scraper-state` in your Apify account). The next run with the same `stateName` skips what you already have and you pay only for new rows. The first run delivers everything that matches. Use a different `stateName` for each schedule or search so they keep separate memories. **Notification when the run ends.** Put a webhook address in `notifyUrl` and the actor sends one JSON POST with the number of new rows, links to the dataset (JSON and CSV) and the run. It works out of the box with Slack and Discord incoming webhooks (the message is in `text` and `content`), with Make, n8n and Zapier catch hooks, and with your own endpoint. By default it is sent only when there is at least one new row; `notifyWhenEmpty` sends it every time. **Schedules.** Create a schedule in Apify Console (hourly, daily, weekly or cron) with `onlyNew` on. A daily schedule gives you a feed of new stories for the price of the new rows only. **Integrations.** Send the results to Make, n8n, Zapier, Google Sheets, Slack, Google Drive, Airtable or any webhooks from the Integrations tab of the Actor, with no code. Apify also offers native actions for Make, n8n and Zapier. **API, SDKs and AI agents.** Every run is available through the Apify API and the official JavaScript and Python clients; the Actor can also be called by AI agents through the Apify MCP server. The example input in the API tab is a working call. **Count only.** `countOnly` returns the number of matching stories and the estimated cost without delivering anything, so you can size a job before paying for it. A note on how "new" is decided here: a story is identified by its Hacker News `id`. A story you already received is not delivered again when its points or comments grow, so `onlyNew` is a feed of stories you have not seen, not a feed of changes. To follow how a story's score develops, run without `onlyNew` and compare runs by `id`. ### Use cases - **Launch and mention monitoring:** watch the new and top lists for your product, your competitors or your open-source project, and get a Slack message when one appears. - **Content and newsletter curation:** pull the best stories of the day or the week above a score you choose, with the link and the discussion link, and feed an editor, a digest or a social queue. - **Trend research:** count how many stories per week mention a language, a database or a technology, and how many points they earn, from `title`, `score` and `postedAt`. - **Show HN tracking for founders and investors:** collect Show HN posts with their points and comment counts to see which launches get traction. - **Hiring signals:** read the job list to see which companies are hiring from the Hacker News community. - **Link analysis:** group stories by `domain` to see which sites and blogs earn the most points, or which ones are never upvoted. - **Datasets for analysis and AI:** a clean, flat table of titles, links, points and times, without personal data. Ready-made examples, each with the input filled in (open one and press Start, or save it as your own task): - [Get the top Hacker News stories with 100+ points](https://apify.com/rowcraft/hacker-news-scraper/examples/hacker-news-top-stories-over-100-points) - [Find new Show HN projects about AI and LLMs](https://apify.com/rowcraft/hacker-news-scraper/examples/show-hn-ai-llm-projects-last-48-hours) - [List the best Hacker News stories that link to GitHub](https://apify.com/rowcraft/hacker-news-scraper/examples/best-hacker-news-stories-linking-to-github) ### What this actor does not do - Read comments or authors: no comment text, no comment authors, no usernames, no user pages and no karma. Only the number of comments is delivered. This is a privacy choice, explained in the Privacy section. - Search the archive: only the current lists are read, up to 500 stories on Top and New and up to 200 on Best. For older stories use the story's own link. - Read the Hacker News website: no page of the site is loaded, and no search service is used. - Post, vote, comment, flag or log in. - Follow the links of the stories: the target page is not opened, so there is no article text. Use a page extractor for that. ### Limits and fair use - **List sizes.** The API gives up to 500 ids for Top and New and up to 200 for Best. On 18 October 2026 the other lists held 160 (Show), 27 (Ask) and 31 (Jobs). The size is set by Hacker News and changes through the day. - **A snapshot, not a stream.** The lists are read once at the start of the run and the rank is the position at that moment. Stories posted during a long run are not added. - **Speed.** Each story is one request. At the default pace of 10 requests a second, 30 stories take a few seconds and the 500 stories of Top about a minute. A larger `requestIntervalMs` makes it slower and gentler. - **No rate limit is published**, and the actor still keeps at most 10 requests in flight, retries a failed request with growing pauses and stops at once on an anti-bot page or after five failed stories in a row. It uses no proxy, no CAPTCHA solving and no cookies. - **robots.txt is read first.** The API host allows the `.json` paths this actor uses. If that ever changes, the run stops and says so in `RUN_SUMMARY` and `COVERAGE`. - **Deleted and dead stories** (removed by their authors or flagged by the community) and ids that now point to nothing are skipped and not charged. - **Memory.** The actor runs in 256 MB; a run holds a few thousand ids and one small item at a time. - The run stops 90 seconds before its timeout with the summary and the **Only new rows** memory saved. ### Data source and licence The data comes from the public Hacker News API on Firebase, `https://hacker-news.firebaseio.com/v0/`, documented in the official repository on [GitHub](https://github.com/HackerNews/API). Its documentation describes the data as public and available in near real time, and says there is no rate limit at present. The actor reads only the id lists (`topstories`, `newstories`, `beststories`, `askstories`, `showstories`, `jobstories`) and the items behind them. The website at [news.ycombinator.com](https://news.ycombinator.com) is not loaded; `hnUrl` is only a link you can open. Titles, links and texts are written by Hacker News users and by the companies that post jobs, and belong to their authors. Keep the `hnUrl` link with a story when you show it, and check the terms of the site before you republish content beyond analysis and monitoring. Rowcraft is independent and not affiliated with or endorsed by Y Combinator or Hacker News. Names are used only to say where the data comes from. ### Privacy: no personal data Rowcraft actors are built not to collect personal data, and this one is stricter than the source: - The author of a story is never copied into a row, a summary or a log. The actor does not request user pages either. - Comments are never requested; only their count is delivered. - The text of Ask HN, Show HN and job posts is delivered as plain text with e-mail addresses, phone numbers and profile links masked, and with the names of contact persons masked where a contact cue stands next to them ("contact", "recruiter", "Mr", "Frau" and similar). A name written inside a normal sentence of a post cannot be told from other words reliably and is not touched; turn `includeText` off if you need rows without any free text. - Nothing is sent to the stories' sites, and the only outside address contacted is the API host. You are responsible for using the data in line with the law that applies to you. ### Other Rowcraft actors All Rowcraft actors read official or public sources, deliver flat rows without personal data and support only-new runs and notifications: - [Ashby Jobs Scraper – Job Postings with Salary Ranges](https://apify.com/rowcraft/ashby-jobs-scraper) — Job postings from Ashby job boards (jobs.ashbyhq.com) by company: title, department, team, locations with country, employment and workplace type, published salary range, full text. - [Broken Link Checker – Find 404s and Dead Links API](https://apify.com/rowcraft/broken-link-checker) — Pay per crawled page, not per link: crawl a site within robots.txt and check every link on each page for 404s, 5xx errors, DNS and TLS failures and redirects. - [Bulk Image Downloader API: Save Page Images with Sizes](https://apify.com/rowcraft/bulk-image-downloader) — Download the images of any web pages, or direct image links, into a key-value store with width, height, format, size and a download link per file. - [Career Site Jobs Scraper: Greenhouse, Lever, Ashby, Workday](https://apify.com/rowcraft/career-site-jobs-scraper) — Job listings from the career pages of the companies you choose on Greenhouse, Lever, Ashby, Workday, Workable, Personio, Recruitee, Teamtailor and Gem, in one run and one format: title, locations, department, salary when published, full text. - [Chrome Web Store Scraper: Users, Ratings & Versions](https://apify.com/rowcraft/chrome-web-store-scraper) — Reads public Chrome Web Store listing pages with no login, no proxy and no personal data: users, rating, category, version, update date, permissions and company developers. - [Domain Email and DNS Checker API: MX, SPF, DMARC, Providers](https://apify.com/rowcraft/domain-email-dns-checker) — Find which email provider, sending services and DNS host each domain uses, from public DNS records only. - [SIRENE France Company Scraper: SIREN, NAF, Revenue, Status](https://apify.com/rowcraft/france-company-register) — French companies from the official Recherche d'entreprises API (INSEE SIRENE + INPI): SIREN and SIRET, legal form, NAF activity, head-office address, employee band, revenue and net income, labels, creation date and VAT number. - [Gem Jobs Scraper – Job Board Listings from jobs.gem.com](https://apify.com/rowcraft/gem-jobs-scraper) — Job listings from Gem job boards (jobs.gem.com) by company: title, department, offices with country, employment type, remote, hybrid or on-site, requisition ID, publication date and full ad text, from the public Gem Job Board API. - [Google Play Apps Scraper: App Details, Installs, Ratings](https://apify.com/rowcraft/google-play-apps-scraper) — Public Google Play app details as flat rows: installs, rating, number of ratings, price, version, last update, minimum Android, ads and in-app purchases. - [Greenhouse Jobs Scraper – Job Listings by Company Board](https://apify.com/rowcraft/greenhouse-jobs-scraper) — Job listings from Greenhouse job boards by company: title, departments, offices, publication date, full ad text, published pay ranges. - [Himalayas Jobs Scraper: Remote Jobs API with Salaries](https://apify.com/rowcraft/himalayas-jobs-scraper) — Remote jobs from Himalayas through its public jobs API: title, company, salary range, seniority, categories, the countries and time zones open to candidates, full description and job link. - [Image to Text OCR API: Extract Text from Images in Bulk](https://apify.com/rowcraft/image-ocr-text-extractor) — Read the text in images given by link or found on web pages, with a local OCR engine and no third-party API. - [KRS Poland Companies Scraper: Register Data, PKD, Status](https://apify.com/rowcraft/krs-poland-companies) — Polish companies from the official KRS register API of the Ministry of Justice: name, KRS, NIP and REGON numbers, legal form, status, address, PKD activity codes, share capital, branches and registration dates. - [LEI Records Scraper: Legal Entity Identifier Lookup API](https://apify.com/rowcraft/lei-records-scraper) — Legal Entity Identifier (LEI) records from the Global LEI Index, read through the public GLEIF API: legal name, legal form, register number, addresses, status, renewal date and parent companies. - [Lever Jobs Scraper – Job Postings from jobs.lever.co](https://apify.com/rowcraft/lever-jobs-scraper) — Job postings from Lever career pages (jobs.lever.co) by company: title, team, department, location, commitment, workplace type, salary range when published, description, requirements and benefits. - [Lighthouse Audit API: Core Web Vitals and Performance Scores](https://apify.com/rowcraft/lighthouse-performance-audit) — Run Lighthouse on any public page and get Performance, Accessibility, Best Practices and SEO scores, lab Core Web Vitals and the biggest savings. - [Norway Company Register Scraper: Brønnøysund Business Data](https://apify.com/rowcraft/norway-company-register-scraper) — Norwegian companies from the Central Coordinating Register for Legal Entities (Brønnøysund, NLOD open data): organisation number, legal form, industry, employees, share capital, addresses, website, VAT and bankruptcy status. - [PDF Text Extractor: Text, Pages and Metadata from PDF Links](https://apify.com/rowcraft/pdf-text-extractor) — Get the text of every PDF you link, page by page, with page count, language and word count. - [Personio Jobs Scraper: Job Listings, Salaries, Departments](https://apify.com/rowcraft/personio-jobs-scraper) — Job ads from any company's Personio careers feed: title, department, office, employment type, schedule, seniority, salary when published, keywords and the full description in the language you choose. - [Platsbanken Scraper: Sweden Jobs, Employers, Org Numbers](https://apify.com/rowcraft/platsbanken-jobs-scraper) — Every job ad on Platsbanken (Arbetsförmedlingen) from the official JobTech API: headline, employer and organisation number, occupation codes, municipality, coordinates, salary type, deadline and full text. - [Recruitee Jobs Scraper – Job Listings by Company Career Site](https://apify.com/rowcraft/recruitee-jobs-scraper) — Job listings from Recruitee career sites by company: title, department, locations with country, employment type, remote flag, published salary and full text, one request per company. - [Remote Jobs Aggregator API: RemoteOK, WeWorkRemotely, Jobicy](https://apify.com/rowcraft/remote-jobs-aggregator) — Remote jobs from RemoteOK, We Work Remotely and Jobicy in one clean list, read from their official feeds and APIs, merged without duplicates. - [Robots.txt and AI Crawler Access Checker API](https://apify.com/rowcraft/robots-ai-crawler-checker) — Check which AI and search crawlers a website lets in: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more, read from robots.txt by the RFC 9309 rules. - [RSS Feed Finder and Reader – Blog and News Article API](https://apify.com/rowcraft/rss-feed-finder-reader) — Find the RSS, Atom and JSON feeds of any website and read their newest articles, from a domain name alone. - [SEO Audit Tool: Titles, Meta Tags, Headings and Links API](https://apify.com/rowcraft/seo-audit-tool) — Audit the on-page SEO of any list of pages or websites: titles, meta descriptions, headings, canonicals, indexing, links, images and structured data, with a score and the issues found per page. - [Shopify Products Scraper: Store Catalog Export to JSON, CSV](https://apify.com/rowcraft/shopify-products-scraper) — Products from any Shopify store's public catalog: title, vendor, type, tags, description, variants with price, compare-at price, SKU and stock, images, options, store currency. - [Sitemap URL Extractor: Every Page with Last-Modified Dates](https://apify.com/rowcraft/sitemap-url-extractor) — Get every page a site lists in its sitemaps, with last-modified dates, from robots.txt through sitemap indexes and .gz files. - [SSL Certificate and Security Headers Checker API](https://apify.com/rowcraft/ssl-security-headers-checker) — Check the TLS certificate and the HTTP security headers of every domain in one run, with an A+ to F grade and the exact issues to fix. - [Store Products Scraper – Shopify, WooCommerce, VTEX](https://apify.com/rowcraft/store-products-scraper) — Export the product catalogue of any Shopify, WooCommerce or VTEX store from its public catalogue endpoint: title, brand, prices, discounts, stock, categories, images and links, one row per product, the same columns on every platform. - [Structured Data Extractor: JSON-LD, Schema.org, Open Graph](https://apify.com/rowcraft/structured-data-extractor) — Pull the schema.org JSON-LD, Microdata, RDFa and Open Graph of any page list into flat rows: types, product price, availability, ratings, organization, breadcrumbs, with validation issues. - [Teamtailor Jobs Scraper – Career Site Jobs by Company](https://apify.com/rowcraft/teamtailor-jobs-scraper) — Job listings from Teamtailor career sites by company: title, department, role, locations with city and country, remote or hybrid status, publication date and full ad text, from the official job feed. - [TED Tenders Scraper: EU Public Procurement Notices API](https://apify.com/rowcraft/ted-tenders-scraper) — EU public procurement notices from TED (Tenders Electronic Daily), the official EU tenders portal, through its public Search API: title, buyer, country, CPV codes, values, deadlines, lots and document links. - [UK Companies House Scraper: Company Search & Profiles API](https://apify.com/rowcraft/uk-companies-house-scraper) — UK companies from the official Companies House API with your free key: search by name, status, type, SIC code, date or location, or look up company numbers. - [UK Tenders Scraper: Find a Tender + Contracts Finder API](https://apify.com/rowcraft/uk-tenders-scraper) — UK public procurement notices from Find a Tender and Contracts Finder, the official UK government portals, through their open OCDS APIs: title, buyer, region, CPV codes, values, deadlines, notice type and links. - [Bulk URL Status and Redirect Chain Checker API](https://apify.com/rowcraft/url-status-redirect-checker) — Check the status code, redirect chain, canonical and indexing signals of every URL in a list, with the final destination and a loop flag. - [Video Frame Extractor: Thumbnails and Frames from Video URLs](https://apify.com/rowcraft/video-frame-extractor) — Extract JPG, PNG or WebP frames from direct video links (MP4, WebM, MOV, MKV, OGV): evenly spaced, every N seconds, at exact times or at scene changes. - [VTEX Products Scraper – Public Catalog Export](https://apify.com/rowcraft/vtex-products-scraper) — Export the catalogue of any VTEX store (Auchan, Chedraui and thousands of retailers in Europe and Latin America) from its public search endpoint: title, brand, EAN, prices, list prices, stock, categories, specifications and images, one row per product. - [Website to Markdown Extractor: Clean Text for LLMs and RAG](https://apify.com/rowcraft/website-markdown-extractor) — Turn a list of web page addresses into clean Markdown and plain text for LLMs and RAG: the main content only, with headings, links, images and tables. - [Website Screenshot API: Full-Page PNG, JPEG and PDF Captures](https://apify.com/rowcraft/website-screenshot-api) — Capture full-page or viewport screenshots of any public web page as PNG, JPEG or WebP, with an optional A4 PDF. - [Website Tech Stack Detector – Technology Lookup API](https://apify.com/rowcraft/website-tech-stack-detector) — Find the technologies behind a list of websites: CMS, e-commerce platform, analytics, tag managers, ad pixels, consent tools, chat, CDN and payments, from one polite home-page request per domain. - [WooCommerce Products Scraper – Store API Export](https://apify.com/rowcraft/woocommerce-products-scraper) — Export every product of a WooCommerce store through its public Store API: title, SKU, prices and sale prices, stock, categories, tags, attributes, ratings, images and links, one row per product. - [Workable Jobs Scraper: Job Listings from Career Pages](https://apify.com/rowcraft/workable-jobs-scraper) — Job listings from any company's Workable career page: title, department, location, remote or on-site, employment type, description, requirements, benefits and salary when published. - [Workday Jobs Scraper – Jobs from myworkdayjobs.com Sites](https://apify.com/rowcraft/workday-jobs-scraper) — Job listings from Workday career sites (myworkdayjobs.com) by company: title, locations, country, posting date, time type, requisition ID and the full description. ### FAQ **What does the Hacker News Scraper API return?** One row per story or job post from the lists you choose: rank, title, link, domain, points, comment count, age, whether it is an Ask HN, Show HN or job post, and the text of text posts. See the Output section for the 18 fields. **Which lists are available?** Top (the front page), New, Best, Ask HN, Show HN and Jobs. You can read one or all six in a run; a story that is in several is delivered once. **Where does the data come from?** From the official Hacker News API on Firebase. The actor does not load the Hacker News website and does not use any search service. **Is it legal?** The actor reads a public API that its operator documents and offers openly, one request per story, politely, and delivers public information without personal data. The content itself belongs to its authors, so how you reuse titles and texts is your responsibility under the law that applies to you. This is not legal advice. **Does it collect personal data?** No personal data: no author names, no comments, no user pages, and e-mail addresses and phone numbers inside post texts are masked. See the Privacy section. **Why can't I get usernames or comments?** On purpose. Usernames and comment texts identify people, and Rowcraft actors deliver data without persons. The number of comments is delivered, because it shows how much a story is discussed. **Why is the result empty?** Usually the filters are too strict for the lists you chose (a high `minScore` on the New list, a domain nobody linked today, a keyword that is not in any current title), or **Only new rows** is on and everything that matches was already delivered to you. Open `RUN_SUMMARY`: it shows how many stories were read, how many your filters dropped and how many were skipped as already delivered, and `COVERAGE` shows each list. Nothing is charged for stories you do not receive, apart from the start event. **Why did I get fewer stories than I asked for?** Either the lists hold fewer stories than your maximum (Ask HN and the job list are short), or fewer stories passed your filters. The `COVERAGE` record shows, per list, how many ids it held and what happened to them. **Can I get older stories?** Not with this actor. It reads the current lists only, so the oldest story you can reach is on the New list, a few hundred stories back. The stories' `hnUrl` links lead to the full discussion. **Do I pay for stories my filters drop, for duplicates or for deleted stories?** No. Only stories delivered to your dataset are charged. Duplicates between lists, filtered stories, deleted or dead stories and stories skipped by **Only new rows** are free, and so is a **Count only** run. **Can I follow a keyword and get a message?** Yes. Use `keywords`, turn on `onlyNew` with a `stateName`, schedule the run in Apify Console and put a Slack, Discord, Make, n8n or Zapier address in `notifyUrl`. **How up to date is it?** The lists are read at the start of each run, so the data is as fresh as Hacker News itself. Schedule the actor as often as you need; every run costs only the new stories when `onlyNew` is on. ### Support and reviews **Problems, missing fields, questions:** open an issue on the **Issues** tab of this Actor. We answer within one working day and fix confirmed bugs in the next release; the changelog lists every change. If this Actor saved you time, a short review on its page helps other users find it and helps us keep improving it. # Changelog This Actor's version history is a separate document: https://apify.com/rowcraft/hacker-news-scraper/changelog.md # Actor input Schema ## `lists` (type: `array`): Which lists to read. A story that is in several lists is delivered once, under the first list in your order, with the others in `alsoIn`. Top is the front page. ## `maxItems` (type: `integer`): Stop after this many stories are delivered. Lists are read in rank order, so 30 on Top gives the 30 highest-ranked stories that pass your filters. 0 = every story of the chosen lists. ## `keywords` (type: `array`): Keep stories whose title or text contains any of these words or phrases, for example "rust", "postgres", "open source". Whole words, not case-sensitive: "ai" finds "AI-powered", not "said". Leave empty for all stories. ## `excludeKeywords` (type: `array`): Drop stories whose title or text contains any of these words or phrases. ## `domains` (type: `array`): Keep only stories that link to these sites, for example "github.com" or "arxiv.org". A link such as https://github.com/user works too. Sub-domains match (gist.github.com is github.com). Ask HN posts have no link and are dropped when this is set. ## `excludeDomains` (type: `array`): Drop stories that link to these sites. ## `minScore` (type: `integer`): Keep stories with at least this many points. 0 = no limit. ## `minComments` (type: `integer`): Keep stories with at least this many comments (only the count is delivered, never the comments). 0 = no limit. ## `postedWithinHours` (type: `integer`): Keep stories posted in the last N hours, for example 48. 0 = any age. ## `includeText` (type: `boolean`): Ask HN, some Show HN and job posts carry a text. Delivered as plain text with e-mail addresses and phone numbers masked. Turn off for a lighter dataset; keywords still search the text. ## `countOnly` (type: `boolean`): Only count how many rows match your filters and estimate the cost at your price tier. Nothing is delivered and nothing is charged apart from the start event. Turn it off to get the data. ## `onlyNew` (type: `boolean`): Remember the rows already delivered to you (in a key-value store in your Apify account) and deliver only rows that appeared since the last run with the same state key. The first run delivers everything that matches; later runs cost only the new rows. Ideal for schedules. ## `stateName` (type: `string`): Name of the memory used by "Only new rows". Use a different name for each schedule or search (for example `stockholm-python`) so they do not share one memory. ## `notifyUrl` (type: `string`): A webhook address that receives one JSON POST with the number of new rows and links to the dataset: a Slack or Discord incoming webhook, a Make, n8n or Zapier catch hook, or your own endpoint. The message is also in the `text` (Slack) and `content` (Discord) fields. ## `notifyWhenEmpty` (type: `boolean`): By default the notification is sent only when at least one row was delivered. ## `requestIntervalMs` (type: `integer`): Minimum time between two requests to the API, default 100 ms (at most 10 requests at once). The official API has no rate limit; raise this to be gentler. Minimum 20. ## Actor input object example ```json { "lists": [ "top" ], "maxItems": 30, "minScore": 0, "minComments": 0, "postedWithinHours": 0, "includeText": true, "countOnly": false, "onlyNew": false, "stateName": "default", "notifyWhenEmpty": false, "requestIntervalMs": 100 } ``` # Actor output Schema ## `results` (type: `string`): One row per Hacker News story or job post, in the default dataset (JSON, CSV, Excel and other formats via the dataset API). ## `runSummary` (type: `string`): Counts per list, filters applied, warnings and why the run stopped (RUN_SUMMARY record in the default key-value store). ## `coverage` (type: `string`): One line per list you asked for: ok, partial, not_read, error, blocked or robots_disallowed, with the ids in the list, how many were read and delivered (COVERAGE record). # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "lists": [ "top" ], "maxItems": 30 }; // Run the Actor and wait for it to finish const run = await client.actor("rowcraft/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "lists": ["top"], "maxItems": 30, } # Run the Actor and wait for it to finish run = client.actor("rowcraft/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "lists": [ "top" ], "maxItems": 30 }' | apify call rowcraft/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,rowcraft/hacker-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/l2396i9WutfGYLGIA/builds/KRbSnjVKG1fQQD8ul/openapi.json