# Hacker News Scraper: Stories, Comments & Who Is Hiring Jobs (`pankajbhatia/hacker-news-scraper`) Actor Search Hacker News stories and comments, get the front page, Show HN and Ask HN, or extract every 'Who is hiring?' job post with company, remote and salary. Official HN search API: fast, reliable, cheap. - **URL**: https://apify.com/pankajbhatia/hacker-news-scraper.md - **Developed by:** [Pankaj Bhatia](https://apify.com/pankajbhatia) (community) - **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.30 / 1,000 hn items This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Hacker News Scraper: Stories, Comments & Who Is Hiring Jobs Get data from **Hacker News** (news.ycombinator.com) as a clean table: - **Search** stories, comments, Show HN, Ask HN, polls and YC job posts by keyword. Sort by newest or most relevant, and filter by points, comments and date. - **Front page**: the stories on the HN front page right now, with points and comment counts. - **"Who is hiring?" job posts**: every job post from the monthly *Ask HN: Who is hiring?* threads, with **company, remote/onsite, visa sponsorship and salary** pulled out of the post, plus the full text. It uses the **official Hacker News search API** (run by Algolia for Y Combinator), so it's fast and reliable, with no blocking or proxies. It can go past the API's usual 1,000-result limit for large date-sorted exports. **Privacy-friendly**: usernames are off by default and email addresses in posts are hidden. ### Who is it for? - **Founders and marketers**: track mentions of your product, competitors or market on HN, and find what launches went viral. - **Recruiters and job seekers**: get every *Who is hiring?* post in a spreadsheet and filter by remote, Python, React or visa. - **Researchers and analysts**: study tech trends, sentiment and discussions over the years. - **AI builders**: collect high-quality tech discussions for summaries, newsletters and datasets. - **Newsletter writers**: a daily digest of the top stories with **Minimum points** and **Posted within 1 day**. ### How to use it 1. Choose **What to get**: search, front page or Who is hiring. 2. For search, type a **query** (e.g. `web scraping`), pick item types, and optionally set minimum points or a date range. 3. For jobs, optionally add **job keywords** (e.g. `python`, `remote`) and how many months of threads to read. 4. Click **Start** and export to Excel, CSV or JSON. ### Input examples ```json { "mode": "search", "query": "web scraping", "itemTypes": ["story"], "sortBy": "relevance", "minPoints": 50, "maxItems": 100 } ``` ```json { "mode": "whoIsHiring", "jobKeywords": ["python"], "hiringMonths": 1, "maxItems": 500 } ``` ### Output examples A story: ```json { "id": "22180559", "type": "story", "title": "Congrats! Web scraping is legal! (US precedent)", "url": "https://parsers.me/us-court-fully-legalized-website-scraping", "hnUrl": "https://news.ycombinator.com/item?id=22180559", "points": 1057, "numComments": 388, "createdAt": "2020-01-29T14:06:37Z", "domain": "parsers.me" } ``` A *Who is hiring?* job post: ```json { "type": "job-post", "thread": "Ask HN: Who is hiring? (October 2026)", "company": "Sidekick Labs", "headerParts": ["Senior SWE, R&D Engineer, Intelligence Engineer", "REMOTE"], "isRemote": true, "visaSponsorship": false, "salaryText": "$170k to $230k", "hnUrl": "https://news.ycombinator.com/item?id=49922609", "text": "Sidekick Labs | Senior SWE, R&D Engineer, ..." } ``` ### How much does it cost? Pay-per-event: a small fee **per item returned** plus a tiny per-run start fee, with no subscription and no separate platform usage costs. For example, 1,000 items cost about $0.30. Set **Maximum cost per run** to cap your spending. ### Use it through the API, integrations or AI agents Run it from the Apify API, on a schedule, or connect it to Make, Zapier, n8n, Slack or Google Sheets. AI agents can call it through the Apify MCP server. ### FAQ **Is it legal?** It uses the official, public Hacker News search API. Usernames are off by default and emails are hidden, so no personal data is collected unless you turn usernames on. **How accurate are the job fields?** Company, remote, visa and salary are read from each post's text, so they're best-effort. The full text is always included. **Found a bug or need a feature?** Open an issue. We usually reply within 24 hours. ### More tools from this developer - [SAM.gov Contract Opportunities Finder & Bid Alerts](https://apify.com/pankajbhatia/sam-gov-contract-opportunities): federal contract opportunities and bid alerts - [Sitemap & URL Extractor](https://apify.com/pankajbhatia/sitemap-url-extractor): get every URL of a website from its sitemaps - [Website to Markdown for AI & RAG](https://apify.com/pankajbhatia/website-to-markdown): turn web pages into clean Markdown for LLMs - [Tech Stack Detector](https://apify.com/pankajbhatia/tech-stack-detector): find what any website is built with - [Apple App Store Reviews Scraper](https://apify.com/pankajbhatia/app-store-reviews): App Store reviews and ratings for any app - [ATS Jobs Scraper](https://apify.com/pankajbhatia/ats-jobs-scraper): jobs from Greenhouse, Lever, Ashby, Workable and more - [Website Screenshot](https://apify.com/pankajbhatia/website-screenshot): full-page, mobile and PDF screenshots in bulk - [PDF to Text & Markdown Extractor](https://apify.com/pankajbhatia/pdf-text-extractor): extract text from PDFs for AI and search - [Domain WHOIS, DNS & SSL Checker](https://apify.com/pankajbhatia/domain-whois-dns-ssl-checker): bulk domain, DNS, email security and SSL checks - [SEO Meta Tags & On-Page Audit](https://apify.com/pankajbhatia/seo-meta-tags-checker): bulk on-page SEO audit with fixes - [Bulk URL Status & Broken Link Checker](https://apify.com/pankajbhatia/url-status-broken-link-checker): status codes, redirect chains and broken links - [RSS & Atom Feed Reader](https://apify.com/pankajbhatia/rss-feed-reader): merge and filter many news feeds - [Academic Paper Search](https://apify.com/pankajbhatia/academic-paper-search): papers from Crossref, arXiv and PubMed - [SEC EDGAR Filings & Financials](https://apify.com/pankajbhatia/sec-edgar-filings-financials): 10-K, 10-Q, 8-K filings and company financials # Actor input Schema ## `mode` (type: `string`): Search stories and comments, get the current front page, or collect job posts from the monthly 'Ask HN: Who is hiring?' threads. ## `query` (type: `string`): Keywords to search for (search mode). Leave empty to get the latest items. ## `itemTypes` (type: `array`): Which kinds of items to include in search mode. ## `sortBy` (type: `string`): Newest first, or most relevant/popular first. ## `minPoints` (type: `integer`): Only stories with at least this many upvotes. 0 = no limit. ## `minComments` (type: `integer`): Only stories with at least this many comments. 0 = no limit. ## `postedWithinDays` (type: `number`): 0 = any date. ## `jobKeywords` (type: `array`): Only keep job posts containing any of these words, e.g. python, remote, react. ## `hiringMonths` (type: `integer`): How many recent monthly threads to read (1 = current month). ## `maxItems` (type: `integer`): Maximum number of items to return. ## `includeUsernames` (type: `boolean`): Add the poster's Hacker News username. Off by default for privacy. ## `redactEmails` (type: `boolean`): Replace email addresses in posts and comments with \[email hidden]. ## Actor input object example ```json { "mode": "search", "query": "web scraping", "itemTypes": [ "story" ], "sortBy": "relevance", "minPoints": 0, "minComments": 0, "postedWithinDays": 0, "hiringMonths": 1, "maxItems": 100, "includeUsernames": false, "redactEmails": true } ``` # Actor output Schema ## `results` (type: `string`): Stories, comments or job posts from Hacker News. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "web scraping" }; // Run the Actor and wait for it to finish const run = await client.actor("pankajbhatia/hacker-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "web scraping" } # Run the Actor and wait for it to finish run = client.actor("pankajbhatia/hacker-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "web scraping" }' | apify call pankajbhatia/hacker-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,pankajbhatia/hacker-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/k5NbLDJpXvjkKf5PQ/builds/gNgKcU2qASHM7OdGr/openapi.json