# Hacker News Who Is Hiring Scraper (`solidcode/hackernews-scraper`) Actor \[💰 $1.8 / 1K] Turn Hacker News 'Ask HN: Who is hiring?' threads into structured job data. Scrape recent months or specific threads, search by keyword, filter by remote, and get company, role, location, salary, tech stack, and contact per posting. - **URL**: https://apify.com/solidcode/hackernews-scraper.md - **Developed by:** [SolidCode](https://apify.com/solidcode) (community) - **Categories:** Jobs, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.80 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hacker News "Who Is Hiring" Scraper Turn Hacker News "Ask HN: Who is hiring?" monthly threads into a clean spreadsheet of job postings — company, role, location, salary, remote status, tech stack, apply links, and contact emails, one row per posting. Every month's thread holds hundreds of comments in free-form text; this actor reads them for you and parses each into structured fields. Built for recruiters, job seekers, and market researchers who need structured Hacker News hiring data without scrolling through hundreds of comments by hand. ### Why This Scraper? - **Structured extraction of 9 fields from free-text postings** — company, role, location, salary, remote status, tech stack, visa sponsorship, apply URL, and contact email are parsed out of each comment automatically, no manual reading. - **60-technology stack detection** — every posting is scanned against a maintained dictionary of 60+ languages, frameworks, databases, and clouds (Python, Rust, React, Kubernetes, PostgreSQL, AWS, and more) and returned as a clean tag array. - **Remote / Hybrid / Onsite classification** — each posting is labeled, and a single "Remote jobs only" toggle keeps just the remote, hybrid, distributed, and work-from-home roles. - **Full-text keyword search across all of Hacker News** — search every comment ever posted (not only the hiring threads) for a stack, company, or phrase like "Rust remote". - **Up to 24 months of hiring history in one run** — pull the last two years of "Who is hiring?" threads in a single pass for hiring-volume analysis and salary trends where postings disclose pay. - **Target any thread by link or ID** — paste a full `news.ycombinator.com/item?id=` link or just the number; works for "Who wants to be hired?" and freelancer threads too. - **Include / exclude keyword filters** — keep only postings mentioning your terms ("senior", "San Francisco") and drop the ones you don't want ("onsite only", "unpaid"), case-insensitive. - **The original `fullText` is always saved** — even when a posting breaks convention and a structured field can't be parsed, the complete posting text is never lost. - **Deduplicated rows** — a posting that appears in both a thread and your keyword search is saved once, never twice. ### Use Cases **Recruiting & Sourcing** - Build a live feed of companies actively hiring this month - Source candidate-facing competitors by role and location - Spot which companies post every month versus one-off hirers - Compile apply links and contact emails into a single outreach list **Job Seeking** - Filter hundreds of postings down to your stack in seconds - Surface only remote or hybrid roles across every company - Track a target company's hiring across consecutive months - Find roles that mention visa sponsorship or relocation **Market & Salary Research** - Pull two years of threads to chart salary ranges where posters disclose them - Measure how often a technology appears month over month - Compare remote-vs-onsite share across the developer market - Map hiring activity by city and region **Competitive & Trend Intelligence** - Watch which startups and scale-ups are expanding headcount - Track adoption curves for languages and frameworks by mention count - Benchmark posted compensation against the field, for the postings that state it - Feed structured hiring signals into dashboards and reports ### Getting Started #### Scrape the Latest Month Leave everything blank to pull the most recent "Who is hiring?" thread: ```json { "monthsBack": 1 } ```` #### Scrape a Specific Thread Paste any thread link or bare ID: ```json { "threadUrls": ["https://news.ycombinator.com/item?id=40563283"] } ``` #### Keyword Search, Remote Only, Filtered Search the last six months, keep only remote roles that mention Rust, and drop anything that says onsite: ```json { "monthsBack": 6, "searchQuery": "Rust remote", "remoteOnly": true, "includeKeywords": ["Rust"], "excludeKeywords": ["onsite only"], "maxResults": 500 } ``` ### Input Reference #### What to Scrape | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `monthsBack` | integer | `1` | How many of the most recent monthly "Who is hiring?" threads to include, counting back from the latest. Leave blank to scrape only specific links or a search below; leave everything blank for the latest month. Max 24. | | `threadUrls` | string\[] | — | Specific Hacker News threads to scrape. Paste full `item?id=` links or just the number. Works for any thread, including "Who wants to be hired?" and freelancer threads. | | `searchQuery` | string | — | Optionally search across all of Hacker News for a word or phrase (e.g. "machine learning"), not just the hiring threads. Leave empty to skip. | #### Filters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `includeKeywords` | string\[] | `[]` | Keep only postings that mention at least one of these words or phrases. Case-insensitive. Leave empty to keep everything. | | `excludeKeywords` | string\[] | `[]` | Drop postings that mention any of these words or phrases. Case-insensitive. Leave empty to exclude nothing. | | `remoteOnly` | boolean | `false` | Keep only postings that mention remote, hybrid, or distributed work. | #### Limits | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `maxResults` | integer | `1000` | The most job postings to save across all selected threads and searches. Set to 0 for no limit. A limit keeps runs fast and predictable. | ### Output Each job posting becomes one flat row. Structured fields are filled in on a best-effort basis from the free-form posting text, so some may be empty when a poster doesn't follow the usual format — but `fullText` always carries the complete posting. ```json { "commentId": "40565123", "threadId": "40563283", "threadTitle": "Ask HN: Who is hiring? (June 2026)", "threadMonth": "June 2026", "author": "janedoe", "company": "Adacore", "role": "Senior Backend Engineer", "location": "San Francisco", "remote": "Remote", "salary": "$150k-$200k", "techStack": ["Python", "PostgreSQL", "AWS", "Kubernetes"], "visa": "Visa sponsorship available", "applyUrl": "https://adacore.com/careers", "email": "jobs@adacore.com", "fullText": "Adacore | Senior Backend Engineer | San Francisco | Remote | $150k-$200k\nWe're hiring engineers to build...", "postedAt": "2026-06-02T14:11:03Z", "hnUrl": "https://news.ycombinator.com/item?id=40565123", "scrapedAt": "2026-06-17T09:30:00Z" } ``` #### Posting & Thread | Field | Type | Description | |-------|------|-------------| | `commentId` | string | Hacker News ID of this job posting | | `threadId` | string | ID of the hiring thread the posting belongs to | | `threadTitle` | string | Full thread title, e.g. "Ask HN: Who is hiring? (June 2026)" | | `threadMonth` | string | Month and year parsed from the title, e.g. "June 2026" | | `author` | string | Hacker News username that posted the job | | `fullText` | string | Complete plain-text job posting — always present | | `hnUrl` | string | Direct link to the posting on Hacker News | | `postedAt` | string | When the posting was published (ISO 8601) | | `scrapedAt` | string | When this run collected the row (ISO 8601) | #### Parsed Job Details Best-effort fields parsed from the posting text. Empty when the poster's format doesn't expose them. | Field | Type | Description | |-------|------|-------------| | `company` | string | Company name from the posting | | `role` | string | Job title / role from the posting | | `location` | string | Location(s) detected in the posting | | `remote` | string | Work arrangement: "Remote", "Hybrid", or "Onsite" | | `salary` | string | Compensation text, captured verbatim | | `techStack` | string\[] | Technologies detected from the 60-entry dictionary | | `visa` | string | Visa sponsorship or relocation note if mentioned | | `applyUrl` | string | First application or company link in the posting | | `email` | string | Contact email if one appears | ### Tips for Best Results - **Leave it blank for a quick test** — running with no input pulls the latest month so you can see the data shape before scaling up. - **Combine search with include filters** — pair `searchQuery` with `includeKeywords` to laser-target a stack: search "remote" broadly, then keep only postings that also mention "Go" or "Elixir". - **Raise `monthsBack` for trend analysis** — set it to 12 or 24 to track technology mentions and (where posters state them) salary ranges over a long window in one run; this is the fastest path to historical hiring data. - **Salary, visa, and email are captured only when the poster states them** — Hacker News postings are free-form, so these fields are filled in where the text spells them out and left empty otherwise; `fullText` always holds the complete posting for anything the parser doesn't pick up. - **Use the "Remote jobs only" toggle, not a keyword** — it catches remote, hybrid, distributed, and work-from-home phrasing that a single keyword would miss. - **Lean on `fullText` for the unusual postings** — when a structured field is empty, the original text is still there, so you never lose a posting that simply broke the usual format. - **Set a `maxResults` cap on broad runs** — a single busy thread can hold 300+ postings; a cap keeps a multi-month run fast and the result count predictable. - **Paste links for non-hiring threads** — `threadUrls` also works on "Who wants to be hired?" and freelancer threads, so the same parsing gives you candidate-side data too. ### Pricing **From $1.80 per 1,000 results** — pay only for the job postings you collect. No compute or time-based charges — you pay per result, plus a small fixed per-run start fee. Bronze, Silver, and Gold subscribers pay progressively less; the table below shows total cost at each discount tier. | Results | No discount | Bronze | Silver | Gold | |--------:|------------:|-------:|-------:|-----:| | 100 | $0.22 | $0.21 | $0.20 | $0.18 | | 1,000 | $2.15 | $2.05 | $1.95 | $1.80 | | 10,000 | $21.50 | $20.50 | $19.50 | $18.00 | | 100,000 | $215.00 | $205.00 | $195.00 | $180.00 | A "result" is one job posting saved to your dataset. Platform fees (compute, storage) are additional and depend on your Apify plan. ### Integrations Export data in JSON, CSV, Excel, XML, or RSS. Connect to 1,500+ apps via: - **Zapier** / **Make** / **n8n** — Workflow automation - **Google Sheets** — Direct spreadsheet export - **Slack** / **Email** — Notifications on new results - **Webhooks** — Trigger custom APIs on run completion - **Apify API** — Full programmatic access ### Legal & Ethical Use This actor is designed for legitimate recruiting research, job searching, and labor-market analysis using publicly visible Hacker News posts. You are responsible for complying with applicable laws and Hacker News' terms of use. Do not use collected contact details for spam, harassment, or any unlawful purpose, and handle any personal data such as emails responsibly. # Actor input Schema ## `monthsBack` (type: `integer`): How many of the most recent monthly 'Who is hiring?' threads to include, counting back from the latest. For example, 1 scrapes only the current month, 6 covers the last half-year. Leave this blank (or 0) to scrape ONLY specific thread links or a keyword search below — when you provide those sources, no recent months are pulled unless you set this. If you leave everything blank, the latest month is scraped by default. ## `threadUrls` (type: `array`): Scrape specific Hacker News threads. Paste full links like 'https://news.ycombinator.com/item?id=40563283' or just the number at the end ('40563283'). Works for any HN thread, including 'Who wants to be hired?' and freelancer threads. ## `searchQuery` (type: `string`): Optionally search across all of Hacker News for a word or phrase, not just the hiring threads. For example 'Rust remote' or 'machine learning'. Full-text keyword search returns up to 1,000 of the most recent matching postings. Leave empty to skip search. ## `includeKeywords` (type: `array`): Keep only job postings that mention at least one of these words or phrases. Matching is case-insensitive. For example 'Python', 'San Francisco', 'senior'. Leave empty to keep everything. ## `excludeKeywords` (type: `array`): Drop job postings that mention any of these words or phrases. Matching is case-insensitive. For example 'onsite only', 'blockchain', 'unpaid'. Leave empty to exclude nothing. ## `remoteOnly` (type: `boolean`): Keep only postings that mention remote, hybrid, or distributed work. ## `maxResults` (type: `integer`): The most job postings to save across all selected threads and searches. Set to 0 for no limit. A limit is recommended to keep runs fast and predictable. ## Actor input object example ```json { "monthsBack": 1, "threadUrls": [ "https://news.ycombinator.com/item?id=40563283" ], "includeKeywords": [], "excludeKeywords": [], "remoteOnly": false, "maxResults": 1000 } ``` # Actor output Schema ## `overview` (type: `string`): Table of job postings with company, role, location, remote status, salary, tech stack, and links. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "monthsBack": 1, "threadUrls": [ "https://news.ycombinator.com/item?id=40563283" ], "searchQuery": "", "includeKeywords": [], "excludeKeywords": [], "remoteOnly": false, "maxResults": 1000 }; // Run the Actor and wait for it to finish const run = await client.actor("solidcode/hackernews-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "monthsBack": 1, "threadUrls": ["https://news.ycombinator.com/item?id=40563283"], "searchQuery": "", "includeKeywords": [], "excludeKeywords": [], "remoteOnly": False, "maxResults": 1000, } # Run the Actor and wait for it to finish run = client.actor("solidcode/hackernews-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "monthsBack": 1, "threadUrls": [ "https://news.ycombinator.com/item?id=40563283" ], "searchQuery": "", "includeKeywords": [], "excludeKeywords": [], "remoteOnly": false, "maxResults": 1000 }' | apify call solidcode/hackernews-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=solidcode/hackernews-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hacker News Who Is Hiring Scraper", "description": "[💰 $1.8 / 1K] Turn Hacker News 'Ask HN: Who is hiring?' threads into structured job data. Scrape recent months or specific threads, search by keyword, filter by remote, and get company, role, location, salary, tech stack, and contact per posting.", "version": "1.0", "x-build-id": "ERRdvqp8mgjOGfgMc" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/solidcode~hackernews-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-solidcode-hackernews-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/solidcode~hackernews-scraper/runs": { "post": { "operationId": "runs-sync-solidcode-hackernews-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/solidcode~hackernews-scraper/run-sync": { "post": { "operationId": "run-sync-solidcode-hackernews-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "monthsBack": { "title": "Recent months to scrape", "minimum": 0, "maximum": 24, "type": "integer", "description": "How many of the most recent monthly 'Who is hiring?' threads to include, counting back from the latest. For example, 1 scrapes only the current month, 6 covers the last half-year. Leave this blank (or 0) to scrape ONLY specific thread links or a keyword search below — when you provide those sources, no recent months are pulled unless you set this. If you leave everything blank, the latest month is scraped by default." }, "threadUrls": { "title": "Specific thread links", "type": "array", "description": "Scrape specific Hacker News threads. Paste full links like 'https://news.ycombinator.com/item?id=40563283' or just the number at the end ('40563283'). Works for any HN thread, including 'Who wants to be hired?' and freelancer threads.", "items": { "type": "string" } }, "searchQuery": { "title": "Keyword search (optional)", "type": "string", "description": "Optionally search across all of Hacker News for a word or phrase, not just the hiring threads. For example 'Rust remote' or 'machine learning'. Full-text keyword search returns up to 1,000 of the most recent matching postings. Leave empty to skip search." }, "includeKeywords": { "title": "Must contain (any of)", "type": "array", "description": "Keep only job postings that mention at least one of these words or phrases. Matching is case-insensitive. For example 'Python', 'San Francisco', 'senior'. Leave empty to keep everything.", "items": { "type": "string" } }, "excludeKeywords": { "title": "Must not contain (any of)", "type": "array", "description": "Drop job postings that mention any of these words or phrases. Matching is case-insensitive. For example 'onsite only', 'blockchain', 'unpaid'. Leave empty to exclude nothing.", "items": { "type": "string" } }, "remoteOnly": { "title": "Remote jobs only", "type": "boolean", "description": "Keep only postings that mention remote, hybrid, or distributed work.", "default": false }, "maxResults": { "title": "Maximum job postings", "minimum": 0, "type": "integer", "description": "The most job postings to save across all selected threads and searches. Set to 0 for no limit. A limit is recommended to keep runs fast and predictable.", "default": 1000 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```