Hacker News Search Scraper

Search Hacker News stories, comments, Ask HN posts, Show HN launches, polls, and author activity. Export clean, structured HN search results for market research, product monitoring, recruiting, developer relations, and trend analysis.

What does Hacker News Search Scraper do?

This actor finds public Hacker News content that matches your query, tags, author, or date range.

It returns normalized dataset rows with titles, links, authors, timestamps, points, comment counts, story IDs, parent IDs, cleaned comment text, and tags.

Use it when you need repeatable Hacker News monitoring without copying results by hand.

Who is it for?

• 🚀 Founders tracking launch feedback and competitor mentions.
• 📈 Investors watching market signals and technical trends.
• 🧑‍💻 Developer relations teams monitoring product discussions.
• 🧲 Recruiters finding active technical communities and hiring threads.
• 📰 Content teams researching developer pain points.
• 🔍 Analysts collecting public technology discussion data.

Why use it?

Hacker News discussions are valuable but hard to export consistently.

This scraper gives you:

• Keyword search across stories and comments.
• Tag filters for stories, comments, Ask HN, Show HN, polls, and authors.
• Date filtering for recent or historical monitoring.
• Clean URLs for both the external page and Hacker News item.
• Structured JSON, CSV, Excel, XML, and API access through Apify datasets.

Common use cases

• Track mentions of your startup or product.
• Find competitor discussions and launch reactions.
• Monitor developer sentiment around a technology.
• Build a list of recent Show HN launches in a niche.
• Export Ask HN discussions for content research.
• Collect Who is Hiring style author activity by tag or user.
• Watch security, AI, infrastructure, or open-source trend keywords.

What data can you extract?

Field	Description
objectId	Hacker News / search object identifier.
type	Story, comment, poll, job, or unknown.
title	Story title or parent story title.
url	External URL when available.
hnUrl	Direct Hacker News item URL.
author	Hacker News username.
createdAt	Creation timestamp.
points	Story points when available.
numComments	Story comment count when available.
storyId	Parent story identifier.
parentId	Parent item identifier.
commentText	Cleaned comment body for comments.
highlight	Search match context.
tags	Hacker News tags attached to the item.

How much does it cost to scrape Hacker News search results?

The actor uses pay-per-event pricing.

• A small one-time start event is charged per run.
• A result event is charged for each Hacker News item saved.
• You control cost with maxItems.
• The default prefill is intentionally small for a cheap first run.

For example, set maxItems to 25 for a quick sample or 500+ for broader monitoring.

Input options

Input	Description
query	Keyword, company, product, domain, or topic.
tags	HN tags such as story, comment, show_hn, ask_hn, poll, or author_pg.
includeComments	Adds matching comments to the export.
author	Convenience filter for a Hacker News username.
dateFrom	Earliest creation date in YYYY-MM-DD format.
dateTo	Latest creation date in YYYY-MM-DD format.
sortBy	Relevance, date, or points.
maxItems	Maximum number of results to save.

Example input

{
  "query": "apify",
  "tags": ["story"],
  "includeComments": false,
  "sortBy": "relevance",
  "maxItems": 25
}

Search stories

To collect story results for a company or product:

1. Enter your keyword in query.
2. Set tags to ["story"].
3. Choose sortBy as relevance or date.
4. Set maxItems.
5. Run the actor and export the dataset.

Search comments

To inspect user discussion and sentiment:

1. Enter your keyword.
2. Use tags: ["comment"] or enable includeComments.
3. Keep maxItems moderate for review workflows.
4. Export commentText, author, createdAt, and hnUrl.

Search Ask HN and Show HN

Use tags to focus on specific HN communities:

• ask_hn for question threads.
• show_hn for launches and demos.
• poll for poll posts.
• story for general submitted stories.
• comment for comments.

Search by author

Use the author input or a tag such as author_pg.

This is useful for:

• Tracking activity by known founders or maintainers.
• Reviewing public posts from a company team member.
• Finding technical communities around an expert user.

Date range filtering

Use dateFrom and dateTo to narrow exports.

Examples:

• dateFrom: 2026-01-01 for current-year monitoring.
• dateFrom: 2025-06-01, dateTo: 2025-06-30 for a monthly report.
• Leave both empty to search all available public history.

Sorting tips

• relevance is best for keyword research.
• date is best for monitoring new discussions.
• points is useful when you want higher-scoring stories first.

Output example

{
  "objectId": "123456",
  "type": "story",
  "title": "Example product launch",
  "url": "https://example.com",
  "hnUrl": "https://news.ycombinator.com/item?id=123456",
  "author": "example_user",
  "createdAt": "2026-06-01T12:00:00Z",
  "points": 42,
  "numComments": 17,
  "storyId": 123456,
  "parentId": null,
  "commentText": null,
  "highlight": "Example product launch",
  "tags": ["story"]
}

Integrations

Use the dataset in:

• Google Sheets for lightweight research workflows.
• Airtable for content and lead databases.
• Notion for research libraries.
• Slack alerts through Apify integrations.
• BI dashboards for trend reporting.
• CRM enrichment when HN discussions mention target accounts.

API usage with Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/hacker-news-search-scraper').call({
  query: 'open source database',
  tags: ['story'],
  sortBy: 'date',
  maxItems: 50
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

from apify_client import ApifyClient

client = ApifyClient('APIFY_TOKEN')
run = client.actor('fetch_cat/hacker-news-search-scraper').call(run_input={
    'query': 'developer tools',
    'tags': ['story'],
    'maxItems': 50,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items[:3])

API usage with cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~hacker-news-search-scraper/runs?token=APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"query":"apify","tags":["story"],"maxItems":25}'

MCP usage

Use this actor from AI tools through Apify MCP.

MCP URL:

https://mcp.apify.com/?tools=fetch_cat/hacker-news-search-scraper

Claude Code setup:

$claude mcp add apify-hacker-news "https://mcp.apify.com/?tools=fetch_cat/hacker-news-search-scraper"

MCP JSON configuration example:

{
  "mcpServers": {
    "apify-hacker-news": {
      "url": "https://mcp.apify.com/?tools=fetch_cat/hacker-news-search-scraper"
    }
  }
}

Example prompts:

• "Search Hacker News for recent discussions about Apify and summarize the main themes."
• "Find Show HN posts mentioning vector databases and extract titles, authors, and links."
• "Collect comments about browser automation tools from the last year."

How to get better results

• Use specific product names rather than broad categories.
• Add date filters for monitoring reports.
• Use comment results when you need sentiment or pain points.
• Use show_hn for launch discovery.
• Use ask_hn for problem and recommendation research.
• Increase maxItems gradually after validating a sample.

Limits and notes

• Only public Hacker News search data is returned.
• Deleted or unavailable items may have fewer fields.
• Some comment results have a parent story title instead of their own title.
• External URLs are not present for every HN item.
• Very broad queries may return many historical results; use maxItems and dates to control scope.

Legality

This actor collects public Hacker News search results. Use the data responsibly, respect applicable laws, and avoid using exported data for spam, harassment, or unwanted outreach. If your workflow involves personal data, make sure you have a lawful basis and follow your local privacy requirements.

Troubleshooting

Why did I get no results?

Try removing restrictive tags or widening the date range. A query with tags, author, and narrow dates can be too specific.

Why are points empty on comments?

Hacker News points are generally attached to stories. Comment rows may not include points.

Why is url empty?

Some HN posts are text posts or comments and do not have an external URL. Use hnUrl to open the HN item.

Related scrapers

• Google News Scraper for broader news monitoring.
• Google Autocomplete Scraper for keyword discovery.
• Y Combinator Companies Scraper for startup/company research.
• Explore other data extraction actors from fetch_cat on Apify for web, search, social, and marketplace research workflows.

FAQ

Can I search only by tag?

Yes. Leave query empty and provide tags such as show_hn or ask_hn.

Can I export comments?

Yes. Use tags: ["comment"] or enable includeComments.

Can I track a username?

Yes. Fill author with the HN username or add a tag like author_pg.

Can I schedule this actor?

Yes. Use Apify schedules to run daily, weekly, or monthly monitoring jobs.

Can I export to CSV?

Yes. Apify datasets support CSV, JSON, Excel, XML, RSS, and API access.

Does this require my Hacker News account?

No. The actor only uses public search data.