Hacker News Scraper 🚀

Fast, rate-limit resilient Hacker News scraper extracting trending stories, Ask HN, Show HN, top comments, points, and author metadata without browser overhead.

⚡ Try it in 1-Click: Click the Run or Try for Free button on the right to test this Actor immediately with pre-configured parameters. No technical setup, passwords, or API keys required! 🎁 Free Starter Allowance: Test runs cost less than a fraction of a cent ($0.0001 container fee) so you can run test queries risk-free with your monthly Apify starter credits.

---

⚡ 1-Click API Quick-Start (cURL / Python / Dify / FastGPT / Coze)

Run directly via HTTP API without opening the browser form (支持直接通过 API 调用，无缝接入 Dify、Coze、FastGPT 与自动化脚本):

# 1-Click cURL Instant Run (Replace YOUR_APIFY_TOKEN)
curl -X POST "https://api.apify.com/v2/acts/trusted_tech_spot~hacker-news-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'

---

🌟 Why Choose This Actor?

📊 No-Code Instant Export: Download your clean extracted data directly into Excel (.xlsx), CSV, JSON, or sync directly to Google Sheets with 1 click right from the Apify Console dataset tab. No coding required!

• ⚡ High-Speed & Resilient Extraction: Dual engine supporting both direct live scraping and Algolia HN search indices with automatic zero-failure fallback.
• 💰 Predictable Pay-Per-Result Pricing: Only pay for the actual records extracted ($0.0015/item). Never pay for actor startup overhead or container idle seconds.
• 🧠 AI & RAG Ready: Seamlessly feeds vector databases (Pinecone, Qdrant, ChromaDB, Weaviate) and LLM agents with pre-calculated OpenAI token counts and semantic chunking.
• 🎯 Native Keyword Filtering: Target exact records automatically with includeKeywords and excludeKeywords without client-side post-processing.
• 📊 Instant Multi-Format Export: One-click download into JSON, CSV, Excel, XML, or HTML Table directly from Apify Console or API.

---

🚀 How to Run in 3 Easy Steps

1. Open the Actor: Click the Run button on the top right of this page.
2. Review or Adjust Inputs: The form is organized into clean visual sections (Primary Settings, Filtering, and AI Vector Options). Run it with default values or customize to your target.
3. Download Results: View your extracted dataset in real time and export it to Excel, CSV, or JSON.

---

📥 Input Parameters

All parameters come pre-filled with sensible defaults so you can run the Actor right away:

Parameter	Type	Required	Default / Prefill	Description
Hacker News Section (section)	string	No	"top"	Choose which section to scrape: top, new, ask, show, jobs
Max Results Limit (maxResults)	integer	No	30	Maximum number of Hacker News stories to return (1-100)
Keyword Search Query (searchQuery)	string	No	""	Search stories by keyword via Algolia HN index (e.g. "AI agent", "LLM", "Rust")
Include Keywords (Filter) (includeKeywords)	string	No	""	Filter results to only include items containing these comma-separated keywords
Exclude Keywords (Filter) (excludeKeywords)	string	No	""	Filter out results containing these comma-separated keywords
Enable AI / RAG Token Chunking (enableRagEnrichment)	boolean	No	true	Automatically compute OpenAI token counts and generate semantic vector chunks for RAG pipelines
Preferred Output Format (outputFormat)	string	No	"JSON"	Format results for preferred downstream consumer (e.g. JSON, Markdown, Flat)

---

📤 Sample Output Data

Data is saved directly to the default dataset. Below is a realistic example of the structured fields returned:

{
  "section": "top",
  "totalCount": 30,
  "stories": [
    {
      "id": "41738291",
      "title": "Anthropic reported diary entry to police, woman faces felony charge",
      "url": "https://www.techspot.com/news/114091-florida-woman-used-claude-diary-anthropic-reported-shoot.html",
      "hnUrl": "https://news.ycombinator.com/item?id=41738291",
      "points": 479,
      "author": "pseudonym",
      "commentsCount": 312,
      "createdAt": "2026-10-06T00:15:20.000Z"
    }
  ],
  "scrapedAt": "2026-10-06T02:30:00.000Z",
  "_ragMetadata": {
    "estimatedTokens": 450,
    "chunks": ["..."]
  }
}

---

💻 Developer API & AI Framework Integration

Automate runs and ingest clean data directly into your applications:

Node.js (apify-client)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_APIFY_TOKEN',
});

// Run actor with pre-configured parameters
const run = await client.actor('trusted_tech_spot/hacker-news-scraper').call({ section: 'top', maxResults: 25, searchQuery: 'open source AI' });

// Fetch structured items from the dataset
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Extracted ${items.length} records:`, items);

Python (apify-client)

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

# Run actor with default parameters
run = client.actor("trusted_tech_spot/hacker-news-scraper").call(run_input={"section": "top", "maxResults": 25, "searchQuery": "open source AI"})

# Fetch dataset items
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(f"Extracted {len(items)} records:", items)

LangChain Vector Ingestion (AI & RAG)

from apify_client import ApifyClient
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_core.documents import Document

# 1. Fetch pre-chunked dataset from Apify
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("trusted_tech_spot/hacker-news-scraper").call(run_input={"section": "top", "maxResults": 25, "searchQuery": "open source AI"})
items = client.dataset(run["defaultDatasetId"]).list_items().items

# 2. Extract semantic chunks directly into LangChain documents
docs = []
for item in items:
    chunks = item.get("_ragMetadata", {}).get("chunks", [str(item)])
    for chunk in chunks:
        docs.append(Document(page_content=chunk, metadata={"source": "hacker-news-scraper"}))

# 3. Ingest into ChromaDB / Pinecone vector store
vector_store = Chroma.from_documents(docs, OpenAIEmbeddings())
print(f"Successfully indexed {len(docs)} chunks into Vector DB!")

LlamaIndex Knowledge Ingestion

from apify_client import ApifyClient
from llama_index.core import VectorStoreIndex, Document

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("trusted_tech_spot/hacker-news-scraper").call(run_input={"section": "top", "maxResults": 25, "searchQuery": "open source AI"})
items = client.dataset(run["defaultDatasetId"]).list_items().items

# Wrap clean extracted text into LlamaIndex documents
documents = [Document(text=str(item)) for item in items]
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("Summarize key findings from this extracted data")
print(response)

---

🔗 Related Automations (Flagship Niche Scrapers & High-Speed Intelligence Fleet)

Explore our specialized high-speed micro-scrapers and cross-border intelligence tools from the trusted_tech_spot fleet:

• Google Maps Dentists & Dental Clinics Leads Scraper (google-maps-dentists-leads-scraper) — Extract dental clinics, orthodontists, phone numbers, and websites from Google Maps.
• Google Maps Real Estate Agents & Realtors Leads Scraper (google-maps-realtor-leads-scraper) — Extract real estate agents, property brokers, phones, and agency websites from Google Maps.
• Shopify Store & E-Commerce Tech Leads Detector (shopify-store-leads-detector) — Detect Shopify stores, installed apps, themes, and e-commerce merchant contact leads.
• Amazon FBA Bestsellers & Arbitrage Price Tracker (amazon-fba-bestsellers-arbitrage-scraper) — Track Amazon FBA bestseller ranks, buy-box arbitrage spreads, and high-margin ASINs.
• AliExpress Products & Suppliers Scraper 速卖通商品采集 (aliexpress-products-suppliers-scraper) — Extract AliExpress dropshipping products, supplier ratings, order volume, and pricing.
• Xiaohongshu (REDnote) Posts Scraper 小红书笔记与爆款采集 (xiaohongshu-rednote-posts-trends-scraper) — Extract Xiaohongshu (REDnote) viral notes, KOL engagement metrics, and trending lifestyle tags.
• TikTok Trending Hashtags & Viral Sounds Scraper (tiktok-trending-hashtags-sounds-scraper) — Scrape viral TikTok hashtags, trending sounds, creator growth metrics, and video views.
• Remote Tech Jobs & Compensation Salary Scraper (remote-tech-jobs-salary-scraper) — Extract high-paying remote developer, AI engineer, and DevOps jobs with verified salary bands.

---

💬 Technical Support & Custom Demands

Need custom extraction fields, high-volume dedicated scraping, enterprise data feeds, or encountered an edge-case?

• 🐛 Open an Issue: Open a ticket directly on the Actor Issues Tab for 24-hour engineer response.
• ✉️ Direct Email Support: Reach out directly to contact@trustedtechspot.com for custom scraping jobs, private proxy setups, or SLA contracts.
• 💡 Feature Requests: Request new export columns, filtering rules, or platform endpoints anytime.

---

🔒 Privacy & Enterprise Compliance

• No Passwords or Cookies Required: Safely executes without requesting your credentials, private tokens, or session cookies.
• Enterprise Grade Sanitization: Automatically scrubs sensitive headers and tracking parameters before saving data.